简介
在本文中，我们将探索如何使用Langgraph框架创建一个层次化的Research Team。这个系统整合了搜索和Web爬取功能，并通过一个监督代理来管理任务的分配，实现自动化的信息检索和处理。
1. 系统架构概述
这个系统由以下三个主要组件组成：
- 搜索代理：使用搜索工具查找相关网页。
- Web爬取代理：从搜索代理提供的URL中提取详细内容。
- 监督代理：负责管理任务流，决定下一个执行的代理。
这些代理通过结构化输出进行通信，并使用Langgraph的状态图来管理任务的执行顺序。
2. 设置搜索工具
首先，我们需要设置一个可以执行网页搜索的工具。在此，我们使用TavilySearchResults，它允许我们进行URL查询并返回搜索结果。以下是使用此工具的代码：
from typing import Annotated, List
import os
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_core.tools import tool
os.environ["TAVILY_API_KEY"] = "your api key"
tavily_tool = TavilySearchResults(max_results=5)
@tool
def scrape_webpages(urls: List[str]) -> str:
"""
Scrape the provided web pages for detailed information.
Args:
urls (List[str]): A list of URLs (strings) to scrape. Each URL should point to a valid webpage
from which content will be extracted. These webpages should contain metadata
such as title and content that will be parsed and returned.
Returns:
str: A string representation of the documents scraped from the URLs, formatted in XML-like tags.
Each document will contain the title and content from the scraped webpage.
Example:
urls = ["http://example.com/page1", "http://example.com/page2"]
result = scrape_webpages(urls)
print(result) # Outputs the formatted scraped content from the provided URLs.
"""
loader = WebBaseLoader(urls)
docs = loader.load()
return "\n\n".join(
[
f'<Document name="{doc.metadata.get("title", "")}">\n{doc.page_content}\n</Document>'
for doc in docs
]
)
3. 监督代理节点
为了协调多个代理的工作，我们定义了一个监督代理，它负责决定哪个代理在何时执行。该代理会基于任务和已完成的步骤来决定下一个执行的代理。
from typing import List, Optional, Literal, TypedDict
from langchain_core.language_models.chat_models import BaseChatModel
from langgraph.graph import StateGraph, MessagesState, START, END
from langchain_core.messages import HumanMessage, trim_messages
from pydantic import BaseModel, Field
class AgentState(MessagesState):
task: Optional[str] =None
search_msg: Optional[str]=None
web_scraper_msg: Optional[str]=None
final_answer: Optional[str]=None
next: Optional[str]=None
def make_supervisor_node(llm: BaseChatModel, members: list[str]) -> str:
options = ["FINISH"] + members
system_prompt = (
"You are a supervisor tasked with managing a conversation between the"
f" following workers: {members}. Given the following user request,"
" respond with the worker to act next. Each worker will perform a"
" task and respond with their results and status. When finished,"
" respond with 'FINISH'."
)
from typing import Literal
from pydantic import BaseModel, Field
from typing import List
class Router(BaseModel):
"""
A model representing a router to determine which worker should act next.
If no workers are needed, the response will be 'FINISH'.
Attributes:
next (Literal): A value indicating the next worker to route to.
If no workers are required, the value will be 'FINISH'.
The options are dynamically generated from the list of members.
final_answer (Optional[str]): A final judgment based on the collected information.
If no final decision is made, this will be `None`.
Example:
If there are workers 'search' and 'web_scraper', the possible values for 'next' are:
'FINISH', 'search', and 'web_scraper'.
If the model can make a final decision based on the information gathered,
it will return that decision in `final_answer`, e.g.,
'The answer is: XYZ'.
"""
next: Literal[*options] = Field(
...,
description=f"Worker to route to next, {members}. If no workers are needed, respond 'FINISH'."
)
final_answer: Optional[str] = Field(
None,
description="The final judgment or decision based on the collected information. "
"If no final decision is needed, this will be None."
)
def supervisor_node(state: AgentState) -> AgentState:
"""An LLM-based router."""
messages = [
{"role": "system", "content": system_prompt}
] + [{"role": "user", "content": f"task: {state['task']} \n \n search: {state['search_msg']} \n \n web_scraper: {state['web_scraper_msg']}"}]
response = llm.with_structured_output(Router).invoke(messages)
if response.next is None:
print(response)
next_ = response.next
if "FINISH" in next_ :
next_ = END
print(response)
return {"next": next_,
"final_answer":response.final_answer,
"messages": [
HumanMessage(content=response.final_answer or "No answer provided", name="supervisor")
]
}
return supervisor_node
4. 搜索和Web爬取节点
我们还需要定义两个具体的工作节点：搜索节点和Web爬取节点，它们分别负责查询搜索结果并从中提取内容。
from langchain_core.messages import HumanMessage
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
llm = ChatOpenAI(
temperature=0,
model="GLM-4-plus",
openai_api_key="your api key",
openai_api_base="https://open.bigmodel.cn/api/paas/v4/"
)
def search_node(state: AgentState) -> AgentState:
search_system_prompt = f"You are a search agent, work with web_scraper agent to complete the task. you should privide the url link for web_scraper agent \n \n web_scraper: {state['web_scraper_msg']}"
search_agent = create_react_agent(llm, tools=[tavily_tool], state_modifier=search_system_prompt)
inputs = {"messages": [("user", f"task: {state['task']}")]}
result = search_agent.invoke(inputs)
return {
"search_msg": result["messages"][-1].content,
"messages": [
HumanMessage(content=result["messages"][-1].content, name="search")
]
}
def web_scraper_node(state: AgentState) -> AgentState:
web_scraper_system_prompt = "You are a web_scraper agent. work with search agent to complete the task. \n \n search: {state['search_msg']}. you can use the tool to get the information"
web_scraper_agent = create_react_agent(llm, tools=[scrape_webpages], state_modifier=web_scraper_system_prompt)
inputs = {"messages": [("user", f"task: {state['task']}")]}
result = web_scraper_agent.invoke(inputs)
return {
"web_scraper_msg": result["messages"][-1].content,
"messages": [
HumanMessage(content=result["messages"][-1].content, name="web_scraper")
]
}
research_supervisor_node = make_supervisor_node(llm, ["search", "web_scraper"])
5. 构建状态图
通过LangChain的状态图，我们可以管理任务流的顺序，确保代理按照正确的顺序执行。
research_builder = StateGraph(AgentState)
research_builder.add_node("supervisor", research_supervisor_node)
research_builder.add_node("search", search_node)
research_builder.add_node("web_scraper", web_scraper_node)
# 定义控制流
research_builder.add_edge(START, "supervisor")
research_builder.add_edge("search", "supervisor")
research_builder.add_edge("web_scraper", "supervisor")
research_graph = research_builder.compile()
6. 展示工作流图
我们可以使用Mermaid图表来可视化整个任务流。下面是展示任务流的代码：
from IPython.display import Image, display
display(Image(research_graph.get_graph().draw_mermaid_png()))
7. 执行任务流
最后，我们可以通过以下代码来执行整个任务流，查询AI代理的相关信息：
for s in research_graph.stream(
{"task": "what is ai agent", "search_msg": "", "web_scraper_msg": ""},
{"recursion_limit": 25}, stream_mode="values"
):
print(s)
print("---")
{'messages': [], 'task': 'what is ai agent', 'search_msg': '', 'web_scraper_msg': ''}
---
next='search' final_answer=None
{'messages': [HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='0541a836-8d1d-4514-8566-ebb42be29727')], 'task': 'what is ai agent', 'search_msg': '', 'web_scraper_msg': '', 'final_answer': None, 'next': 'search'}
---
{'messages': [HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='0541a836-8d1d-4514-8566-ebb42be29727'), HumanMessage(content='Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', additional_kwargs={}, response_metadata={}, name='search', id='8b2be555-b5f2-4e61-8cb2-3e5b6deef9cf')], 'task': 'what is ai agent', 'search_msg': 'Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', 'web_scraper_msg': '', 'final_answer': None, 'next': 'search'}
---
next='web_scraper' final_answer=None
{'messages': [HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='0541a836-8d1d-4514-8566-ebb42be29727'), HumanMessage(content='Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', additional_kwargs={}, response_metadata={}, name='search', id='8b2be555-b5f2-4e61-8cb2-3e5b6deef9cf'), HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='c0dfc718-02bc-4db0-8748-1eb06ae35b30')], 'task': 'what is ai agent', 'search_msg': 'Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', 'web_scraper_msg': '', 'final_answer': None, 'next': 'web_scraper'}
---
{'messages': [HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='0541a836-8d1d-4514-8566-ebb42be29727'), HumanMessage(content='Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', additional_kwargs={}, response_metadata={}, name='search', id='8b2be555-b5f2-4e61-8cb2-3e5b6deef9cf'), HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='c0dfc718-02bc-4db0-8748-1eb06ae35b30'), HumanMessage(content='AI agent is an artificial intelligence program that can interact with humans or other systems, and perform tasks based on the inputs it receives. It can be used for a wide range of applications, such as customer service, data analysis, and language translation.', additional_kwargs={}, response_metadata={}, name='web_scraper', id='d5c32bea-bafa-4334-b59b-5f503d044661')], 'task': 'what is ai agent', 'search_msg': 'Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', 'web_scraper_msg': 'AI agent is an artificial intelligence program that can interact with humans or other systems, and perform tasks based on the inputs it receives. It can be used for a wide range of applications, such as customer service, data analysis, and language translation.', 'final_answer': None, 'next': 'web_scraper'}
---
next='FINISH' final_answer='An AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It can interact with humans or other systems and is used in various applications like customer service, data analysis, and language translation. For more detailed information, you can refer to this link: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/).'
{'messages': [HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='0541a836-8d1d-4514-8566-ebb42be29727'), HumanMessage(content='Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', additional_kwargs={}, response_metadata={}, name='search', id='8b2be555-b5f2-4e61-8cb2-3e5b6deef9cf'), HumanMessage(content='No answer provided', additional_kwargs={}, response_metadata={}, name='supervisor', id='c0dfc718-02bc-4db0-8748-1eb06ae35b30'), HumanMessage(content='AI agent is an artificial intelligence program that can interact with humans or other systems, and perform tasks based on the inputs it receives. It can be used for a wide range of applications, such as customer service, data analysis, and language translation.', additional_kwargs={}, response_metadata={}, name='web_scraper', id='d5c32bea-bafa-4334-b59b-5f503d044661'), HumanMessage(content='An AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It can interact with humans or other systems and is used in various applications like customer service, data analysis, and language translation. For more detailed information, you can refer to this link: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/).', additional_kwargs={}, response_metadata={}, name='supervisor', id='6aca4c07-dd3d-445e-9c29-9d1e611f9041')], 'task': 'what is ai agent', 'search_msg': 'Here is a URL link that provides information about what an AI agent is: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/). This link explains that an AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It also discusses the advantages of AI agents in various sectors. You can use this link for further details.', 'web_scraper_msg': 'AI agent is an artificial intelligence program that can interact with humans or other systems, and perform tasks based on the inputs it receives. It can be used for a wide range of applications, such as customer service, data analysis, and language translation.', 'final_answer': 'An AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It can interact with humans or other systems and is used in various applications like customer service, data analysis, and language translation. For more detailed information, you can refer to this link: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/).', 'next': '__end__'}
---
s['final_answer']
'An AI agent is an autonomous system or entity capable of sensing its environment, processing information, making decisions, and taking actions to achieve specific goals. It can interact with humans or other systems and is used in various applications like customer service, data analysis, and language translation. For more detailed information, you can refer to this link: [What is AI Agent](https://geekflare.com/ai/what-is-ai-agent/).'
结论
通过以上步骤，我们成功构建了一个基于Langgraph创建一个层次化的Research Team，其中包括搜索、Web爬取和监督代理。每个代理负责一个特定任务，系统通过状态图管理代理的执行顺序，确保任务能够顺利完成。这种方式非常适合用于复杂的自动化信息检索和处理任务。
参考链接：https://langchain-ai.github.io/langgraph/tutorials/multi_agent/hierarchical_agent_teams/
标签：web,search,AI,scraper,agent,Research,ai,分层,Team From： https://blog.csdn.net/qq_41472205/article/details/144309338