-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Introduce Agent #175
base: main
Are you sure you want to change the base?
Conversation
…erimental into fix/pipeline_run
…erimental into fix/pipeline_run
…erimental into fix/pipeline_run
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Pull Request Test Coverage Report for Build 13048849975Details
💛 - Coveralls |
I want to share some initial thoughts and questions, mostly at a high level. Agent as a component
|
# Conflicts: # haystack_experimental/__init__.py # haystack_experimental/core/__init__.py # haystack_experimental/core/pipeline/__init__.py
6a3ffb4
to
6a3f3e9
Compare
Thanks for the initial feedback!
Yes, exactly, this allows us to integrate it into pipelines. See the notebook for an example of how using it in a pipeline is beneficial. Pros:
Cons:
I tried to keep the use of ToolContext light. It's only used in two places. For users implementing tools on the other hand, ToolContext becomes more central and they need to understand it to use it. We can keep things easy though because ToolContext is optional, so if users don't need it, they don't even need to know it exists. I found the possibility to pass additional context to tools from outside the agent, as well as passing context between tools, and passing outputs from tools to downstream components in the pipeline very useful when implementing agents. I'm not entirely happy with how ToolContext needs to be used by users implementing tools though. I'd be very happy about alternative suggestions.
ComponentTool can be used with Agent without any problems. The only thing that I didn't implement is support for ToolContext. I think that for additional inputs it would be fairly easy to implement. We could just check the input sockets of the components and pass any Assume we move that responsibility to
Ideally, I'd want to leave that decision to the user but I'm not sure how.
I agree that using a pipeline is more verbose but I wouldn't say it's more complex. My main reasons were:
Yes, I know. I interpreted handoff as handing off control of the execution flow, so the idea was that the agent basically hands off to the pipeline. This handoff could also mean that it hands off to another Agent, if there is another Agent in the pipeline. However, I see how this might confuse users. What about I merged main into the PR to remove the diff from the pipeline.run refactoring. You can also ignore the github components in the examples folder for now. There are more components than we actually need for the example notebook. My intention was to add a second agent as an example but I didn't get to it yet. |
Here’s some high-level feedback and brainstorming inspired by this PR. I’m not diving into fine-grained details yet, as high-level considerations are more relevant at this stage. My initial reaction was similar to @anakin87—why is Agent a component? However, after reviewing the details, I believe this design has potential. It enables developers to easily combine deterministic and dynamic (LLM-driven) agent step at the appropriate level of granularity, while supporting the modular, LEGO-like construction of complex, multi agent AI systems. To motivate the feedback and examples below, I’ve imagined what it would take to develop a [Google’s Deep Research](https://blog.google/products/gemini/google-gemini-deep-research/) clone using Haystack. This feedback highlights a few ideas we could incorporate: human feedback mechanisms, programmable handoffs, and two distinct approaches to multi-agent systems/workflows that we should prototype for developer experience (DX) in real examples. 1) Support a Callable Handoff ConditionConcept: Currently, the handoff condition can be Exampledef custom_handoff_condition(llm_output: ChatMessage, tool_result: Optional[ChatMessage]) -> bool:
# Exit when the LLM output contains “DONE” or when it’s particularly short
return "DONE" in llm_output.text
agent = Agent(
model="openai:gpt-4",
tools=[...],
handoff=custom_handoff_condition, # Programmable stop condition
) Why It Helps
2) Human FeedbackHuman feedback is essential for workflows requiring oversight or validation. Depending on the use case, it can be incorporated in one of two ways: A) As a Pipeline ComponentFor guaranteed user interaction at specific points, include a dedicated component for user sign-off: pipe.add_component("humanFeedback", HumanFeedbackComponent(...))
# Enforces explicit "Is this okay?" checkpoints B) As a Tool in an Agent’s ToolsetAlternatively, let the Agent dynamically decide when to involve the user, based on its reasoning: def ask_user_feedback(question: str) -> str:
response = input(f"{question} => ")
return response
human_approval_tool = Tool(
name="human_approval",
function=ask_user_feedback,
description="Ask the user if the partial result or plan is acceptable"
) Why It Helps
3) How Do We Build Multi-Agent Systems in Haystack?When designing workflows involving multiple specialized Agents (we've all see the examples of these online involving mutiple agents), Haystack offers two primary approaches. Each serves different use cases, depending on the need for deterministic execution or LLM-driven orchestration. 3A) Modular Agents as Components in a PipelineConcept: Agents can function as individual components in a pipeline. For instance, a “WebSearchAgent,” “SummarizerAgent,” and “ReviewAgent” can process data sequentially or iteratively. Why It Helps
Example# Agents
webSearchAgent = Agent(model="gpt-4", tools=[...])
summarizerAgent = Agent(model="gpt-4", tools=[])
reviewAgent = Agent(model="gpt-4", tools=[], system_prompt="Review text. If not correct, request improvements.")
# Pipeline
pipe.add_component("webSearchAgent", webSearchAgent)
pipe.add_component("summarizerAgent", summarizerAgent)
pipe.add_component("reviewAgent", reviewAgent)
# Connections
pipe.connect("webSearchAgent.messages", "summarizerAgent.messages")
pipe.connect("summarizerAgent.messages", "reviewAgent.messages")
# Optional: Loop back to SummarizerAgent if ReviewAgent requests changes This approach is ideal for workflows that require deterministic execution where the pipeline graph guarantees precise control. 3B) Master Agent + Tools (Including PipelineTool)Concept: Use the Why It Helps
Example# (A) Sub-pipeline for "ReportMaker"
reportMakerPipeline = Pipeline()
reportMakerPipeline.add_component("webSearchAgent", webSearchAgent)
reportMakerPipeline.add_component("summarizerAgent", summarizerAgent)
reportMakerPipeline.connect("webSearchAgent.messages", "summarizerAgent.messages")
report_maker_tool = PipelineTool(
pipeline=reportMakerPipeline,
name="report_maker",
description="Multi-step process (web search + summary)."
)
# (B) OpenAPITool for final report submission
openapi_tool = OpenAPITool(
spec="https://example.com/submit-report", # Hypothetical endpoint
name="submit_report",
description="Submit the final report"
)
# (C) Master Agent orchestrating tools
masterAgent = Agent(
model="gpt-4",
tools=[report_maker_tool, openapi_tool, human_approval_tool],
system_prompt="""
1) Call 'human_approval' for user feedback when necessary.
2) Use 'report_maker' to generate and refine the report.
3) Submit the final report using 'submit_report'.
"""
)
# Usage
user_message = [ChatMessage.from_user("Research the best AI marketing campaigns of the last 3 years")]
result = masterAgent.run(messages=user_message) This approach is ideal for workflows requiring emergent, dynamic LLM planning, where the master Agent adapts based on context. In summary, this direction shows promise imho. I’d love to double down on developing actual complex agent systems and feel the developer experience firsthand—this would allow us to provide more constructive feedback on how to proceed and build upon these ideas. |
Agent
This PR introduces an
Agent
component that slots right into Haystack's pipelines.An Agent consists of an LLM and tools. The LLM repeatedly call tools until a handoff-condition is met. By default, the
Agent
returns when the LLM produces a pure text message without any tool calls.You run the
Agent
by passing a list of chat messages. The agent returns a list of chat messages including the full sequence of tool calls and tool results.Under the hood, the
Agent
uses a cyclic pipeline with the following components:ChatGenerator
ToolInvoker
ConditionalRouter
BranchJoiner
Usage Example
This is a simple Agent that can perform web search to answer a user's question.
handoff
Currently, the
Agent
exits the loop when thehandoff
-condition is met.The default is
Agent(handoff="text")
which means that the agent exits the loop when it produces a text-only response.Users can set
handoff
to the name of any tool, which will cause the agent to return, after that tool was called.For the web search example, if we set
handoff="web_search"
, the agent would return after theweb_search
-tool was called and executed once.Extending 'handoff'
We could extend
handoff
and allow users to pass a Jinja2-expression in addition to tool names and "text".The
ConditionalRouter
has access to the message history, the last message generated from theChatGenerator
and the last tool result message from theToolInvoker
. Let's say we wanted to return once theChatGenerator
generates a message that contains the word 'Done':Agent(handoff="{{ 'Done' in llm_messages[0].text }}")
This isn't implemented yet, but we could do it.
Passing additional context to tools
In many cases, it is useful to pass additional inputs to a tool or to return results from a tool directly to the pipeline.
Assume we want to add a list of all the search results that the agent viewed to construct an answer with sources.
Additionally, we want to restrict our web search agent to only search on Wikipedia.
This can be achieved by defining
input_variables
andoutput_variables
when initializing theAgent
and by defining tools that acceptctx
.Apart from passing additional inputs to tools and getting additional outputs from the agent, we can also pass values from one tool to another by writing to and reading from
ToolContext.outputs
.To be discussed:
This mechanism does not work with
ComponentTool
yet. We should discuss how the same behaviour could be implemented there.I opted for a simple dataclass used as
ToolContext
. This dataclass does not perform type checking at runtime. Theoretically, we could implement a version ofToolContext
using Pydantic dynamic models, that would validate types at runtime but it's a more complex implementation, brings reliance on Pydantic and I think it has limited benefits.Tracing
Tracing generally works with all of our different tracing integrations. However, since the
Agent
uses a pipeline internally, it would be great if we could associate that trace with the pipeline that called the agent. This is a general enhancement that would also be useful forSuperComponent
as implemented here. I think we should tackle it in a separate PR though.How did you test it?
Notes for the reviewer
Checklist
fix:
,feat:
,build:
,chore:
,ci:
,docs:
,style:
,refactor:
,perf:
,test:
.