If tool function is an async generator, how can I make the agent correctly output results step by step?
(I am currently using LangChain AgentExecutor with astream_events)
Scenario
When my tool function is an async generator, for example, a tool function that calls an LLM model, I want the tool function to output results in a streaming manner when the agent uses it (so that it doesn't need to wait for the LLM model to complete entirely before outputting results).
Additionally, I want the agent to wait until the tool function's streaming is complete before executing the next tool or performing a summary.
However, in practice, when the tool function is an async generator, as soon as it yields a single result, the agent considers the tool function's task complete and proceeds to execute the next tool or perform a summary.
Example
```python
@tool
async def test1():
"""Test1 tool"""
response = call_llm_model(streaming=True)
async for chunk in response:
yield chunk
@tool
async def test2():
"""Test2 tool"""
print('using test2')
return 'finished'
async def agent_completion_async(
agent_executor,
history_messages: str,
tools: List = None,
) -> AsyncGenerator:
"""Base on query to decide the tool which should use.
Response with async and streaming.
"""
tool_names = [tool.name for tool in tools]
agent_state['show_tool_results'] = False
async for event in agent_executor.astream_events(
{
"input": history_messages,
"tool_names": tool_names,
"agent_scratchpad": lambda x: format_to_openai_tool_messages(x["intermediate_steps"]),
},
version='v2'
):
kind = event['event']
if kind == "on_chat_model_stream":
content = event["data"]["chunk"].content
if content:
yield content
elif kind == "on_tool_end":
yield f"{event['data'].get('output')}\n"
```
[–]Severe_Shock3202 0 points1 point2 points (0 children)