Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OllamaPromptDriver doesn't stream responses when using tools #1666

Open
shhlife opened this issue Feb 10, 2025 · 1 comment
Open

OllamaPromptDriver doesn't stream responses when using tools #1666

shhlife opened this issue Feb 10, 2025 · 1 comment
Labels
blocked Can't proceed due to external factors

Comments

@shhlife
Copy link

shhlife commented Feb 10, 2025

In Griptape 1.3.0, the issue with OllamaPromptDriver not working with Tools when stream was enabled was resolved, but now the responses aren't streaming at all.

See this example:

from griptape.drivers import OllamaPromptDriver
from griptape.structures import Agent
from griptape.tools import DateTimeTool
from griptape.utils import Stream

driver = OllamaPromptDriver(model="llama3.2", stream=True)
agent = Agent(prompt_driver=driver, tools=[DateTimeTool()])

for artifact in Stream(agent).run("What's today's date? Write a song about it"):
    print(artifact.value, end="", flush=True)

If you run this with the tools the response won't stream - but if you disable the tool it does.

@collindutter
Copy link
Member

@shhlife unfortunately this is blocked on the ollama side. See:
ollama/ollama#5796
ollama/ollama#7886

@collindutter collindutter added the blocked Can't proceed due to external factors label Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Can't proceed due to external factors
Projects
None yet
Development

No branches or pull requests

2 participants