openai[patch]: default to invoke on o1 stream() #27983
Conversation
dosubot
bot
added
the
size:M
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
| ) -> bool: | ||
| if isinstance(response_format, type) and is_basemodel_subclass(response_format): | ||
| # TODO: Add support for streaming with Pydantic response_format. | ||
| warnings.warn("Streaming with Pydantic response_format not yet supported.") |
There was a problem hiding this comment.
should we also emit a warning in case of o1, for clarity about why results aren't streamed?
There was a problem hiding this comment.
maybe. just seems like model is something you're more likely to let end users configure (eg the way langsmith playground does) in which case warnings would be annoying and unhelpful
dosubot
bot
added
the
lgtm
| ) -> bool: | ||
| if isinstance(response_format, type) and is_basemodel_subclass(response_format): | ||
| # TODO: Add support for streaming with Pydantic response_format. | ||
| warnings.warn("Streaming with Pydantic response_format not yet supported.") |
There was a problem hiding this comment.
FYI this warning is raised even when a non-streaming API is used (e.g. ainvoke). This is because _generate_with_cache (called by invoke) calls _should_stream internally:
I don't think it has any bad side effect, besides that the warning used to be raised for a real issue, and it may now be a false positive. However, it is possible that invoke used to stream beforehand and won't anymore 🤔
@baskaryan I'd love to know if we can safely bump if we are using ainvoke with structured_outputs + pydantic
No description provided.