[instrumentation] litellm streaming #1227
Labels
instrumantation: litellm
related to litellm and litellm proxy
Comments
dosubot
bot
added
the
instrumantation: litellm
keyur2maru
added a commit
to keyur2maru/openinference
that referenced
this issue
Feb 3, 2025
- import ModelResponseStream to handle streaming responses - implement `_finalize_streaming_span` to process streamed tokens - update `_acompletion_wrapper` to detect and trace streaming results - ensure span attributes capture message roles, content, and usage stats - properly end spans for both standard and streaming responses This change enables instrumentation for streaming responses, ensuring tracing captures relevant metadata for streamed tokens, resolves Arize-ai#1227. Signed-off-by: Keyur Maru <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: