(Fixes) OpenAI Streaming Token Counting + Fixes usage track when litellm.turn_off_message_logging=True

[ishaan-jaff]

(Fixes) OpenAI Streaming Token Counting + Fixes usage track when litellm.turn_off_message_logging=True

Bug Description

• For OpenAI streaming the Usage tracked in callbacks was != Usage from OpenAI API

Key changes made

• For streaming ensure logging callbacks use usage from OpenAI api response
• When users don't send include_usage to OpenAI API, litellm will still pass include_usage to OpenAI APIs to guarantee accurate token counting
• Unit testing for both scenarios

Test Cases

1. test_stream_token_counting_gpt_4o

   • Tests if token usage tracking works correctly when stream_options={"include_usage": True}
   • Verifies that the logging callback records the same usage metrics as returned by OpenAI API

2. test_stream_token_counting_without_include_usage

   • Tests token usage tracking without explicitly setting include_usage
   • Verifies that the logging callback records the same usage metrics as returned by OpenAI API

3. test_stream_token_counting_with_redaction

   • Tests token usage tracking when message logging is disabled (litellm.turn_off_message_logging=True)
   • Verifies that the logging callback records the same usage metrics as returned by OpenAI API

Relevant issues

Type

🆕 New Feature
✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locally

If UI changes, send a screenshot/GIF of working UI fixes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 31, 2025 10:55pm

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!