Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails #7500

Merged
merged 7 commits into from
Jan 2, 2025

Conversation

ishaan-jaff
Copy link
Contributor

@ishaan-jaff ishaan-jaff commented Jan 2, 2025

(fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails

sending async requests containing images.
Very quickly, the memory usage spikes up and the proxy shuts down

Key fixes:

  • use truncate_standard_logging_payload_content for all logging integrations. Ensures no field in standard_logging_payload is more than length 10_000
  • for gcs_bucket logging, clear the in-memory queue even if the current batch has a failure

Relevant issues

Type

🐛 Bug Fix
✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locally

If UI changes, send a screenshot/GIF of working UI fixes

Copy link

vercel bot commented Jan 2, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 2, 2025 4:07am

@@ -271,7 +267,6 @@ def create_datadog_logging_payload(
status = DataDogStatus.ERROR

# Build the initial payload
truncate_standard_logging_payload_content(standard_logging_object)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moving this to be common logic for all logging integrations

@ishaan-jaff ishaan-jaff merged commit 26a37c5 into main Jan 2, 2025
25 of 29 checks passed
rajatvig pushed a commit to rajatvig/litellm that referenced this pull request Jan 16, 2025
…ntent` to `standard_logging_payload` and ensure GCS flushes queue on fails (BerriAI#7500)

* use truncate_standard_logging_payload_content

* update truncate_standard_logging_payload_content

* update dd logger

* update gcs async_send_batch

* fix code check

* test_datadog_payload_content_truncation

* fix code quality
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant