Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Get rate limit headers from request #385

Open
aniskand opened this issue Sep 18, 2024 · 0 comments
Open

Feature Request: Get rate limit headers from request #385

aniskand opened this issue Sep 18, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@aniskand
Copy link

Feature Description

Provide rate limit headers through API.

Problem it Solves

OpenAI provides rate limit headers in the response,

https://platform.openai.com/docs/guides/rate-limits/rate-limits-in-headers

however there is currently no way to access through the API.

Proposed Solution

I'm thinking of 3 possible solutions:

  1. Add separate functions that return response with headers. I created a pull request with an example of this, [FEATURE]Get response headers from messages api #380 but I'm not really in love with this solution. The only benefit of it is that it doesn't change anything for existing users of the API and is fairly straightforward code wise. It's definitely not an ideal solution though.

  2. Add a parameter to the RequestOptions that indicates if user wants headers and include them (or not) in a parent object that includes the response object + headers. This would be fairly simple and clean code wise, but would essentially break the API for all current users.

  3. Keep a persistent store of the rate limit header values in the OpenAI client object and update with every call. I kind of like this solution the best as it would decouple the rate limit headers from the response and make them always available. It also shouldn't break the API for any existing users. It could be a bit complicated to implement due to concurrency issues but I don't think it's insurmountable.

Please feel free to suggest other solutions. Thanks.

Additional Context

Add any other context, examples, or references that might help illustrate the feature or its benefits.

@aniskand aniskand added the enhancement New feature or request label Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant