You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
however there is currently no way to access through the API.
Proposed Solution
I'm thinking of 3 possible solutions:
Add separate functions that return response with headers. I created a pull request with an example of this, [FEATURE]Get response headers from messages api #380 but I'm not really in love with this solution. The only benefit of it is that it doesn't change anything for existing users of the API and is fairly straightforward code wise. It's definitely not an ideal solution though.
Add a parameter to the RequestOptions that indicates if user wants headers and include them (or not) in a parent object that includes the response object + headers. This would be fairly simple and clean code wise, but would essentially break the API for all current users.
Keep a persistent store of the rate limit header values in the OpenAI client object and update with every call. I kind of like this solution the best as it would decouple the rate limit headers from the response and make them always available. It also shouldn't break the API for any existing users. It could be a bit complicated to implement due to concurrency issues but I don't think it's insurmountable.
Please feel free to suggest other solutions. Thanks.
Additional Context
Add any other context, examples, or references that might help illustrate the feature or its benefits.
The text was updated successfully, but these errors were encountered:
Feature Description
Provide rate limit headers through API.
Problem it Solves
OpenAI provides rate limit headers in the response,
https://platform.openai.com/docs/guides/rate-limits/rate-limits-in-headers
however there is currently no way to access through the API.
Proposed Solution
I'm thinking of 3 possible solutions:
Add separate functions that return response with headers. I created a pull request with an example of this, [FEATURE]Get response headers from messages api #380 but I'm not really in love with this solution. The only benefit of it is that it doesn't change anything for existing users of the API and is fairly straightforward code wise. It's definitely not an ideal solution though.
Add a parameter to the RequestOptions that indicates if user wants headers and include them (or not) in a parent object that includes the response object + headers. This would be fairly simple and clean code wise, but would essentially break the API for all current users.
Keep a persistent store of the rate limit header values in the OpenAI client object and update with every call. I kind of like this solution the best as it would decouple the rate limit headers from the response and make them always available. It also shouldn't break the API for any existing users. It could be a bit complicated to implement due to concurrency issues but I don't think it's insurmountable.
Please feel free to suggest other solutions. Thanks.
Additional Context
Add any other context, examples, or references that might help illustrate the feature or its benefits.
The text was updated successfully, but these errors were encountered: