Skip to content

Commit

Permalink
fix(openai): support all o series models (#1031)
Browse files Browse the repository at this point in the history
Before this change, since `max_completion_tokens` was not set for `o` series models, the completion request will time out sometimes. This makes sure it converts the `max_tokens` parameter to `max_completion_tokens` for `o` series models.

I tested this change with `gpt-4o-mini`, `o1-mini` and `o3-mini`, and they all still work as expected.
  • Loading branch information
larrylv authored Jan 5, 2025
1 parent 81e5f19 commit ec5d1ab
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions lua/avante/providers/openai.lua
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,18 @@ M.get_user_message = function(opts)
)
end

M.is_o_series_model = function(model)
return model and string.match(model, "^o%d+") ~= nil
end

M.parse_messages = function(opts)
local messages = {}
local provider = P[Config.provider]
local base, _ = P.parse_config(provider)

-- NOTE: Handle the case where the selected model is the `o1` model
-- "o1" models are "smart" enough to understand user prompt as a system prompt in this context
if base.model and string.find(base.model, "o1") then
if M.is_o_series_model(base.model) then
table.insert(messages, { role = "user", content = opts.system_prompt })
else
table.insert(messages, { role = "system", content = opts.system_prompt })
Expand Down Expand Up @@ -150,9 +154,10 @@ M.parse_curl_args = function(provider, code_opts)
headers["Authorization"] = "Bearer " .. api_key
end

-- NOTE: When using "o1" set the supported parameters only
-- NOTE: When using "o" series set the supported parameters only
local stream = true
if base.model and string.find(base.model, "o1") then
if M.is_o_series_model(base.model) then
body_opts.max_completion_tokens = body_opts.max_tokens
body_opts.max_tokens = nil
body_opts.temperature = 1
end
Expand Down

2 comments on commit ec5d1ab

@pidgeon777
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@larrylv you wrote:

I tested this change with gpt-4o-mini, o1-mini and o3-mini, and they all still work as expected.

Does this mean that o3-mini is now available in GitHub Copilot Chat? Which are the models you currently have access to, if I may ask?

@larrylv
Copy link
Contributor Author

@larrylv larrylv commented on ec5d1ab Jan 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pidgeon777 it's an internal model that we have access to (I work at OpenAI). still not released to public yet.

Please sign in to comment.