Skip to content

Commit

Permalink
fix(openai): support all o series models (yetone#1031)
Browse files Browse the repository at this point in the history
Before this change, since `max_completion_tokens` was not set for `o` series models, the completion request will time out sometimes. This makes sure it converts the `max_tokens` parameter to `max_completion_tokens` for `o` series models.

I tested this change with `gpt-4o-mini`, `o1-mini` and `o3-mini`, and they all still work as expected.
  • Loading branch information
larrylv authored and brewinski committed Jan 10, 2025
1 parent e09de5f commit 5471689
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions lua/avante/providers/openai.lua
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,18 @@ M.get_user_message = function(opts)
)
end

M.is_o_series_model = function(model)
return model and string.match(model, "^o%d+") ~= nil
end

M.parse_messages = function(opts)
local messages = {}
local provider = P[Config.provider]
local base, _ = P.parse_config(provider)

-- NOTE: Handle the case where the selected model is the `o1` model
-- "o1" models are "smart" enough to understand user prompt as a system prompt in this context
if base.model and string.find(base.model, "o1") then
if M.is_o_series_model(base.model) then
table.insert(messages, { role = "user", content = opts.system_prompt })
else
table.insert(messages, { role = "system", content = opts.system_prompt })
Expand Down Expand Up @@ -150,9 +154,10 @@ M.parse_curl_args = function(provider, code_opts)
headers["Authorization"] = "Bearer " .. api_key
end

-- NOTE: When using "o1" set the supported parameters only
-- NOTE: When using "o" series set the supported parameters only
local stream = true
if base.model and string.find(base.model, "o1") then
if M.is_o_series_model(base.model) then
body_opts.max_completion_tokens = body_opts.max_tokens
body_opts.max_tokens = nil
body_opts.temperature = 1
end
Expand Down

0 comments on commit 5471689

Please sign in to comment.