Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 fix: update message roles for specific Azure OpenAI models #6222

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

hcygnaw
Copy link
Contributor

@hcygnaw hcygnaw commented Feb 16, 2025

💻 变更类型 | Change Type

  • ✨ feat
  • 🐛 fix
  • ♻️ refactor
  • 💄 style
  • 👷 build
  • ⚡️ perf
  • 📝 docs
  • 🔨 chore

🔀 变更说明 | Description of Change

Fix #6201

system role切换成user role或者developer role。

📝 补充信息 | Additional Information

当使用 Azure OpenAI 的 O-series 模型(如 o1, o1-mini 等)时,如果设定了 system prompt 会导致以下错误:

"error": {
  "message": "Unsupported value: 'messages[0].role' does not support 'system' with this model.",
  "type": "invalid_request_error",
  "param": "messages[0].role",
  "code": "unsupported_value"
}

Copy link

vercel bot commented Feb 16, 2025

@cygnaw is attempting to deploy a commit to the LobeHub Team on Vercel.

A member of the Team first needs to authorize it.

@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Feb 16, 2025
@lobehubbot
Copy link
Member

👍 @hcygnaw

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献,请确保您已经遵循了我们的贡献指南,我们会尽快审查它。
如果您遇到任何问题,请随时与我们联系。

@dosubot dosubot bot added the 🐛 Bug Something isn't working | 缺陷 label Feb 16, 2025
Copy link

codecov bot commented Feb 16, 2025

Codecov Report

Attention: Patch coverage is 85.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 91.83%. Comparing base (a0a9592) to head (e3b31d2).
Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
src/libs/agent-runtime/azureOpenai/index.ts 85.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6222      +/-   ##
==========================================
- Coverage   91.86%   91.83%   -0.04%     
==========================================
  Files         684      684              
  Lines       62196    62311     +115     
  Branches     3095     2908     -187     
==========================================
+ Hits        57137    57221      +84     
- Misses       5059     5090      +31     
Flag Coverage Δ
app 91.83% <85.00%> (-0.04%) ⬇️
server 97.90% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -33,9 +33,28 @@ export class LobeAzureOpenAI implements LobeRuntimeAI {
const { messages, model, ...params } = payload;
// o1 series models on Azure OpenAI does not support streaming currently
const enableStreaming = model.includes('o1') ? false : (params.stream ?? true);

// Convert 'system' role to 'user' or 'developer' based on the model
const systemToUserModels = new Set([
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我感觉是不是反了?目前支持 develop 的反而是少数。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是抄的这边的代码。我只有o1-mini的模型可以用来测试,改了之后是可以跑通的

export const pruneReasoningPayload = (payload: ChatStreamPayload) => {
// TODO: 临时写法,后续要重构成 model card 展示配置
const disableStreamModels = new Set([
'o1',
'o1-2024-12-17'
]);
const systemToUserModels = new Set([
'o1-preview',
'o1-preview-2024-09-12',
'o1-mini',
'o1-mini-2024-09-12',
]);
return {
...payload,
frequency_penalty: 0,
messages: payload.messages.map((message: OpenAIChatMessage) => ({
...message,
role:
message.role === 'system'
? systemToUserModels.has(payload.model)
? 'user'
: 'developer'
: message.role,
})),
presence_penalty: 0,
stream: !disableStreamModels.has(payload.model),
temperature: 1,
top_p: 1,
};
};

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不能照抄。

https://github.com/lobehub/lobe-chat/pull/5714/files

你自己看这里的变更,这个变换是加了约束的:

if (model.startsWith('o1') || model.startsWith('o3')) {}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

根据这篇帖子里的讨论,developer现在取代了过去的system。

我这边试过Azure的gpt-4o和gpt-4o-mini,都是能正常使用developer的role的。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果这个provider 仅局限于 azure openai 的话估计没问题。但保不准有人用这个 provider 来调 phi-4 等其他模型吧?

@@ -33,9 +33,28 @@ export class LobeAzureOpenAI implements LobeRuntimeAI {
const { messages, model, ...params } = payload;
// o1 series models on Azure OpenAI does not support streaming currently
const enableStreaming = model.includes('o1') ? false : (params.stream ?? true);

// Convert 'system' role to 'user' or 'developer' based on the model
const systemToUserModels = new Set([
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不能照抄。

https://github.com/lobehub/lobe-chat/pull/5714/files

你自己看这里的变更,这个变换是加了约束的:

if (model.startsWith('o1') || model.startsWith('o3')) {}

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Feb 18, 2025
@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 Bug Something isn't working | 缺陷 size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Azure OpenAI o-series 模型不支持 system role,导致报错
4 participants