Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using Structured Outputs, only strings fields are marked as required #376

Closed
techjb opened this issue Nov 4, 2024 · 10 comments · Fixed by #375 or RageAgainstThePixel/com.openai.unity#303
Assignees
Labels
bug Something isn't working openai bug

Comments

@techjb
Copy link

techjb commented Nov 4, 2024

Bug Report

Overview

When using Structured Outputs, only strings fields are marked as required. According to OpenAI documentation "all fields or function parameters must be specified as required".
When adding number or boolean fields, they are not marked as required and OpenAI API responds with an error.

Screenshots

Part of the input schema
Image

Response from OpenAI:
Image

@techjb techjb added the bug Something isn't working label Nov 4, 2024
@StephenHodgson
Copy link
Member

Pretty sure I've already fixed this is #375

@StephenHodgson StephenHodgson linked a pull request Nov 4, 2024 that will close this issue
@StephenHodgson
Copy link
Member

They recently updated how schemas work on the backend that started breaking existing implementation.

@techjb
Copy link
Author

techjb commented Nov 4, 2024

Great!
Additionally, I'm missing the Description field. When I add the [Description("text... bla bla...")] attribute to the class, it doesn't appear in the object's description field. I hope this can also be fixed.

@StephenHodgson
Copy link
Member

StephenHodgson commented Nov 4, 2024

Great! Additionally, I'm missing the Description field. When I add the [Description("text... bla bla...")] attribute to the class, it doesn't appear in the object's description field. I hope this can also be fixed.

it's not an additional attribute, but part of the FunctionAttribute:

public FunctionAttribute(string description = null)

[Function(description: "blah blah")]
private void Function()
{
}

@techjb
Copy link
Author

techjb commented Nov 4, 2024

Look at this example in the OpenAI documentation:
https://platform.openai.com/docs/guides/structured-outputs#all-fields-must-be-required

Properties also contains descriptions to give more context.

@StephenHodgson
Copy link
Member

Look at this example in the OpenAI documentation: https://platform.openai.com/docs/guides/structured-outputs#all-fields-must-be-required

Properties also contains descriptions to give more context.

I'm well aware.

FunctionPropertyAttribute also supports descriptions, as well as other things:

public FunctionPropertyAttribute(string description = null, bool required = false, object defaultValue = null, params object[] possibleValues)

I've already fixed some of this recently just needs to get ported. Recently OpenAI updated their rules on this stuff and broke compatibility.

I simply disabled strict for the tool definitions generated by the built in function attributes, which should fix this issue.

If you provide your own schema definitions, then that is another story.

@StephenHodgson
Copy link
Member

FWIW if you're using .net 9, then there is a json schema generator that should be compliant:

https://devblogs.microsoft.com/dotnet/system-text-json-in-dotnet-9/#json-schema-exporter

@techjb
Copy link
Author

techjb commented Nov 4, 2024

Thank you for the response. So, is there currently any way to provide attributes with descriptions to obtain a structured outputs?

@StephenHodgson
Copy link
Member

StephenHodgson commented Nov 4, 2024

Thank you for the response. So, is there currently any way to provide attributes with descriptions to obtain a structured outputs?

see last comment about .net 9 json schema exporter. Likely I'll remove/deprecate the custom impl in this library once .net 8 EOL comes around.

@techjb
Copy link
Author

techjb commented Nov 5, 2024

Finally I was able to add my own schema. I did it using the library Newtonsoft.Json.Schema.

JSchemaGenerator generator = new()
{
    DefaultRequired = Required.AllowNull,
    GenerationProviders =
    {
        new StringEnumGenerationProvider(),
    }
};

JSchema jSChema = generator.Generate(typeof(OpenAIStructuredOutput));
SetAdditionalPropertiesFalse(jSChema); // this method set AllowAdditionalProperties=false to all objects
var schema = jSChema.ToString();
var jsonSchema = new global::OpenAI.JsonSchema("OpenAIStructuredOutput", schema);


var chatRequest = new ChatRequest(
    messages: messages,
    model: MODEL_NAME,
    temperature: TEMPERATURE,
    responseFormat: ChatResponseFormat.JsonSchema,
    jsonSchema: jsonSchema
    );

ChatResponse chatResponse = Task.Run(async () => await OpenAIClient.ChatEndpoint.GetCompletionAsync(chatRequest)).Result;
var structuredOutput = JsonConvert.DeserializeObject<OpenAIStructuredOutput>(chatResponse.FirstChoice);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working openai bug
2 participants