Structured Outputs

Structured Outputs

YouTube Video is available:

Introduction

Structured Outputs is a feature in OpenAI's API that ensures the model generates responses adhering to a specified JSON Schema. This guide will walk you through how to use Structured Outputs with the Betalgo.OpenAI library.

Benefits of Structured Outputs

• Reliable type-safety: No need to validate or retry incorrectly formatted responses
• Explicit refusals: Safety-based model refusals are now programmatically detectable
• Simpler prompting: No need for strongly worded prompts to achieve consistent formatting

Getting Started

To use Structured Outputs with Betalgo.OpenAI, you'll need to specify a JSON Schema in your API call. This schema defines the structure of the response you expect from the model.

Supported Models

Structured Outputs are available in the following models:

• gpt-4o-mini-2024-07-18 and later
• gpt-4o-2024-08-06 and later

Basic Usage

Here's a basic example of how to use Structured Outputs with Betalgo.OpenAI:

var completionResult = await sdk.ChatCompletion.CreateCompletion(new()
{
    Messages = new List<ChatMessage>
    {
        ChatMessage.FromSystem("You are a helpful math tutor. Guide the user through the solution step by step."),
        ChatMessage.FromUser("how can I solve 8x + 7 = -23"),
    },
    Model = "gpt-4o-2024-08-06",
    ResponseFormat = new ResponseFormat()
    {
        Type = StaticValues.CompletionStatics.ResponseFormat.JsonSchema,
        JsonSchema = new()
        {
            Name = "math_response",
            Strict = true,
            Schema = PropertyDefinition.DefineObject(
                new Dictionary<string, PropertyDefinition>
                {
                    {
                        "steps", PropertyDefinition.DefineArray(
                            PropertyDefinition.DefineObject(
                                new Dictionary<string, PropertyDefinition>
                                {
                                    { "explanation", PropertyDefinition.DefineString("The explanation of the step") },
                                    { "output", PropertyDefinition.DefineString("The output of the step") }
                                },
                                new List<string> { "explanation", "output" },
                                false,
                                "A step in the mathematical process",
                                null
                            )
                        )
                    },
                    {
                        "final_answer", PropertyDefinition.DefineString("The final answer of the mathematical process")
                    }
                },
                new List<string> { "steps", "final_answer" },
                false,
                "Response containing steps and final answer of a mathematical process",
                null
            )
        }
    }
});

if (completionResult.Successful)
{
    Console.WriteLine(completionResult.Choices.First().Message.Content);
}
else
{
    if (completionResult.Error == null)
    {
        throw new("Unknown Error");
    }
    Console.WriteLine($"{completionResult.Error.Code}: {completionResult.Error.Message}");
}

Alternative Approach Using Betalgo.OpenAI.Utilities (Experimental)

Betalgo.OpenAI.Utilities offers an experimental feature to generate JSON schemas from C# types. This can simplify the process of defining schemas, especially for complex types. However, please note that this is an experimental feature, so use it with caution in production environments.

Here's an example of how to use this approach:

Console.WriteLine("Chat Completion Testing is starting:");
try
{
    var completionResult = await sdk.ChatCompletion.CreateCompletion(new()
    {
        Messages = new List<ChatMessage>
        {
            ChatMessage.FromSystem("You are a helpful math tutor. Guide the user through the solution step by step."),
            ChatMessage.FromUser("how can I solve 8x + 7 = -23")
        },
        Model = "gpt-4o-2024-08-06",
        ResponseFormat = new()
        {
            Type = StaticValues.CompletionStatics.ResponseFormat.JsonSchema,
            JsonSchema = new()
            {
                Name = "math_response",
                Strict = true,
                Schema = PropertyDefinitionGenerator.GenerateFromType(typeof(MathResponse))
            }
        }
    });
    if (completionResult.Successful)
    {
        var response = JsonSerializer.Deserialize<MathResponse>(completionResult.Choices.First().Message.Content!);
        foreach (var responseStep in response?.Steps!)
        {
            Console.WriteLine(responseStep.Explanation);
            Console.WriteLine(responseStep.Output);
        }
        Console.WriteLine("Final:" + response.FinalAnswer);
    }
    else
    {
        if (completionResult.Error == null)
        {
            throw new("Unknown Error");
        }
        Console.WriteLine($"{completionResult.Error.Code}: {completionResult.Error.Message}");
    }
}
catch (Exception e)
{
    Console.WriteLine(e);
    throw;
}

This approach uses PropertyDefinitionGenerator.GenerateFromType() to automatically generate the JSON schema from a C# type (MathResponse in this case). This can significantly reduce the amount of manual schema definition required.

Note: The Betalgo.OpenAI.Utilities library and its PropertyDefinitionGenerator are experimental features. Use them with caution in production environments and always thoroughly test the generated schemas.

Future Developments

With the release of .NET 9, a third option for generating JSON schemas will become available. The System.Text.Json namespace will offer built-in capabilities for generating JSON schemas from C# types, providing a standardized and potentially more robust solution for schema generation.

Defining the JSON Schema

The JSON Schema is defined using the PropertyDefinition class. Here's a breakdown of the schema definition:

1. Use PropertyDefinition.DefineObject() to create an object schema.
2. Define properties using a Dictionary<string, PropertyDefinition>.
3. For array properties, use PropertyDefinition.DefineArray().
4. For string properties, use PropertyDefinition.DefineString().
5. Specify required properties and whether additional properties are allowed.

Handling Responses

The response will be in the completionResult.Choices.First().Message.Content property. You can then parse this JSON string into your desired C# object structure.

Error Handling

Always check if the completion was successful using the Successful property. If it's not successful, you can access the error details through the Error property.

Best Practices

1. Clear Instructions: Include clear instructions in your system or user messages about the expected output format.
2. Error Handling: Implement robust error handling to deal with potential issues like network errors or API limitations.
3. Validation: Although Structured Outputs ensures the response matches your schema, it's still a good practice to validate the parsed data in your application.

Limitations and Considerations

• All fields in your schema must be specified as required.
• Objects have limitations on nesting depth (up to 5 levels) and size (up to 100 properties total).
• Some type-specific keywords (like minLength, maximum, etc.) are not supported.

Conclusion

Structured Outputs in Betalgo.OpenAI provides a powerful way to ensure consistent, structured responses from OpenAI's language models. By defining a clear schema and following best practices, you can create robust applications that leverage the full potential of these models.

For more detailed information, refer to the OpenAI Structured Outputs documentation.