Feat: Structured Outputs

[andgordio]

What

Adds support for Structured Outputs.

Why

From the official guides:

Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don't need to worry about the model omitting a required key, or hallucinating an invalid enum value.

Affected Areas

• Extends ChatQuery.ResponseFormat

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[horsejockey]

This looks good to me.

[sirily11]

I wonder how to include a description on each field? This might be helpful for schema generation similar to openai api spec.

[andgordio]

I wonder how to include a description on each field? This might be helpful for schema generation similar to openai api spec.

@sirily11 I haven’t found a way to incorporate descriptions into the protocol without requiring an explicit schema design or relying on code generation tools. The proposed solution instead leverages descriptive property names and the context provided within the prompt itself. This approach has consistently worked well for my (admittedly limited) use cases.

While it’s a trade-off, I believe this approach is preferable to a more complex and maintenance-heavy solution that would require defining an explicit schema (including all nested objects). For example:

extension MovieInfo: StructuredOutput {
    static var structuredData: [StructuredOutputData<MovieInfo>] {
        [
            StructuredOutputData(
                keyPath: \.title,
                name: "title",
                description: "Description of the title property",
                type: String.self
            ),
            StructuredOutputData(
                keyPath: \.director,
                name: "director",
                description: "Description of the director property",
                type: String.self
            ),
            StructuredOutputData(
                keyPath: \.release,
                name: "release",
                description: "Description of the release property",
                type: Date.self
            ),
            StructuredOutputData(
                keyPath: \.cast,
                name: "cast",
                description: "Description of the cast property",
                type: Array<String>.self
            ),
        ]
    }
}

Here’s the proposed alternative API for comparison:

extension MovieInfo: StructuredOutput {
    static let example: Self = {
        .init(
            title: "Earth",
            director: "Alexander Dovzhenko",
            release: Calendar.current.date(from: DateComponents(year: 1930, month: 4, day: 1))!,
            cast: ["Stepan Shkurat", "Semyon Svashenko", "Yuliya Solntseva"]
        )
    }()
}

[sirily11]

I wonder how to include a description on each field? This might be helpful for schema generation similar to openai api spec.

@sirily11 I haven’t found a way to incorporate descriptions into the protocol without requiring an explicit schema design or relying on code generation tools. The proposed solution instead leverages descriptive property names and the context provided within the prompt itself. This approach has consistently worked well for my (admittedly limited) use cases.

While it’s a trade-off, I believe this approach is preferable to a more complex and maintenance-heavy solution that would require defining an explicit schema (including all nested objects). For example:

extension MovieInfo: StructuredOutput {
    static var structuredData: [StructuredOutputData<MovieInfo>] {
        [
            StructuredOutputData(
                keyPath: \.title,
                name: "title",
                description: "Description of the title property",
                type: String.self
            ),
            StructuredOutputData(
                keyPath: \.director,
                name: "director",
                description: "Description of the director property",
                type: String.self
            ),
            StructuredOutputData(
                keyPath: \.release,
                name: "release",
                description: "Description of the release property",
                type: Date.self
            ),
            StructuredOutputData(
                keyPath: \.cast,
                name: "cast",
                description: "Description of the cast property",
                type: Array<String>.self
            ),
        ]
    }
}

Here’s the proposed alternative API for comparison:

extension MovieInfo: StructuredOutput {
    static let example: Self = {
        .init(
            title: "Earth",
            director: "Alexander Dovzhenko",
            release: Calendar.current.date(from: DateComponents(year: 1930, month: 4, day: 1))!,
            cast: ["Stepan Shkurat", "Semyon Svashenko", "Yuliya Solntseva"]
        )
    }()
}

I have an idea using swift macro. The schema defination could be simplified to

@Schema
class MyModel {

@Property(description: "Some description", example: "Hello world")
let name: String
}

This proposed API similar to Swift Data package which minimize the required code to write the schema as well as provides the similar API compares to the Python and JS official examples.

[Prince2k3]

When will this be merged? I am in need of this PR

[DanielhCarranza]

When will this be merged? I am in need of this PR

This repo moves too slow, you may need to fork it and apply the changes, otherwise you will wait until eternity

[kominak]

When will this be merged? I am in need of this PR

This repo moves too slow, you may need to fork it and apply the changes, otherwise you will wait until eternity

I'm already using @andgordio's fork in a production code, and it works amazingly well. Saved me a ton of time and effort. Would be great to see it merged...

[nezhyborets]

Thanks for a great contribution @andgordio ! It looks good. The only thing I see is they say with Structured Outputs they added refusal field which is maybe better to handle somehow. And we'd also later have to implement structure outputs for tools. Otherwise perfect. I'll try to look into "refusal" moment to see if I can append it quickly to this PR

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[nezhyborets]

It may need refactoring, so I've added a task and will merge