Repository navigation
Replies: 3 comments
|
@cberes If you only target one model/provider that supports native structured output, setting |
|
Yes, you can pass a dynamically generated JSON Schema directly to Spring AI and you don't need to subclass Spring AI exposes the schema directly through String jsonSchema = objectMapper.writeValueAsString(schema);
var options = StructuredOutputChatOptions.builder()
.outputSchema(jsonSchema)
.build();
var response = chatClient.prompt()
.options(options)
.user(prompt)
.call()
.content();
JsonNode result = objectMapper.readTree(response);Provider-specific options can also expose the same The For portability, |
|
Yes, dynamic schemas work fine with native structured outputs, and you don't need How native structured output picks up the schema
import java.util.function.Function;
import org.springframework.ai.converter.StructuredOutputConverter;
import tools.jackson.core.JacksonException;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
public final class JsonSchemaOutputConverter<T> implements StructuredOutputConverter<T> {
private final ObjectMapper mapper;
private final String schema;
private final Function<JsonNode, T> mapping;
public JsonSchemaOutputConverter(ObjectMapper mapper, JsonNode schema, Function<JsonNode, T> mapping) {
this.mapper = mapper;
this.schema = schema.toString();
this.mapping = mapping;
}
@Override
public String getJsonSchema() {
return schema;
}
// Used only when native structured output isn't applied (prompt-based fallback)
@Override
public String getFormat() {
return """
Your response must be a JSON object that conforms to this JSON Schema.
Do not include any explanation or markdown code fences.
```%s```
""".formatted(schema);
}
@Override
public T convert(String text) {
try {
return mapping.apply(mapper.readTree(text));
} catch (JsonProcessingException e) {
throw new IllegalStateException("Model output is not valid JSON", e);
}
}
}Usage: public <T> T structured(String prompt, JsonNode schema, Function<JsonNode, T> mapping) {
return chatClient.prompt()
.user(prompt)
.call()
.entity(new JsonSchemaOutputConverter<>(objectMapper, schema, mapping),
spec -> spec.useProviderStructuredOutput());
}
Portability
Your Model it as |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I'm currently using OpenAI's API with a json_schema response format. My understanding is that Spring AI's native structured outputs is the equivalent, so when I switch to Spring AI I'd like to use native structured outputs. However I make the schema dynamically. Is that something I can do with Spring AI?
The schema looks something like this, but as I said it's dynamic. The fields in AiResponse may be different each time.
ValueTypeis one of the 4 subtypes below, except we don't actually have aMyEnum; we build the schema'senumproperty dynamically.I found a way to do this where I build the schema by hand using Jackson. And also I use Jackson's readTree to get a JsonNode, then extract the fields manually. Below is an example.
My understanding is that I have to use a BeanOutputConverter if I want to use native structured outputs. Is that correct?
Is there a better or simpler way to do this? I worry that this won't be portable to other LLMs if I have to build the schema by hand.
All reactions