Structured output and tool calling
Free text is fine for a chat bubble, but code needs data. Two mechanisms get JSON out of a model reliably.
Structured output - you pass a JSON Schema and the API constrains generation so the answer matches it.
Tool calling - you describe functions (name, description, parameter schema). The model does not run them; it replies with "call getWeather with { city: 'Berlin' }", your code executes it and sends the result back, and the model continues.
text
user -> model: "Weather in Berlin?"
model -> app: tool_use getWeather({ city: "Berlin" })
app -> model: tool_result { temp: 14 }
model -> user: "It's 14°C in Berlin."One schema for the API and TypeScript
ts
import { z } from 'zod';
const Ticket = z.object({
title: z.string().max(80),
priority: z.enum(['low', 'medium', 'high']),
tags: z.array(z.string()).max(5),
});
type Ticket = z.infer<typeof Ticket>;
const res = await llm.generate({
messages,
responseFormat: { type: 'json_schema', schema: z.toJSONSchema(Ticket) },
});
// Never trust it blindly - validate at the boundary
const parsed = Ticket.safeParse(JSON.parse(res.text));
if (!parsed.success) {
// retry with the validation error in the prompt, or fall back
}Why still validate
- Not every model or provider guarantees schema conformance.
- Generation can be cut by
max_tokens, leaving invalid JSON. - A valid shape can still hold invalid values - an id that does not exist, a date in the past.
Tool calling in the UI
- Show tool activity ("Searching orders...") - it explains the pause.
- Ask for confirmation before tools with side effects (sending email, deleting data).
- Validate tool arguments exactly like user input - the model can be manipulated into passing anything.
Treat model output like an external API response: a schema in, a schema check out, and TypeScript types derived from the same source.
updated
did you know it?