There is a gap between using an LLM in a chat interface and deploying one in a production pipeline. In a chat interface, you read the output. You notice when it's wrong. You ask again. The feedback loop is you.
In a production pipeline, there is no you. The output gets parsed, stored, forwarded, and acted on. If the model returns a malformed date, a missing required field, or a plausible-sounding value that violates your domain rules, the pipeline either crashes somewhere downstream — ideally — or silently propagates the error, which is worse.
Making LLM output reliable in production means replacing your attentive reading with an automated structure. Three techniques do the work: schema validation, retry logic, and fallback paths. They are not alternatives — they are layers. Each depends on the one before it.
The baseline: treat agent output as guilty until proven innocent
Before getting into the techniques, a number worth internalizing: in a production agentic pipeline built by a senior engineer specifically to catch failures, 68% of agent-generated changes contained bugs before automated validation ran.
Not 5%. Not 20%. The majority.
This figure comes from an engineer who had invested heavily in their validation infrastructure — and still found that most of what the agents produced needed to be caught before it was safe to review. The validation wasn't a safety net for occasional failures. It was the primary quality gate.
If you are running LLM outputs directly into production without validation, you are not skipping a nice-to-have. You are removing the gate entirely.
Layer 1: Schema validation — the contract at the output boundary
The cleanest way to think about schema validation is through Design by Contract: a component's behavior is specified as a postcondition — what it guarantees to return. If the output violates that postcondition, the call failed, regardless of how plausible the output looks.
Your schema specifies the postcondition for what the model must return. Write it before you wire up the LLM call, not after. This forces you to answer the questions that actually matter — what fields are required, what types are acceptable, what formats are valid, what shape is expected — before you have a model response to distract you. The schema is a thinking tool first, a validation mechanism second.
What a schema should enforce:
- Required fields — if your downstream code will key-access a field, it must exist.
- Types — a date string is not a datetime object; an integer is not a string representation of a number.
- Formats — enums, identifiers, ISO dates; "any string" is not a postcondition.
- Shape — object vs. array; nesting depth; field allowlist (no unexpected additions).
What a schema cannot enforce:
- Whether the reasoning section actually supports the conclusion.
- Whether the medical diagnosis is clinically valid.
- Whether the generated code is logically correct.
Schema validation catches structural failures. Semantic failures — subtle wrongness, stale patterns, incoherence — require a different layer.
In Python, Pydantic is the standard tool: define your expected output as a typed class, parse the
LLM's JSON response against it, and handle ValidationError cleanly. The instructor
library goes further by constraining generation to the schema rather than validating after the fact.
In TypeScript, Zod does the same job. The specific library matters less than the habit: every LLM
output boundary gets a schema, and validation failure is handled explicitly, not silently.
The Design by Contract principle that applies here is "crash early": when a
postcondition is violated, fail immediately rather than propagating corrupted state downstream. A
downstream KeyError three function calls later is harder to debug than a
ValidationError at the output boundary. Fail at the point of violation.
Layer 2: Retry logic — the self-correction loop
Schema validation tells you whether the output is valid. It does not fix it.
The natural response is to retry. But there is a meaningful difference between a good retry and a bad one, and getting it wrong just makes things expensive.
Bad retry: resend the exact same prompt. This resamples from the same distribution. You get a different output with the same structural properties. The failure rate is not zero, but it is roughly the same as the first attempt. You are spending inference cost to get a coin flip.
Good retry: feed the validation error back into the prompt. The validation failure is a signal — the only signal you have about why the output was wrong. Discarding it and retrying is equivalent to fixing a bug by running the same failing test again without changing the code.
Error-informed retry looks like this:
retry_context = ""
for attempt in range(MAX_RETRIES):
response = call_llm(base_prompt + retry_context)
try:
return OutputSchema.model_validate_json(response)
except ValidationError as e:
retry_context = (
f"\n\nYour previous response failed validation: {e}. "
f"Ensure the response is valid JSON matching the required schema."
)
return fallback_handler(last_error)
The retry_context is the schema error, fed back as additional context. Each retry is
more constrained than the last. The model has new information about what it got wrong.
A few disciplines that make retry logic actually work:
Bound your retries. Two or three is usually enough. Beyond that, you are likely hitting a structural problem that better context will not resolve — and you are burning tokens and latency.
Carry the state across retries. The retry prompt should include the reason for the prior failure. Some implementations also include the prior failed output, which gives the model something to correct rather than regenerate from scratch.
Each retry should add specificity. If the first retry says "your output was
invalid," the second retry should say "your output was invalid because the start_date
field was absent." The error message should get more precise, not more verbose.
Layer 3: Fallback paths — bounded degradation
Retries run out. When they do, you need a policy.
The instinct is often to return an error. That is sometimes right — but "fail loudly" is one option on a spectrum, not the default. The right fallback depends on how critical the output is and what safe alternatives exist.
- Simplify the request. If the full output specification is too complex for the model to satisfy consistently, ask for less. Request fewer fields, a simpler structure, a narrower scope. Partial output delivered reliably is often more useful than complete output delivered inconsistently.
- Degrade to a different model. Some output types are harder for some models. If a smaller or specialized model handles structured output more reliably for your specific task, route to it as a fallback.
- Return a structured error. Rather than propagating failure silently or crashing, return an explicit structured error: what was requested, what was returned, and why it failed. This makes the failure visible to the calling system and actionable for monitoring.
- Use a hardcoded default. If the output is optional or if a safe default exists, use it. This is appropriate when the AI value-add is real but not critical — "summarize this document" can fall back to "summary unavailable" without breaking the user experience.
- Fail loudly. When there is no safe default, when correctness matters more than availability, when silent degradation would be worse than interruption — fail explicitly with a structured error. This is the crash-early principle applied at the system level.
The selection rule: match the fallback to the output's criticality, not to the developer's preference for clean code. A healthcare pipeline that silently defaults to an empty diagnosis is not robust — it is dangerous.
Beyond structure: inferential sensors and trajectory evaluation
Schema validation catches structural failures. The four predictable AI failure modes include two that schema cannot reach: subtle wrongness (code that looks right, passes tests, and is still wrong) and stale patterns (outdated idioms that compile fine but represent bad practice).
For these, the production harness needs a second layer of sensors: inferential sensors — LLM-as-judge checks, semantic-similarity validation, and domain-specific rule engines. These are slower and more expensive than schema validation, so they belong later in the pipeline (after schema passes, before the output reaches critical downstream systems).
A further extension for high-stakes pipelines: trajectory evaluation. Rather than only checking whether the output is correct, trajectory evaluation checks whether the agent actually performed its verification steps — did it run the test suite before claiming it passed, did it read the spec before implementing? A fluent output that skipped its verification steps is a more dangerous failure than one with a visible error, because the next run on different inputs will fail without warning.
Trajectory evaluation requires logging the agent's full tool-call sequence, which means investing in observability infrastructure. For pipelines where the agent is operating autonomously and the cost of a missed failure is high, it is the difference between knowing "this output is valid" and knowing "this output was produced correctly."
The stack in one picture
LLM call
↓
[Schema validation] ← fast, deterministic, cheap
PASS → downstream
FAIL → validation error
↓
[Retry with error context] ← error feeds repair
PASS → downstream
EXHAUSTED
↓
[Fallback path] ← simplify / degrade / fail loudly
↓
[Inferential sensor] ← semantic quality check; slower
↓
downstream consumerEach layer catches what the layer before it cannot. Schema catches structure. Retry uses the schema failure to repair. Fallback bounds the worst case. Inferential sensors catch semantics. Together, they replace the attentive reading you do in interactive use — structurally, automatically, at scale.
The underlying shift is the same one that applies to every layer of an AI system: the human's job is not to catch each failure individually, but to design the structure that systematically catches failures. Schema validation is cheap and catches the most common failures. Retry with error context is straightforward and dramatically better than blind retry. Fallback paths are explicit policy decisions that every production AI pipeline needs, regardless of the specific implementation.
None of these requires waiting for the model to improve. They require deciding, before wiring up the LLM call, what "valid output" means — and encoding that decision as a schema, a retry policy, and a fallback. The schema is the contract. Write it first.