Natural language is excellent for humans.

It is often a poor interface for software.

Suppose an LLM analyzes a support ticket and responds:

This looks like a high-priority billing issue.

A human understands that immediately.

But application code now has to infer:

category = BILLING
priority = HIGH

That creates unnecessary ambiguity.

For production systems, we often want the model to return something closer to a typed API response:

{
  "category": "BILLING",
  "priority": "HIGH",
  "requiresHumanReview": false
}

This is the purpose of structured output.

Structured outputs allow us to move from:

Model generates prose

toward:

Model generates data that conforms to an application contract

This becomes essential when model output influences:

• business logic
• routing
• workflows
• tool calls
• persistence
• API responses
• automation

The key principle is:

If software needs to consume model output, define an explicit contract instead of parsing arbitrary prose.

---

1. Natural Language Is Flexible by Design

Suppose we ask:

What is the category and priority of this request?

"I was charged twice and need this fixed today."

The model might answer:

This appears to be a high-priority billing request.

Or:

Category: BILLING
Priority: HIGH

Or:

I'd classify it under billing, with high urgency.

Or:

BILLING / HIGH

All four answers communicate roughly the same meaning.

That flexibility is useful for humans.

It is inconvenient for programs.

Imagine trying to support every possible variation:

if (response.contains("billing")) {
    ...
}

This approach becomes fragile very quickly.

---

2. Why Prose Parsing Fails

Consider:

if (response.contains("HIGH")) {
    priority = Priority.HIGH;
}

The model returns:

This request is not HIGH priority.

Your code still sees:

HIGH

and produces the wrong result.

Or:

The priority should be MEDIUM rather than HIGH.

Again, contains("HIGH") fails.

You could keep adding parsing rules:

if (...)
else if (...)
else if (...)

but eventually you have built a fragile natural-language parser around probabilistic output.

That is usually the wrong abstraction.

---

3. Define an Application Contract

Suppose our application needs this result:

public record SupportClassification(
        SupportCategory category,
        Priority priority,
        boolean requiresHumanReview
) {
}

With:

public enum SupportCategory {
    BILLING,
    ACCOUNT,
    DELIVERY,
    OTHER
}

and:

public enum Priority {
    LOW,
    MEDIUM,
    HIGH
}

Now the application contract is clear.

A valid result must contain:

category
priority
requiresHumanReview

and category cannot suddenly become:

PAYMENTS

unless the domain explicitly allows it.

This is much stronger than:

String response;

---

4. Structured Output Is About Reducing Ambiguity

Compare:

This seems fairly urgent and probably relates to billing.

with:

{
  "category": "BILLING",
  "priority": "HIGH"
}

The structured version reduces several ambiguities:

Which category?
Which priority?
What fields exist?
How should the application parse them?

It creates a clearer boundary between:

Probabilistic Model

and:

Deterministic Application

Conceptually:

Model
  │
  ▼
Structured Candidate
  │
  ▼
Validation
  │
  ▼
Typed Application Object

---

5. "Return JSON" Is Only the First Step

A common approach is to prompt:

Return JSON in this format:

{
  "category": "...",
  "priority": "..."
}

The model may correctly return:

{
  "category": "BILLING",
  "priority": "HIGH"
}

But it may also return:

Here is the JSON:

{
  "category": "BILLING",
  "priority": "HIGH"
}

Now the response contains prose around the JSON.

Or:

{
  "category": "PAYMENT",
  "priority": "URGENT"
}

This is valid JSON.

But it violates the application's domain.

Or:

{
  "category": "BILLING"
}

The priority field is missing.

Or:

{
  "category": 5,
  "priority": true
}

Again, valid JSON syntax.

Wrong schema.

Therefore:

Valid JSON is not the same as valid application data.

---

6. Syntax Validation and Semantic Validation Are Different

Suppose the model returns:

{
  "category": "BILLING",
  "priority": "HIGH",
  "refundAmount": 900
}

The JSON may be:

syntactically valid

The schema may also allow:

refundAmount: number

But suppose company policy says:

automatic refunds cannot exceed €500

Then the response is structurally valid but semantically unacceptable.

We need to distinguish three layers.

Syntax

Is the output parseable?

Valid JSON?

Schema

Does it match the expected structure?

Required fields?
Correct types?
Allowed enum values?

Domain Semantics

Is the value actually permitted?

Refund amount within policy?
Customer authorized?
Order exists?

Conceptually:

Model Output
    │
    ▼
Syntax Validation
    │
    ▼
Schema Validation
    │
    ▼
Domain Validation
    │
    ▼
Application Use

All three may matter.

---

7. JSON Schema

A schema defines the structure expected from the model.

Conceptually:

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": [
        "BILLING",
        "ACCOUNT",
        "DELIVERY",
        "OTHER"
      ]
    },
    "priority": {
      "type": "string",
      "enum": [
        "LOW",
        "MEDIUM",
        "HIGH"
      ]
    },
    "requiresHumanReview": {
      "type": "boolean"
    }
  },
  "required": [
    "category",
    "priority",
    "requiresHumanReview"
  ]
}

This communicates a much stronger contract than:

Return category and priority.

The schema defines:

• field names
• field types
• allowed values
• required fields

Depending on the model and provider, the API may be able to use this schema to constrain generation.

---

8. Constrained Generation

Some modern model APIs support structured-output modes where the model is constrained to produce output compatible with a supplied schema.

Conceptually:

Prompt
   +
JSON Schema
      │
      ▼
Model
      │
      ▼
Schema-Constrained Generation
      │
      ▼
Structured Output

This is stronger than merely writing:

Please return valid JSON.

The provider may restrict the possible output structure during generation.

That can significantly improve reliability for machine-facing tasks.

But it is important to understand what this does not guarantee.

---

9. Structured Generation Does Not Guarantee Correct Meaning

Suppose the schema requires:

{
  "category": "BILLING | ACCOUNT | DELIVERY | OTHER"
}

The model returns:

{
  "category": "ACCOUNT"
}

The output perfectly satisfies the schema.

But the customer said:

I was charged twice.

The correct category should have been:

BILLING

Structured generation solved:

output format

It did not solve:

reasoning correctness

This distinction is fundamental.

A schema can constrain the shape of an answer. It cannot guarantee that the answer is true.

---

10. Structure Reliability vs Behavioral Reliability

Structured outputs improve one dimension:

Can my application safely parse the result?

Evaluation addresses another:

Did the model choose the correct result?

Conceptually:

             Model Reliability
             /               \
            ▼                 ▼
Structure Reliability   Behavioral Reliability

Example:

{
  "category": "ACCOUNT"
}

Structure:

VALID

Behavior:

WRONG

You need both dimensions.

---

11. Java Types Make the Contract Explicit

Instead of carrying model output around as:

String

prefer domain-oriented types where possible.

For example:

public record SupportClassification(
        SupportCategory category,
        Priority priority,
        boolean requiresHumanReview
) {
}

Then application code can operate on:

classification.category()

rather than:

parseCategory(response)

This makes the interface clearer.

It also allows normal Java validation and testing.

---

12. Enums Are Especially Valuable

Suppose the model can return:

high
HIGH
urgent
critical
very high
P1

Your application now needs interpretation logic.

Instead, define:

public enum Priority {
    LOW,
    MEDIUM,
    HIGH
}

and constrain the model to those values.

Now:

HIGH

has one clear meaning.

Enums reduce the output space.

This is especially useful for:

• classification
• workflow routing
• status selection
• action selection

---

13. Use Domain Types Instead of Generic Strings

Weak contract:

public record Analysis(
        String category,
        String priority,
        String action
) {
}

Stronger:

public record Analysis(
        SupportCategory category,
        Priority priority,
        RecommendedAction action
) {
}

Why?

Because:

String

can contain anything.

Domain types express what values are actually valid.

This is normal software engineering.

LLM integration does not change that.

---

14. Optional Fields Should Be Intentional

Suppose an incident-analysis result can either identify a likely cause or abstain.

A weak schema might be:

{
  "cause": "...",
  "missingInformation": "..."
}

What if no cause is known?

Should cause be:

null

or:

"unknown"

or omitted?

Define the contract explicitly.

For example:

public enum AnalysisStatus {
    LIKELY_CAUSE_IDENTIFIED,
    INSUFFICIENT_EVIDENCE
}

Then:

public record IncidentAnalysis(
        AnalysisStatus status,
        String likelyCause,
        List<String> evidence,
        List<String> missingInformation
) {
}

You might later tighten this design further.

The important point is that absence should have defined semantics.

---

15. Model "Confidence" Requires Care

You might be tempted to request:

{
  "category": "BILLING",
  "confidence": 0.93
}

What exactly does:

0.93

mean?

Unless the system has a calibrated interpretation, it may simply be another generated number.

It does not automatically mean:

93% probability that this classification is correct

Model-generated confidence can sometimes be useful as a feature.

But do not treat it as statistically calibrated without evidence.

A more useful contract may sometimes be:

{
  "category": "BILLING",
  "needsReview": true,
  "ambiguityReason": "Message also contains an account-access issue."
}

This produces inspectable information instead of fake precision.

---

16. Prefer Explicit Uncertainty States

Instead of forcing:

BILLING
ACCOUNT
DELIVERY
OTHER

for every input, perhaps your real application needs:

AMBIGUOUS

or:

INSUFFICIENT_INFORMATION

For example:

public enum ClassificationStatus {
    CLASSIFIED,
    AMBIGUOUS,
    INSUFFICIENT_INFORMATION
}

Then:

public record SupportClassification(
        ClassificationStatus status,
        SupportCategory category,
        String explanation
) {
}

This allows the model to represent legitimate uncertainty.

Otherwise, the schema itself may force hallucination.

---

17. Design the Schema Around the Workflow

Suppose an application has this workflow:

Classification
      │
      ├── LOW risk
      │      ▼
      │   automated processing
      │
      └── HIGH risk
             ▼
         human review

The model output should provide exactly what the workflow needs.

For example:

public record ReviewDecision(
        RiskLevel riskLevel,
        boolean humanReviewRequired,
        String reason
) {
}

Do not ask for 20 fields just because the model can generate them.

Every extra field:

• consumes tokens
• adds complexity
• creates another potential inconsistency
• increases validation requirements

The schema should serve the use case.

---

18. Avoid Letting the Model Reconstruct Data You Already Have

Suppose the application already knows:

orderId = 8192

Do you need the model to return:

{
  "orderId": "8192",
  "category": "BILLING"
}

Maybe.

But if orderId is already trusted application state, asking the model to echo it can introduce unnecessary failure.

The model might return:

8193

Now you have conflicting values.

A good rule is:

Do not ask the model to generate authoritative data that the application already owns.

Model output should focus on what actually requires model capability.

---

19. Never Trust Model-Generated Identifiers Blindly

Suppose the model returns:

{
  "customerId": "9271",
  "action": "ISSUE_REFUND"
}

Do not assume:

customerId = 9271

is valid because it came from structured output.

Before executing:

load customer
authorize access
validate order ownership

The model may have:

• misunderstood
• hallucinated
• selected the wrong identifier

Structured output does not make identifiers authoritative.

---

20. Structured Outputs Are Excellent for Classification

Classification naturally maps to constrained domains.

For example:

public enum TicketCategory {
    BILLING,
    ACCOUNT,
    DELIVERY,
    OTHER
}

Then:

public record TicketClassification(
        TicketCategory category
) {
}

This is much cleaner than asking:

Explain what kind of support problem this is.

and parsing the prose.

Whenever the expected output belongs to a small known set, structured output is usually a strong fit.

---

21. Structured Outputs Are Excellent for Extraction

Suppose the user writes:

Please schedule a meeting with Nur next Tuesday at 3 PM
about the API migration.

The application might need:

{
  "person": "Nur",
  "date": "2026-08-11",
  "time": "15:00",
  "subject": "API migration"
}

A Java representation:

public record MeetingRequest(
        String person,
        LocalDate date,
        LocalTime time,
        String subject
) {
}

This is a natural use of an LLM:

Unstructured Language
        │
        ▼
Model
        │
        ▼
Structured Representation

But extracted data should still be validated.

For example:

Does Nur exist?
Is the date valid?
Is the user authorized to schedule the meeting?

---

22. Extraction and Resolution Are Different Problems

Suppose the user says:

Send €50 to Jalisa.

The model extracts:

{
  "amount": 50,
  "currency": "EUR",
  "recipient": "Jalisa"
}

That is interpretation.

The application still needs to resolve:

"Jalisa"

into an authoritative recipient record.

Conceptually:

Natural Language
      │
      ▼
Model Extraction
      │
      ▼
recipientName = "Jalisa"
      │
      ▼
Application Resolution
      │
      ▼
Recipient ID

Do not make the model invent internal IDs when deterministic resolution is available.

---

23. Structured Outputs Can Represent Decisions

Suppose an incident assistant needs to recommend the next action.

Define:

public enum IncidentAction {
    CONTINUE_INVESTIGATION,
    ROLLBACK,
    ESCALATE,
    REQUEST_MORE_DATA
}

Then:

public record IncidentRecommendation(
        IncidentAction action,
        List<String> evidence,
        String explanation
) {
}

The model can propose a structured decision.

But if ROLLBACK has real production consequences, the surrounding system should decide whether the model is allowed to execute it.

A recommendation and an authorized action are different things.

---

24. Structured Output Is Not Tool Calling

These concepts are related but distinct.

Structured output:

Model returns data

Example:

{
  "category": "BILLING"
}

Tool calling:

Model requests an application capability

Example:

{
  "tool": "getOrder",
  "arguments": {
    "orderId": "8192"
  }
}

Tool calls are themselves structured model outputs.

But their meaning is different because they can lead to execution.

We will study tool calling in depth in the next module.

---

25. Tool Arguments Need the Same Validation

Suppose a tool schema expects:

{
  "orderId": "string",
  "amount": "number"
}

The model generates:

{
  "orderId": "8192",
  "amount": 50000
}

The structure may be perfect.

Should the refund execute?

Not necessarily.

The application must still validate:

Does order 8192 exist?

Does the current user own it?

How much was actually paid?

Is this refund allowed?

Does it require approval?

A structured tool call is a proposal to invoke a capability.

It is not automatic authorization.

---

26. Validation Belongs After Parsing

Suppose we deserialize:

SupportClassification classification =
        objectMapper.readValue(
                response,
                SupportClassification.class
        );

Successful deserialization means:

the JSON could be mapped to this Java shape

It does not necessarily mean:

the result is acceptable

You may still need:

validate(classification);

For example:

if (classification.category() == null) {
    throw new InvalidModelOutputException();
}

or domain-specific validation.

Parsing and validation are separate responsibilities.

---

27. Bean Validation Can Help

Java applications may use validation annotations for application-level constraints.

Conceptually:

public record RefundProposal(
        @NotNull String orderId,
        @NotNull BigDecimal amount,
        @NotNull RefundReason reason
) {
}

Potentially:

@DecimalMin("0.01")

for numeric constraints.

The exact validation approach is less important than the principle:

Model output should pass through normal application validation before being trusted.

LLM output is external input.

Treat it accordingly.

---

28. Cross-Field Validation Matters

Some constraints involve relationships between fields.

Suppose:

{
  "decision": "APPROVE",
  "requiresHumanReview": true
}

These values may conflict with your domain semantics.

Or:

{
  "status": "INSUFFICIENT_EVIDENCE",
  "likelyCause": "Database connection pool exhaustion"
}

Perhaps your schema allows both fields individually.

But together they make little sense.

Cross-field invariants belong in domain validation.

For example:

if (status == INSUFFICIENT_EVIDENCE
        && likelyCause != null) {
    throw new InvalidAnalysisException();
}

Schema validation cannot express every business invariant cleanly.

---

29. Validate Against Authoritative State

Suppose model output says:

{
  "refundAmount": 89.00
}

The order service says the customer paid:

€69.00

Which wins?

The authoritative system.

Conceptually:

Model Proposal
      │
      ▼
refundAmount = €89
      │
      ▼
Order Service
      │
      ▼
paidAmount = €69
      │
      ▼
Validation
      │
      ▼
Reject / Correct / Review

Model output should never silently override authoritative business state.

---

30. What Should Happen When Structured Output Is Invalid?

Suppose the model produces output that:

• cannot be parsed
• violates the schema
• contains unsupported values
• violates domain constraints

You need a failure strategy.

Possible options include:

Retry
Ask model to repair
Use fallback
Request clarification
Human review
Fail safely

The correct choice depends on the task.

---

31. Retrying Can Be Reasonable for Formatting Failures

Suppose the task is low risk and the model returns malformed output.

A bounded retry may be reasonable.

Conceptually:

Model Request
     │
     ▼
Invalid Structure
     │
     ▼
Retry with Error Context
     │
     ▼
Valid Structure

But retries should be bounded.

Do not create:

while (!valid) {
    callModelAgain();
}

This can produce:

• uncontrolled cost
• latency
• loops

Set explicit limits.

---

32. Repair Prompts Are a Fallback, Not the Ideal Contract

A common pattern is:

Your previous response was invalid JSON.

Please fix it.

This may work.

But if the provider supports true schema-constrained structured output, prefer that where appropriate.

Architecture quality generally progresses like:

Free-Form Prose
      ↓
Prompted JSON
      ↓
Schema-Constrained Output
      ↓
Validation
      ↓
Domain Validation

Each step reduces uncertainty.

---

33. Do Not Silently Coerce Dangerous Values

Suppose the model returns:

{
  "priority": "URGENT"
}

but your application supports:

LOW
MEDIUM
HIGH

A tempting mapper is:

if ("URGENT".equals(value)) {
    return Priority.HIGH;
}

This may be reasonable in some low-risk normalization tasks.

But silently inventing mappings can hide model failures.

For important contracts, you may prefer:

unsupported value
→ validation failure

Then measure how often it happens.

Silent coercion can make poor model behavior invisible.

---

34. Distinguish Recoverable and Non-Recoverable Failures

Example:

Model output is missing a comma.

Potentially recoverable.

Example:

Model selected a customer that the user is not authorized to access.

Do not "repair" this by trying to make the tool call work.

That is an authorization failure.

A useful classification is:

Formatting Failure
Schema Failure
Semantic Failure
Authorization Failure
Policy Failure

Different failures require different responses.

---

35. Structured Outputs Improve Testing

Suppose the model returns:

SupportClassification result;

Tests can assert:

assertEquals(
        SupportCategory.BILLING,
        result.category()
);

This is much easier than comparing entire prose responses.

You can separately test:

Parsing
Schema Validation
Domain Validation
Behavioral Correctness

For example:

Input:
"I was charged twice."

Expected:
category = BILLING

The wording of any explanation can vary without breaking the core contract.

---

36. Do Not Over-Test Generated Explanations with Exact Strings

Suppose the model returns:

{
  "category": "BILLING",
  "explanation": "The customer reports being charged twice."
}

Another run returns:

{
  "category": "BILLING",
  "explanation": "The message describes a duplicate charge."
}

Both may be correct.

Your deterministic test can assert:

category == BILLING

while the explanation may require semantic evaluation rather than exact string matching.

Structured outputs let us separate deterministic fields from flexible language.

---

37. Schema Evolution Is an API Design Problem

Suppose version 1 returns:

public record Classification(
        SupportCategory category
) {
}

Later you add:

Priority priority

Now the contract becomes:

public record Classification(
        SupportCategory category,
        Priority priority
) {
}

Questions arise:

Are older consumers compatible?

Is the new field required?

What happens during rollout?

Are stored historical results still readable?

Structured model outputs should be treated like other application contracts.

Schema evolution matters.

---

38. Version Important Structured Contracts

If structured output becomes part of a persistent workflow, consider explicit versioning.

Conceptually:

{
  "schemaVersion": 2,
  "category": "BILLING",
  "priority": "HIGH"
}

You may not need this for every small feature.

But for:

• persisted agent state
• asynchronous workflows
• events
• long-running tasks

schema evolution can become important.

The LLM does not remove compatibility concerns.

---

39. Do Not Expose Internal Model Schemas Directly as Public APIs Without Thought

Suppose your internal model output contains:

{
  "category": "BILLING",
  "internalReason": "...",
  "modelConfidence": 0.81,
  "promptVersion": "v12"
}

Should your public API return all of that?

Probably not automatically.

Consider mapping:

Internal AI Result
       │
       ▼
Application Model
       │
       ▼
Public API Response

This prevents internal model implementation details from becoming public contracts.

---

40. Keep the Model-Facing Schema Focused

A common mistake is to create one giant result object:

public record AiResult(
        String category,
        String priority,
        String summary,
        String explanation,
        String sentiment,
        String action,
        String response,
        List<String> tags,
        ...
) {
}

for every request.

This increases:

• generation complexity
• validation complexity
• token usage
• inconsistency

Prefer task-specific contracts.

For classification:

TicketClassification

For extraction:

MeetingRequest

For incident analysis:

IncidentAnalysis

Specific interfaces are easier to reason about.

---

41. Smaller Schemas Often Produce More Reliable Results

Suppose the model must produce 30 fields.

Every field is another opportunity for:

• hallucination
• inconsistency
• missing values
• incorrect relationships

If only five fields are required for the workflow, return five.

A useful principle:

Ask the model to generate only information the application actually needs from the model.

This improves both reliability and efficiency.

---

42. Separate Extraction from Business Decisions

Suppose the user says:

Refund my last order because it arrived broken.

One model operation might extract:

{
  "intent": "REFUND_REQUEST",
  "reason": "DAMAGED_PRODUCT"
}

Then deterministic application logic can:

Resolve latest order
Verify ownership
Load payment
Check policy
Determine approval requirement

This is often cleaner than asking the model to return:

{
  "intent": "REFUND_REQUEST",
  "orderId": "8192",
  "refundAmount": 89,
  "eligible": true,
  "executeAutomatically": true
}

when most of those fields can be determined from authoritative systems.

Use the model for ambiguity.

Use software for facts and guarantees.

---

43. Structured Outputs Can Create Safer Boundaries

Consider:

User Request
     │
     ▼
LLM
     │
     ▼
Natural-Language Interpretation
     │
     ▼
Structured Intent
     │
     ▼
Validation
     │
     ▼
Application Logic

For example:

{
  "intent": "CHANGE_DELIVERY_ADDRESS",
  "newAddress": {
    "city": "Tallinn",
    "country": "Estonia"
  }
}

The application can now inspect and validate the intent before any state change occurs.

This is much safer than letting generated prose directly control behavior.

---

44. The Model Should Propose; the Application Should Decide

A useful architecture for consequential actions is:

Model
  │
  ▼
Structured Proposal
  │
  ▼
Application Validation
  │
  ▼
Authorization
  │
  ▼
Policy
  │
  ▼
Execute

For example:

{
  "requestedAction": "CANCEL_ORDER",
  "orderReference": "latest order"
}

The application resolves:

latest order

to an actual order.

Then validates:

ownership
status
cancellation eligibility

Then performs the operation.

This is a recurring pattern in production agent systems.

---

45. A Practical Java Example

Suppose we are building a support classifier.

Domain types:

public enum SupportCategory {
    BILLING,
    ACCOUNT,
    DELIVERY,
    OTHER
}

public enum Priority {
    LOW,
    MEDIUM,
    HIGH
}

Output contract:

public record SupportClassification(
        SupportCategory category,
        Priority priority,
        boolean requiresHumanReview
) {
}

Application capability:

public interface SupportClassifier {

    SupportClassification classify(
            String customerMessage
    );
}

Conceptually, the model adapter performs:

Customer Message
      │
      ▼
Prompt + Schema
      │
      ▼
Model
      │
      ▼
Structured Response
      │
      ▼
Deserialize
      │
      ▼
Validate
      │
      ▼
SupportClassification

Notice what the rest of the application receives:

SupportClassification

not:

String

That is a much stronger boundary.

---

46. Validation Example

Suppose:

public SupportClassification validate(
        SupportClassification result
) {
    if (result.category() == null) {
        throw new InvalidModelOutputException(
                "Category is required"
        );
    }

    if (result.priority() == null) {
        throw new InvalidModelOutputException(
                "Priority is required"
        );
    }

    return result;
}

This is simple deterministic validation.

Later, domain rules may be more sophisticated.

The important point is:

Structured model output should pass through application-owned validation before it becomes trusted state.

---

47. Structured Outputs Still Need Observability

Suppose invalid structured output normally occurs:

0.2% of requests

After a model migration:

7% of requests

That is a major regression.

You may want metrics such as:

structured_output_parse_failures

schema_validation_failures

domain_validation_failures

repair_attempts

structured_output_success_rate

AI reliability should be measurable.

Do not quietly catch every exception and hide the failure rate.

---

48. Track Why Validation Failed

Instead of:

Invalid model output

prefer useful categories:

INVALID_JSON

MISSING_REQUIRED_FIELD

UNSUPPORTED_ENUM_VALUE

DOMAIN_CONSTRAINT_FAILURE

AUTHORIZATION_FAILURE

Then production metrics can answer:

Are models formatting incorrectly?

Are schemas poorly designed?

Are values semantically wrong?

Are models attempting unauthorized operations?

Different problems require different fixes.

---

49. Structured Output Does Not Eliminate Evaluation

Suppose your structured-output success rate is:

99.99%

That means almost every response matches the schema.

It does not mean:

99.99% correct

You still need behavioral evaluation.

For a classifier:

classification accuracy

For extraction:

field correctness

For decision support:

decision quality

For grounded answers:

faithfulness to evidence

Schema compliance and task success are different metrics.

---

50. When Structured Output Is Not Necessary

Not every LLM response needs a schema.

Suppose the application asks:

Explain eventual consistency to a student.

The output is meant directly for a human.

Natural language is appropriate.

Likewise:

Write a concise description of this course.

may not need structured output unless metadata is also required.

Use structured outputs primarily when:

software needs to reliably consume the result

Do not add schema complexity where free-form language is actually the desired product.

---

51. A Useful Decision Rule

Ask:

Who consumes this result next?

If the answer is:

A human

free-form language may be appropriate.

If the answer is:

Application code
Workflow engine
Database
Tool executor
Router
Another service

structured output is usually preferable.

Conceptually:

                 Model Output
                  /         \
                 ▼           ▼
              Human       Software
                 │           │
                 ▼           ▼
              Prose       Structure

Not universally, but this is a good default.

---

52. Structured Output and Agent Engineering

Later, agents will repeatedly make decisions such as:

Which tool should I use?

What arguments should I provide?

Should I continue?

Should I ask the user?

Should I finish?

Should this require human approval?

These decisions are much safer when represented explicitly.

Conceptually:

{
  "nextAction": "USE_TOOL",
  "tool": "getOrderStatus",
  "arguments": {
    "orderId": "8192"
  }
}

rather than:

I think I should probably check the order status now.

This is one reason structured generation sits at the foundation of agent systems.

---

53. Strong Mental Model

Think of structured output as a boundary:

Probabilistic World
       │
       ▼
     Model
       │
       ▼
Structured Candidate
       │
========================
       │
Deterministic Boundary
       │
       ▼
Parse
Validate Schema
Validate Domain
Authorize
       │
       ▼
Trusted Application State

The line in the middle matters.

The model can propose.

The application decides whether the proposal is valid enough to use.

---

Practical Exercise — Replace Prose with a Contract

You are given this model response:

This customer appears to have a high-priority billing issue
and should probably be reviewed by a person.

Design:

1. Java enums
2. a Java record
3. an equivalent JSON structure

The application needs:

category
priority
requiresHumanReview

Avoid using generic String fields where a finite domain exists.

---

Practical Exercise — Find the Validation Layers

Suppose the model returns:

{
  "orderId": "8192",
  "refundAmount": 800,
  "decision": "APPROVE"
}

The schema says:

orderId = string
refundAmount = number
decision = APPROVE | REJECT | HUMAN_REVIEW

Company rules say:

Maximum automatic refund = €500

The order database says:

Order 8192 total = €650

Answer:

1. Is the JSON syntactically valid?
2. Does it satisfy the basic schema?
3. Does it satisfy company policy?
4. Can the application safely execute the refund?
5. Which checks belong outside the model?

---

Practical Exercise — Design an Extraction Schema

Input:

Schedule a meeting with Nur next Tuesday at 3 PM
to discuss the agent evaluation design.

Design a Java record for the extracted intent.

Consider:

person
date
time
subject

Then answer:

Which fields should be extracted by the model, and which should be resolved or validated by the application?

For example, should the model generate an internal contactId?

Explain your choice.

---

Practical Exercise — Handle Uncertainty

You are building a support classifier.

Sometimes a message legitimately belongs to more than one possible category.

Design a result contract that can represent:

classified
ambiguous
insufficient information

Avoid forcing the model to always return one confident category.

Then describe what the application should do for each state.

---

Practical Exercise — Diagnose Structured Output Failures

Classify each case as primarily:

Syntax Failure
Schema Failure
Semantic Failure
Authorization Failure

Case A

{"category":

Case B

{
  "category": "PAYMENT"
}

when allowed values are only:

BILLING
ACCOUNT
DELIVERY
OTHER

Case C

{
  "category": "BILLING"
}

for:

I forgot my password.

Case D

{
  "customerId": "9271",
  "action": "READ_ACCOUNT"
}

where the authenticated user has no access to customer 9271.

Explain which layer should detect each failure.

---

Design Exercise — Model Proposal Boundary

Design a workflow for:

A customer asks an AI assistant to cancel their latest order.

The model should interpret the request but should not directly determine authoritative order IDs or bypass cancellation policy.

Your architecture should include:

User
Model
Structured Intent
Order Resolution
Authorization
Cancellation Policy
Order Service
Final Response

Clearly identify where the probabilistic boundary ends and deterministic execution begins.

---

Questions You Should Be Able to Answer

Before moving to the next lesson, make sure you can explain these clearly.

1. Why is free-form prose often a poor interface for software?

Because multiple natural-language expressions can represent the same meaning, making deterministic parsing fragile.

2. What is structured output?

Model output constrained or represented according to an explicit machine-readable contract.

3. Is valid JSON enough?

No.

The output may be valid JSON while violating the expected schema or business semantics.

4. What are the main validation layers?

Typically:

syntax
schema
domain semantics
authorization/policy where relevant

5. What does schema-constrained generation improve?

It improves structural reliability by restricting output toward the expected schema.

6. Does schema-constrained generation guarantee correctness?

No.

The model can produce perfectly valid structure containing the wrong values.

7. Why are Java enums useful for model outputs?

They reduce the allowed output space and express application-owned domains explicitly.

8. Should model-generated identifiers be trusted automatically?

No.

Identifiers should usually be resolved or validated against authoritative application state.

9. What is the difference between extraction and resolution?

Extraction interprets information from unstructured input.

Resolution maps that interpretation to authoritative application entities.

10. Why should model output be validated even after successful deserialization?

Because deserialization proves that data fits a Java shape, not that it is semantically valid or authorized.

11. When is structured output particularly useful?

When the result will be consumed by application code, workflows, routers, tools, databases, or other services.

12. When may free-form text be preferable?

When the primary consumer is a human and flexible natural-language generation is the desired result.

13. Why should agents use structured decisions where possible?

Because agent actions, tool calls, routing decisions, and state transitions need explicit machine-readable interfaces.

14. What is the safest mental model for consequential structured output?

Treat it as a proposal generated by a probabilistic model, then validate and authorize it before execution.

---

Key Takeaways

Natural language is designed for flexibility.

Software interfaces usually need explicit contracts.

Structured outputs bridge that gap.

Instead of:

This looks like an urgent billing problem.

we can request:

{
  "category": "BILLING",
  "priority": "HIGH"
}

and map that result into Java types.

But structure solves only part of the problem.

A model can produce:

valid JSON
+
valid schema
+
wrong decision

Therefore a production pipeline should look like:

Model
  │
  ▼
Structured Candidate
  │
  ▼
Parse
  │
  ▼
Schema Validation
  │
  ▼
Domain Validation
  │
  ▼
Authorization / Policy
  │
  ▼
Application Use

The most important principle is:

Structured output makes probabilistic model behavior easier for deterministic software to consume. It does not make the model deterministic.

Use schemas and typed Java contracts to reduce ambiguity.

Use deterministic validation to enforce invariants.

Use authoritative systems to validate facts.

Use evaluation to determine whether the model is actually making correct decisions.

In the next lesson, we will study Embeddings and Semantic Representation.

We will learn how text can be represented as vectors, how semantic similarity works, why embedding models are different from generative models, and how these ideas later enable retrieval, semantic search, memory, and RAG.