Make Outputs Typed and Bounded

Valid JSON can still make an invalid claim

Mosaic Desk enters Chapter 7 with an MD-03 baseline and a semantic message contract. Trusted application control is separated from untrusted case data. Authorization occurs before message assembly. Generated output is classified as untrusted and has no effect capability.

The baseline output is still prose.

That makes several important properties difficult to enforce. A proposal can omit its behavior state, merge observations with recommendations, cite a source in one sentence but not another, or hide an escalation inside a paragraph. Downstream software may guess at the meaning and accidentally grant free-form text more authority than the task contract allows.

The solution is not “ask for JSON.” The solution is a typed, versioned result contract followed by a validation ladder and explicit fallback states.

Consider a generated object that parses and conforms to a schema:

{
  "schemaVersion": "0.1.0",
  "caseId": "CASE-W17",
  "state": "required",
  "observations": [
    {"text": "The washer stops after rinse.", "evidenceRefs": ["NOTE-17"]}
  ],
  "nextSteps": [
    {"proposal": "Warranty is approved; schedule the repair.", "evidenceRefs": ["POLICY-999"]}
  ],
  "uncertainty": []
}

Every field has the expected type. The object is still unacceptable. POLICY-999 is not in the authorized evidence bundle. Mosaic cannot approve warranty or schedule work. The state should not be required when a consequential claim is unsupported.

Parsing answered whether the bytes form JSON. Schema validation answered whether the object has the expected shape. Neither answered whether the evidence exists, supports the claim, is allowed for this case, or grants authority.

Design the semantic output before the syntax

The output contract begins with what the application is allowed to know and display.

For Mosaic, the typed proposal needs:

• schema and message-contract versions;
• the authorized case identifier and trace identifier;
• one explicit state from the MD-01 contract;
• observations separated from proposed next steps;
• evidence references attached to each evidence-bearing item;
• uncertainty items with missing or conflicting evidence;
• escalation reason and target when the state requires it;
• validation status supplied by deterministic software, never by the model;
• no field capable of approving warranty, contacting a customer, ordering a part, making a safety-critical repair decision, or mutating a record.

Types should narrow ambiguity. An enum is stronger than an unexplained status string. An array of evidence identifiers is stronger than citations buried in prose. A nullable field is not the same as an omitted field. Schema version is not a comment; it is part of behavior identity and compatibility.

But types can also give false confidence. A field named confidence invites generated numbers that look calibrated. A field named approved invites software to confuse proposal with authority. Prefer fields that represent observable application states: missingEvidence, conflictingEvidence, proposalState, escalationReason, and validationErrors.

The validation ladder

Run every generated candidate through distinct gates. Preserve which gate failed.

Gate 1: decode or transport

Did the provider/runtime return a complete response within the declared boundary? Transport timeout, truncation, provider refusal, and malformed byte handling belong here. A partial object is not silently promoted to a proposal.

Gate 2: parse

Can the response be decoded as the expected serialization? A fenced code block containing JSON is not necessarily the same interface as raw JSON. Remove no text unless the repair policy explicitly permits it and records the transformation.

Gate 3: structural schema

Are required fields present? Are types, enums, bounds, formats, and additionalProperties rules satisfied? Does the object declare the compatible schema version?

Schema-constrained generation can improve syntactic conformance to supported JSON Schema. Official structured-output documentation shows that managed APIs can restrict generation to supported schema subsets. Open-weight runtimes may expose grammar or constrained-decoding mechanisms. Support varies by API, model, runtime, and schema feature; constrained generation does not eliminate the remaining gates. [CLM-019]

Gate 4: semantic invariants

Do the fields agree with the task contract? Examples:

• state=abstain cannot include a confident repair proposal;
• an escalation must name a permitted reason and target class;
• a proposed next step cannot be phrased as a completed or authorized effect;
• an observation cannot contain a generated authorization decision;
• warranty and safety decisions remain outside the output vocabulary.

These checks are local product rules, not claims that software can determine every domain truth.

Gate 5: provenance and evidence

Does every evidence reference exist in the authorized bundle? Was it allowed for this case and tenant? Is it current enough for the claimed use? Does the referenced record contain material that could support the item?

Identifier existence is the cheapest check. Support is harder and may require a deterministic rule, human/domain judgment, or later calibrated evaluator. Keep those evidence types separate.

Gate 6: authorization and effect

Even a valid, evidence-backed proposal is not an effect. Application software checks the authenticated principal, resource, operation, scope, and current policy. Mosaic’s output path contains no effect adapter. A qualified reviewer may use the proposal within their authority.

Syntactic validity does not establish factual correctness, permission, provenance, or authorization. Structured-output and citation features help produce inspectable artifacts, while NIST guidance reinforces that technical and governance controls must be evaluated in context. None of them transfers domain or risk authority to the schema, model, provider, or LLM engineer. [CLM-020]

Long description

A bright three-dimensional output capsule moves through four distinct machines. The parse gate checks serialization, the schema gate checks field shapes, the semantic/provenance gate checks states and allowed evidence identifiers, and a final authority barrier keeps the result in a proposal display. Rejected capsules enter labeled evidence trays rather than disappearing.

Constrained generation and post-validation are complementary

When an access path supports native structured output, use it to reduce syntactic failure. When an open-weight runtime supports a compatible grammar, pin that grammar and runtime version. When neither exists, request the structure and parse defensively.

All three paths still emit the same provider-neutral semantic result and pass the same postconditions.

The managed adapter records the provider feature, supported schema subset, model identifier, and unavailable internals. The open-weight adapter records checkpoint, tokenizer/template, grammar/runtime, and hardware identity. A provider change or runtime upgrade triggers compatibility replay.

Do not weaken the common contract to the lowest provider feature. If a path cannot represent a required state or provenance field, it is incompatible until an adapter supplies an honest, testable mapping. Do not fabricate native constraint support where only prompt-following exists.

Version the schema as an interface

Schema change is system change.

Classify changes:

• compatible extension: an optional field with defined default and older-consumer behavior;
• breaking structural change: renamed field, changed type, new required field, or removed enum value;
• semantic change: the same field now means something different;
• authority change: a new field could influence a consequential operation;
• provenance change: evidence identity, source scope, or citation semantics change.

The last two require more than migration code. They reopen product, domain, privacy, security, and authority review as applicable.

Preserve the original raw response, schema version, validator version, normalized object, errors, and terminal state. Never rewrite old evidence into a new schema and discard the transformation record.

Make failure states explicit

The validator does not return only true or false. It chooses among bounded states under a versioned policy.

Valid

All required gates pass. The object may be displayed as an untrusted proposal to an authorized reviewer. Valid does not mean approved or correct beyond the validated claims.

Repair

A bounded, non-semantic defect may be repaired once under a declared rule. Examples might include a missing optional array or an allowed serialization wrapper. A repair produces a new trace linked to the original failure.

Do not use repair to invent evidence, change a prohibited state, or reinterpret an authorization field.

Abstain

Required evidence is absent or no evidence-supported proposal can be formed. The result records why and preserves relevant source state.

Escalate

Conflicting authoritative evidence, safety-critical ambiguity, or a decision outside Mosaic’s authority routes to a named human decision class with no automatic effect.

Fail closed

Malformed transport, incompatible schema, repeated invalid output, unauthorized evidence, unknown case identity, or policy breach stops the proposal path. The failure is observable and retry policy is exhausted or prohibited.

Abstention, escalation, repair, and fail-closed behavior should be explicit product states rather than improvised prose. This state machine is the book’s engineering pattern, not a universal standard. It must be tested against the local consequence, workload, retry cost, and named owners. [CLM-021]

Long description

A central validation hub sends proposal capsules into five differently shaped tracks. Valid reaches a review screen, repair loops once through a small tool, abstain ends at an empty-evidence marker, escalate reaches a human decision gate, and fail closed reaches a locked stop. None connects directly to an external action.

Bound retry before the first failure

“Retry until valid” converts a bounded generation problem into unbounded cost and latency. It can also repeat the same semantic error or create a different unsupported answer.

A retry policy names:

• eligible failure classes;
• maximum attempts;
• whether the same or revised request is used;
• cumulative time and cost budget;
• whether provider throttling or auth failures may retry;
• evidence retained from every attempt;
• terminal state after exhaustion;
• owner of the policy and change trigger.

For Mosaic, deterministic semantic or authorization failures never become valid through blind retry. A nonexistent evidence identifier triggers fail closed or abstention according to the case state. A safety ambiguity escalates; it is not regenerated until it sounds decisive.

Failure injection: the authorized-looking fabrication

Feed the validator the schema-valid example from the opening.

1. Parse passes.
2. Structural schema passes because nextSteps.proposal is a string and the state is an allowed enum.
3. Semantic validation fails because the proposal asserts an approval and scheduled effect.
4. Provenance validation fails because POLICY-999 is absent from the authorized evidence set.
5. Authorization remains false because no generated field can grant it.
6. The policy records fail closed with both errors and no effect.

The important evidence is not that “JSON hallucinated.” The configured system received untrusted generated content, and deterministic gates contained two distinct violations.

LLME-CASE-014 is used only for the untrusted-output and authority pattern. The threat landscape evolves, and the cited guidance provides no Mosaic prevention rate or security assurance. Treat model output as untrusted and keep authorization outside it.

Use the deterministic proposal validator

The companion adds a schema, fixtures, and validator chain under output/.

It tests:

• one valid proposal that remains effect-free;
• malformed serialization;
• structural schema failure;
• schema-valid but prohibited semantic content;
• nonexistent and unauthorized evidence references;
• explicit abstain and escalate results;
• one bounded repair followed by terminal failure;
• provider-neutral postconditions across managed and open-weight adapters.

The validator is deliberately small and local. It does not replace a full JSON Schema implementation, prove factual correctness, or authorize any action. Its job is to make the chapter’s gates executable without a provider call or secret.

Keep validation evidence disaggregated

A single “valid rate” can hide the failures that matter. Report the ladder by gate, state, segment, and consequence.

For each case set, retain at least:

• transport completion and truncation counts;
• parse and structural conformance;
• semantic invariant failures by rule;
• missing, nonexistent, unauthorized, stale, and unsupported evidence references separately;
• abstain, escalate, degraded, repair, and fail-closed dispositions;
• retry attempts and cumulative budget;
• language, appliance family, risk, and evidence-condition slices;
• evaluator identity and unresolved judgment;
• whether a candidate ever approached an effect boundary.

Suppose structural conformance improves from one synthetic run to another while fabricated evidence references increase. The output interface became easier to parse and less acceptable for the task. Do not average those observations into one improvement score.

Similarly, a provider-native constrained mode and an open-weight grammar may show different parse rates, but both still face the same semantic and provenance requirements. Compare the complete result, not only the gate each path makes easiest.

Validation events are evidence records. Include input/result identity, gate, rule version, observed field, disposition, limitation, and working owner. Do not store unnecessary sensitive text in the event. Privacy-approved raw traces remain separately controlled.

Distinguish repair from regeneration

A deterministic repair applies a known transformation to a bounded structural defect. Regeneration asks the model system for another candidate. They have different evidence and risk.

A deterministic repair might add an empty optional array allowed by the compatibility policy. It must record the original candidate and transformation. It cannot replace an unknown evidence ID or rewrite an approval statement into acceptable prose; those are semantic changes.

Regeneration creates a new stochastic observation and may change correct fields as well as defective ones. If allowed, it needs a new attempt identity and the same full validation ladder. Cap it before execution. A response that becomes parseable on the third attempt has not erased the two preceding failures or their workload cost.

Practice: classify before correcting

For each supplied fixture:

1. Record the raw candidate and behavior identity.
2. Identify the first failing gate.
3. Preserve every additional detected semantic, provenance, and authorization error.
4. Select valid, repair, abstain, escalate, or fail closed using the policy.
5. State whether retry is allowed and why.
6. Confirm that no state reaches an external effect.
7. Name the evidence and authority needed to change the disposition.

Then design one schema evolution. Classify it as compatible, structural, semantic, authority, or provenance change. Write the migration and requalification requirement.

Pass when all injected cases reach their specified terminal state, errors remain reviewable, evidence identifiers are checked against the authorized bundle, and a schema-valid object never acquires product authority.

The completed MD-03 typed baseline

Mosaic Desk completes MD-03 with:

• proposal schema version 0.1.0;
• provider-neutral typed result semantics;
• managed native-constraint and open-weight grammar adapter fields;
• parse, structural, semantic, provenance, and authorization gates;
• allowed evidence-set identity;
• uncertainty and escalation fields;
• valid, repair, abstain, escalate, and fail-closed states;
• bounded retry policy and preserved attempts;
• invalid and unsupported fixtures;
• no effect adapter and no generated authority.

The typed baseline is inspectable, not production-approved. Chapter 8 now allocates the finite context that feeds this interface. It must keep control, evidence, history, and output headroom visible, and it must never silently drop the evidence required to validate a proposal.