Build an Inspectable Vertical Slice

A successful provider call is not a vertical slice

Patchwork’s first demo accepts a query, calls a model, and prints a list. It works during the meeting. The developer’s credential is present, the network is responsive, the fixture is familiar, and every returned field happens to look plausible.

Nothing important has been proved.

The demo does not authenticate the actor, enforce purpose, filter source permission, validate candidate identity, apply incompatibility rules, represent abstention, bound timeout, prevent retry amplification, preserve trace evidence, or prove that invalid output cannot create a product or tool effect. It shows that an API can answer once.

An inspectable vertical slice is the smallest end-to-end path that carries the real obligations of a product behavior. It is thin in breadth, not shallow in consequence. It can run with synthetic data and a deterministic adapter, but it crosses the same logical boundaries that a release path would cross. It succeeds, refuses, abstains, degrades, and fails in named ways. It exposes enough state to locate the responsible layer.

Production-readiness research spans data, model, code, infrastructure, tests, and monitoring. A slice that exercises only the learned component omits the surfaces most likely to decide whether the behavior is dependable. [CLM-037]

Vertical, not horizontal

A horizontal slice implements one layer broadly. A team might build a complete retrieval service without an interface or a polished chat screen backed by mocked policy. Both can be valuable component work. Neither proves a user consequence path.

A vertical slice implements one narrow task through all required layers:

• request and identity;
• query schema and task scope;
• permitted synthetic data;
• retrieval and ranking;
• deterministic policy;
• optional provider adapter;
• structured result validation;
• user-visible state;
• trace;
• fallback;
• tool/effect boundary;
• tests.

Patchwork chooses one supported request: discover candidate common pump seals from explicit text and shaft diameter. It does not add accounts, production storage, real images, marketplace integrations, messaging, purchase, or autonomous tools. Those omissions are explicit.

Define what the slice must prove

PF-06 v0.1 has six proof obligations.

1. Local execution: default behavior needs no network, secret, paid service, or external account.
2. Stable interfaces: query, candidate, provider envelope, policy, result, trace, and tool proposal have versions and validators.
3. Visible states: success, clarification, abstention, degradation, prohibition, and relevant failure reasons are distinguishable.
4. Effect safety: invalid, partial, late, or unauthorized output produces no product result or external effect.
5. Evidence: one trace connects input, versions, transitions, policy, fallback, and zero-effect outcome.
6. Replaceability: an optional provider adapter can be added behind the stable seam without changing product authority.

Passing these obligations does not prove representative quality, scale, security authorization, domain correctness, or readiness. It gives the team something real enough to evaluate.

The repository map is part of inspectability

The companion keeps files by obligation:

companion/
  contracts/
    behavior-contract.schema.json
    applied-ai-interfaces.schema.json
    evaluation-case.schema.json
  data/
    generate-dataset.mjs
    patchwork-data-card.json
  evaluation/
    generate-cases.mjs
  lib/
    interfaces.mjs
    retrieval.mjs
    vertical-slice.mjs
    evaluation.mjs
  tests/
    chapter-03.test.mjs
    chapter-05.test.mjs
    chapter-06.test.mjs
    chapter-07.test.mjs
    chapter-08.test.mjs
    chapter-09.test.mjs
  run.mjs

The layout is deliberately plain. It avoids framework lifecycle and provider-specific abstractions so a reader can follow the state. A production codebase may use packages, services, generated types, and deployment configuration. The invariant is that contracts and effects remain visible.

Begin at ingress

The slice accepts a structured request object and actor context. The request includes an ID, family, text, explicit shaft diameter and unit, and optional constructed image features. The actor includes authentication and capability scopes.

Ingress performs checks in order:

1. authenticate the actor;
2. authorize patchwork:discover;
3. validate request shape and bounds;
4. check supported scope;
5. normalize only explicit values;
6. assign trace and version context;
7. enter retrieval.

Order matters. An unauthorized actor should not cause catalog retrieval or provider cost. An unsupported category should stop before broad candidate discovery. A missing unit should produce a contract state, not a default conversion.

Data validation belongs at system boundaries and must address more than JSON syntax. Structural checks can catch missing fields and type changes; semantic checks address units, permitted categories, source meaning, and training/serving assumptions. [CLM-038]

Use the deterministic reference path

The local adapter is not a fake answer pasted into a test. It implements the stable provider interface deterministically over the current authorized candidates.

For a success, it returns:

{
  "schemaVersion": 1,
  "adapterVersion": "local-0.1.0",
  "outputVersion": "0.1.0",
  "status": "ok",
  "candidateIds": ["PW-L001", "PW-L004"],
  "explanation": "Structured evidence is available for review; compatibility is not guaranteed."
}

The adapter can be instructed to return timeout or malformed output. Because it is deterministic, every reader can reproduce failure tests without internet access or provider drift.

The reference path teaches an architectural seam. It does not claim that a production model will have the same quality, latency, cost, or failure distribution. When a live adapter is added, it must conform to the envelope and pass its own evidence suite.

Validate output in two stages

Structured output reduces parsing ambiguity. It does not transfer application responsibility to a provider. Provider interfaces can constrain shape, while the application still owns business invariants, authorization, and effects. [CLM-039]

Stage one: envelope validation

Check schema version, adapter version, status, required fields, types, length bounds, unknown fields, and status-specific conditions. An ok envelope needs an output version and candidate array. A timeout envelope must not pretend to contain a complete result.

Stage two: state validation

Check every candidate ID against the exact current eligible set. Check policy version, source permission, freshness, deterministic exclusions, request generation, and allowed behavior claim. Validate explanation content against structured evidence. Check that the result’s state matches the behavior contract.

Suppose the adapter returns an ID from a previous request. The ID is syntactically valid and may exist in the catalog. It is not authorized for the current result. The slice abstains with UNVALIDATED_CANDIDATE and emits no result.

Suppose the adapter returns a valid candidate with the phrase guaranteed compatible. The candidate may remain usable in structured presentation, but the explanation fails semantic policy. Depending on configuration, discard the explanation and degrade to structured evidence, or block the entire result. Do not silently display the overclaim.

Make state transitions explicit

The slice records transitions rather than scattering booleans:

• received / REQUEST_ACCEPTED;
• authorized / DISCOVERY_SCOPE_PRESENT;
• respond|clarify|abstain|prohibited / RETRIEVAL_DECISION;
• degraded / DEPENDENCY_TIMEOUT;
• abstain / INVALID_PROVIDER_OUTPUT;
• respond / VALIDATED_STRUCTURED_RESULT;
• presented / NO_EXTERNAL_EFFECT.

The sequence makes impossible states testable. A request cannot become presented without authorized. An invalid provider response cannot have a later VALIDATED_STRUCTURED_RESULT. An authorization failure cannot include candidates. A timeout cannot carry a partial product result.

State machines also handle late responses. Each request needs an identity and generation. If correction creates a new request generation, a late old response is discarded. If the user navigates away, the result can be marked abandoned. A future queue implementation needs the same contract across processes.

The end-to-end trace

The trace is an execution artifact, not a dump. It records contract, interface, dataset, retrieval, policy, and adapter versions. It records safe candidate references, state transitions, reason codes, fallback, and effect attempts. It does not need full raw user or seller content.

Use trace IDs in errors and tests. A test should be able to say: this request failed at provider-envelope validation under interface 0.1.0, emitted INVALID_PROVIDER_OUTPUT, and recorded zero effects.

Build the success path without hiding limits

Synthetic request PW-Q001 asks for a common 16 mm seal. The actor is authenticated with discovery scope. Retrieval produces permitted, fresh candidates, removes denied and stale records, applies deterministic incompatibility, and deduplicates listing clusters. The local adapter echoes the eligible IDs in a valid envelope. Application validation verifies membership. Policy emits respond. The interface result carries candidates, evidence wording, and guaranteesCompatibility: false.

Success proves:

• the stable interfaces compose;
• permission and policy precede presentation;
• the local adapter is replaceable;
• the result remains structured;
• the user claim is bounded;
• the trace is correlated;
• effect count is zero.

It does not prove the ordering is useful to real buyers. Chapter 9 turns the consequence into cases; Chapters 10-12 build judgment and experiments.

Make abstention useful

The slice must abstain for a reason and preserve an allowed next action.

For a 999 mm query, retrieval finds no eligible candidate. The result is abstain / RETRIEVAL_ABSTAIN. It offers correction or ordinary search. It does not call the adapter to invent a listing.

For a missing unit, the behavior also abstains, but the reason differs. The interface can ask whether the measurement is millimeters or inches. For a missing measurement, the state is clarify, not generic failure. For an excluded family, the state is prohibited and the system does not broaden search.

These distinctions matter operationally. A spike in missing units calls for input design work. A spike in empty eligible evidence may indicate catalog coverage or permission failure. A spike in prohibited requests may indicate expectation or abuse problems. One fallback string would hide the layer.

Inject invalid output

The deterministic adapter’s invalid-output scenario omits schemaVersion and outputVersion, and supplies a string instead of an array. This is intentionally obvious so the contract mechanic is clear.

The application records validation errors, produces abstain / INVALID_PROVIDER_OUTPUT, and returns:

• no result object;
• no tool proposal;
• no external effect;
• a trace with the failing boundary;
• an allowed fallback disposition.

Production invalidity can be subtler: truncated arrays, unknown enums, non-finite scores, duplicate IDs, stale request IDs, plausible but unauthorized candidates, or semantic contradiction. The same rule applies: do not salvage unvalidated fragments into consequence.

Partial success is a product decision, not a parser convenience. If structured candidate evidence is independently validated before an optional explanation fails, the contract may permit a degraded structured result. If candidate production itself is partial and completeness matters, abstain. Name which part is optional.

Inject dependency timeout

The adapter’s timeout scenario returns a bounded failure envelope. The orchestrator does not wait indefinitely and does not display partial content. It emits degraded / DEPENDENCY_TIMEOUT.

Timeout behavior depends on component role:

• optional explanation timeout: show structured evidence if already validated;
• required ranking timeout: use an approved deterministic baseline or abstain;
• permission service timeout: fail closed for assisted evidence;
• trace sink timeout: follow the trace policy; never silently drop mandatory evidence;
• tool dependency timeout: do not assume effect failure or success; recover through idempotent status.

Retries can amplify load and duplicate cost. Distributed-systems guidance recommends bounded timeouts, backoff, jitter, and careful retry placement; learned calls and effects add semantic and cost concerns. [CLM-040] Patchwork v0.1 does not retry the local adapter. A future adapter must declare whether calls are safe to repeat and where the retry budget lives.

Do not retry every layer. If client, gateway, orchestrator, and adapter each retry three times, one request can multiply dramatically. Place retry at one owning layer and cap total attempts and time.

Inject authorization failure

Run the same valid request with an actor who lacks patchwork:discover. The slice terminates with prohibited / AUTHORIZATION_FAILED after ingress.

Assert:

• retrieval did not create a result;
• no provider call is represented;
• no candidate content appears;
• no tool proposal exists;
• effect count is zero;
• the trace contains only safe identity/scope metadata.

Authentication says who or what presented credentials. Authorization says whether that principal may perform this operation under current context. A service-to-provider credential does not satisfy user authorization.

Keep effects absent until earned

The slice implements proposeSellerQuestion, not sendSellerQuestion.

The proposal includes kind, candidate ID, question, effect: draft-only, and confirmed: false. Validation requires the supported kind, a bounded question, current authorized candidate, and the hard-coded non-effect state. The function returns effects: [].

Why implement even a draft? It lets the team exercise the proposal boundary and user editing without acquiring external risk. It also prevents a later team from assuming provider function-calling output is executable authority.

If messaging is proposed later, require:

• a new task and value case;
• actor and recipient identity;
• permission and scope;
• content validation;
• explicit confirmation over the final content;
• idempotency identity;
• delivery-state model;
• timeout and ambiguous-outcome recovery;
• rate and abuse controls;
• audit and retention;
• tests and bounded rollout;
• named product, security, privacy, and other applicable authority.

The simplest safe recovery from an unearned effect is not to build it.

The anatomy of a structured result

A Patchwork result needs:

• schema and result version;
• terminal behavior state and reason code;
• contract clause IDs;
• candidate IDs from the current eligible set;
• matched, unresolved, and conflicting evidence;
• source, freshness, and component versions;
• explanation mode and limitation;
• correction and fallback actions;
• trace reference;
• guaranteesCompatibility: false;
• empty effects.

Do not include a field named confidence without semantics. Is it model probability, rank score, calibration bin, evidence completeness, or policy state? Prefer named evidence dimensions. Chapter 10 will define consequential criteria and thresholds.

Four kinds of tests

Unit tests

Test pure rules: query normalization, unit handling, lexical score, vector-like score, permission decision, deterministic incompatibility, provider-envelope validation, and tool-proposal validation.

Unit tests localize logic. They do not prove boundary composition.

Contract tests

Test stable interfaces using valid and invalid examples. Require versions, reject missing fields, forbid unknown effect states, and verify adapters conform. A provider adapter can run these without the complete application.

Contract tests establish shape and declared semantics. They do not prove task quality.

Integration tests

Run request through authentication, retrieval, policy, adapter, validation, presentation result, and trace. Verify version correlation and candidate membership.

Integration tests prove components cooperate under fixtures. They do not prove representative use.

Failure-path tests

Inject abstention, invalid output, timeout, authorization failure, stale data, unknown permission, conflicting evidence, duplicate candidate, instruction-like text, and unavailable optional explanation. Assert terminal state, fallback, trace, and effect absence.

Failure tests are not leftovers after happy-path completion. For uncertain systems, they define the product.

Test evidence, not implementation trivia

A brittle test snapshots an entire generated string. A durable test checks that the explanation does not add facts, that required evidence remains visible, and that structured fallback works when prose changes.

A brittle test asserts private function order. A durable test asserts that unauthorized actors cannot reach retrieval results.

A brittle test expects one provider’s refusal text. A durable contract maps provider-specific statuses to stable application states and preserves the raw provider category only in bounded diagnostics.

A brittle test expects one ranking score. A durable test asserts that deterministic incompatibility dominates any score and that excluded IDs cannot appear.

Test the behavior contract and boundary invariants. Use implementation-specific tests only where the implementation itself carries risk.

Feature and configuration control

Feature flags can bound exposure and provide rollback. They can also create an untested state space.

Patchwork needs a small explicit configuration:

• retrieval mode;
• optional explanation enabled/disabled;
• adapter identity and version;
• timeout budget;
• structured fallback enabled;
• supported family set;
• contract and policy version.

Every supported combination needs a disposition. The default local configuration uses deterministic retrieval, local adapter, no external tools, and structured evidence. If explanation is disabled, behavior remains useful. If an adapter fails, fallback is explicit.

Do not let a flag bypass permission or deterministic policy. Controls should not be optional merely because implementation is feature-controlled. Record flag state in traces and evaluation runs.

Idempotency is semantic

An idempotency key does not make an unsafe operation safe by itself. It lets a component recognize repeated attempts under defined storage and time semantics.

For a read-only discovery request, duplicate computation mainly affects cost and trace. For a future seller message, duplicate execution affects another person. The effect gateway would need a client-generated operation ID bound to actor, recipient, final content hash, permission, and confirmation version. It would need a durable outcome state: not attempted, pending, confirmed, succeeded, failed definitely, or outcome unknown.

Never retry an outcome-unknown effect as if it definitely failed. First reconcile with the external system. Chapter 13 develops recovery semantics; this slice keeps all external effects at zero.

Queues and asynchronous work

The local path is synchronous, but the contracts anticipate asynchronous execution.

If ranking or explanation enters a queue:

• serialize schema and contract versions;
• bind the actor and purpose without copying unnecessary credentials;
• set an expiry after which the result is no longer useful;
• make duplicates detectable;
• make ordering assumptions explicit;
• reject work for superseded request generations;
• carry trace correlation;
• define cancellation and abandonment;
• validate again before presentation or effect.

A queue accepts work; it does not preserve semantic freshness automatically. By the time a result returns, permission, policy, source, or user input may have changed. Revalidate the conditions that matter at the last responsible boundary.

Observability begins inside the slice

The slice emits signals tied to the promise:

• request count by safe task segment;
• terminal behavior states;
• reason-code distribution;
• permission and freshness exclusions;
• candidate count and empty rate;
• adapter status and latency;
• schema and semantic validation failures;
• structured fallback rate;
• effect proposal and effect completion count;
• version dimensions;
• trace completeness.

Zero effects is an assertion in this version, not merely an expected average. A nonzero effect count should fail tests and stop release.

Avoid raw user text in routine metrics. Store only what a named diagnostic purpose justifies. Chapter 15 will develop privacy-conscious signals; the slice establishes the schema and redaction boundary.

Version from the first runnable commit

Version contract, interface set, dataset, generator, retrieval, policy, adapter, and evaluation suite. Record source revision or commit in execution evidence. Secure development, provenance, and traceable version practices begin during implementation, not when a team prepares launch paperwork. [CLM-041]

Versioning does not require pretending every fixture is stable. Mark PF-06 as 0.1.0, status draft, synthetic. A breaking interface change increments the relevant version and updates cases. A wording change may leave the behavior contract unchanged but still change the explanation component version.

Keep change histories meaningful. Latest is not evidence.

Specify every interface by behavior

An interface definition needs more than field names. For every field, record purpose, producer, consumer, allowed values, missing behavior, sensitivity, stability, and relationship to authority.

Query interface

The query carries current-task input. Its id correlates one attempt; it is not a user identity. family is selected from an approved scope list. text is untrusted and size-bounded. shaftDiameter preserves numeric value and explicit unit. imageFeatures, when present in the teaching fixture, are constructed values with a representation version.

The query contract must answer:

• Are unknown fields rejected or ignored?
• Are empty strings distinct from absent fields?
• Which Unicode and locale transformations occur?
• How are non-finite, negative, or extreme numbers handled?
• Is unit inference ever permitted?
• Which field identifies a correction or superseded request?
• Which actor and purpose are bound outside the payload?

Patchwork rejects ambiguous dimensions rather than choosing a locale default. It preserves the original value and adds normalized values only after explicit conversion.

Candidate interface

A candidate is evidence from retrieval, not an approved result. It includes candidate and canonical-cluster IDs, source and source version, synthetic status, provenance, freshness, permission state, channel scores, ordering score, matched fields, unresolved fields, and inert source content.

Candidate validation checks that:

• the source is eligible for actor, purpose, environment, and time;
• required provenance and versions exist;
• the representation is compatible with the query representation;
• the record is not stale under the behavior clause;
• deterministic incompatibility has not been detected;
• channel scores are finite and retain their semantics;
• duplicate clusters are resolved under a declared rule;
• source content cannot alter control instructions.

Ranked-result interface

The ranked result references candidates already validated. It may add order and reason evidence. It may not erase exclusions or manufacture candidate fields. If a ranker returns a new ID, validation fails. If it returns only part of the list, the application decides whether partial order is permitted; absence never silently means low rank.

Policy-decision interface

Policy emits a state, reason codes, contract clause IDs, evidence references, allowed next actions, prohibited effects, and policy version. Policy output is deterministic-control data only when it came from the approved executable policy under the recorded version. A string from a model saying state: respond is not a policy decision.

Explanation interface

Explanation is optional presentation material. It identifies the exact frozen evidence packet, style/version, provider status, supported claims, and limitation. It cannot select candidates, change policy, or add an effect. The application can delete it without deleting the product’s structured evidence.

Trace interface

Trace events include an event name, state, reason, timestamp or deterministic sequence, component version, safe evidence references, and access/retention class. They exclude secrets and unnecessary payloads. The trace validator rejects impossible transitions and unknown sensitive fields.

These behavioral definitions keep types from becoming decorative.

Design errors as first-class output

Throwing an exception can be appropriate inside a function. At a product boundary, map failure to a stable error or behavior state.

Patchwork distinguishes:

• INVALID_REQUEST: malformed or unsupported shape;
• AUTHENTICATION_FAILED: no valid identity;
• AUTHORIZATION_FAILED: identity lacks current capability;
• OUTSIDE_SCOPE: task or family prohibited;
• MISSING_REQUIRED_INPUT: clarification possible;
• AMBIGUOUS_INPUT: abstention or clarification required;
• NO_ELIGIBLE_EVIDENCE: retrieval completed but nothing can support the behavior;
• CONFLICTING_EVIDENCE: authorized facts disagree under no resolution rule;
• DEPENDENCY_TIMEOUT: dependency exceeded budget;
• INVALID_PROVIDER_OUTPUT: envelope or content validation failed;
• UNVALIDATED_CANDIDATE: output references evidence outside current state;
• PRESENTATION_UNAVAILABLE: structured or ordinary fallback may remain;
• TRACE_UNAVAILABLE: follow explicit audit policy;
• INTERNAL_INVARIANT_VIOLATION: stop and alert.

Each code maps to user behavior separately. A user need not see internal component names. They should receive an honest, actionable state: supply a unit, try ordinary search, review conflicting evidence, or understand that assisted results are unavailable.

Keep cause chains inside bounded diagnostics. If permission service timeout produced no eligible evidence, retain both the public state and internal cause without exposing restricted system detail.

Use an effect ledger

Even when the allowed effect set is empty, represent effects explicitly. The slice returns an effects array. Tests assert it is empty for success and every failure.

Why not simply omit tool code? Because a visible ledger gives future changes a hard seam. A proposal cannot become an effect accidentally without changing a contract and tests. Instrumentation can alert if effect count differs from the edition’s expectation.

An effect ledger entry for a future capability would need:

• operation ID and idempotency key;
• actor and delegated capability;
• target and environment;
• final validated argument hash;
• confirmation record;
• policy and component versions;
• attempt state and timestamp;
• external correlation ID;
• definite or unknown outcome;
• compensation/recovery state;
• audit access and retention.

Never store only success: false. Network failure after submission may mean the remote side completed the action. An unknown outcome requires reconciliation, not blind retry.

Prove that presentation is downstream

Teams sometimes validate the provider response and then let a UI template infer product meaning. The template may turn score into a percentage, hide unresolved fields, or show the first candidate under a best match badge.

Add presentation contract tests:

• every displayed candidate came from the current policy-approved result;
• matched, unresolved, and conflicting fields have distinct semantics;
• permission and internal exclusion details are not disclosed improperly;
• raw ordering scores are not presented as fit probability;
• abstention does not render an empty success card;
• degraded structured evidence is visually distinct from full behavior;
• correction creates a new request generation;
• stale responses cannot overwrite current state;
• the ordinary-search fallback does not inherit an assisted compatibility label;
• no hidden form submits a draft automatically.

The frontend is part of the AI system because it determines what the user can perceive and do. A correct policy object can still become harmful behavior through misleading presentation.

Walk all five outcomes boundary by boundary

Outcome 1: success

Input is valid and authorized. Retrieval returns permitted candidates. The adapter envelope conforms. Candidate IDs are members of the eligible set. Policy returns respond. Presentation shows structured evidence and limitation. Trace finishes NO_EXTERNAL_EFFECT.

Assertions:

• state equals respond;
• result schema validates;
• all result candidates exist in eligible retrieval output;
• guaranteesCompatibility is false;
• transition sequence contains authorization before presentation;
• effects are empty.

Outcome 2: abstention

The query is structurally valid but no eligible evidence supports it. Retrieval returns no candidates after deterministic controls. The adapter is unnecessary. Policy returns abstain. Presentation offers correction or ordinary search.

Assertions:

• state equals abstain;
• candidate and result arrays are empty or absent under the contract;
• no generator fallback occurs;
• reason identifies empty or ambiguous evidence;
• effects are empty.

Outcome 3: invalid output

The adapter returns malformed or semantically invalid output. The boundary validator rejects the entire provider result. The orchestrator records errors. No partial candidate is rendered and no tool proposal is produced.

Assertions:

• state equals abstain;
• reason equals INVALID_PROVIDER_OUTPUT or the narrower validation code;
• result is absent;
• validation errors identify the interface without copying unsafe payloads;
• effects are empty.

Outcome 4: timeout

The dependency exceeds its budget. The orchestrator marks the call terminal. If independently validated structured evidence is an approved fallback, the state may degrade and show it. The teaching implementation returns degraded without a partial result.

Assertions:

• state equals degraded;
• reason equals DEPENDENCY_TIMEOUT;
• no late output can change the completed response;
• retry count stays inside budget;
• effects are empty.

Outcome 5: authorization failure

The actor is known but lacks discovery scope. The request stops before retrieval and adapter execution.

Assertions:

• state equals prohibited;
• reason equals AUTHORIZATION_FAILED;
• no candidate/result content is returned;
• trace does not disclose protected data;
• effects are empty.

Together these tests define the slice more accurately than its success screenshot.

Treat test doubles as production contracts

A poor test double always returns the answer the test expects. A useful deterministic adapter models the provider contract’s meaningful states.

It should support:

• valid structured output;
• invalid schema;
• semantically invalid candidate;
• empty output;
• explicit refusal where relevant;
• timeout;
• partial/truncated response;
• version mismatch;
• deterministic latency or cost metadata if budgets are tested.

Keep the test double simpler than a real provider. It need not simulate model quality. It must be honest about which failure mechanics it exercises.

Contract tests should run against every optional live adapter using recorded non-secret fixtures or a controlled sandbox. A live adapter passing them establishes boundary conformance under the tested version, not product quality.

Secure the implementation path

The vertical slice is the first point where abstract threats become code review questions.

Dependencies and provenance

Pin and review packages under repository policy. Record source revision. Avoid dynamic downloads in the default run. A locally reproducible companion reduces supply-chain and version ambiguity but does not eliminate it.

Secrets

Default execution uses none. Optional provider adapters read credentials through approved runtime configuration, never source or fixture files. Tests verify that missing credentials disable the optional adapter rather than crash the structured path.

Input limits

Bound text, arrays, numeric ranges, and nested depth before expensive work. Reject malformed Unicode or binary content according to the interface. Input bounds protect reliability as well as security.

Output limits

Bound provider response size and parse incrementally only when partial semantics are explicitly supported. Reject non-finite numbers, unknown enum values, duplicate IDs, and unexpected fields.

Isolation

Provider adapters receive minimal data and no tool credentials. Seller content is encoded as data. The tool gateway has no external connector in v0.1. Trace access is separate from product access.

Failure disclosure

User errors avoid stack traces, source existence leaks, provider secrets, and denied candidate titles. Internal records retain enough safe context for diagnosis.

These controls are implementation evidence. Security authority still reviews their adequacy for a real environment.

Review code by boundary

Ordinary code review often follows files. Applied AI review should also follow consequences.

For one change, ask:

1. Which behavior clauses can change?
2. Which trust boundary receives new information?
3. Which state transition is added or altered?
4. Can permission be bypassed or evaluated too late?
5. Can output become an effect?
6. Which failure can now propagate farther?
7. Which trace fields or sensitive data change?
8. Which component, contract, dataset, evaluator, and suite versions must increment?
9. Which tests replay, and which new cases are required?
10. Which owner corrects failure and which authority approves consequence?

A one-line mapping change can alter semantics more than a large refactor. Review diff size is not risk size.

Put evidence in continuous integration

The smallest CI gate for the companion runs:

• JSON parsing and publication validation;
• contract validators;
• deterministic data generation and leakage checks;
• retrieval unit tests;
• boundary and interface tests;
• five vertical-slice scenarios;
• evaluation case validation, split hashing, coverage, and contamination tests;
• site build and reference checks;
• diff whitespace checks.

The gate should fail on nonzero external effects, missing versions, an uncovered required invariant, or changed split hash without an intentional suite update.

Do not treat a green pipeline as release authorization. It proves the automated checks under the recorded environment. Review, evaluation, operational evidence, and authority remain separate.

Store test manifests with component versions and seeds. If a result cannot be reproduced after a dependency update, the evidence has weakened even if the new run passes.

Know when to widen the slice

Expand only when the current path answers its question and the next uncertainty can change a decision.

Possible next increments:

• add a second supported family to test scope abstraction;
• add explicit image-feature input to test multimodal candidate generation;
• add optional explanation to test semantic validation and fallback;
• add a real provider adapter in a controlled environment;
• add persistent state to test correction races and recovery;
• add a review queue for conflicting evidence;
• add bounded shadow traffic with authorized synthetic or replay input.

Each increment brings new cases and authority. Do not expand UI polish while permission failure remains untested. Do not add tools while result semantics remain unstable. Do not scale infrastructure before one request’s state can be explained.

The vertical slice is a decision instrument. Its width follows uncertainty, not backlog enthusiasm.

Preserve the ordinary product path

AI assistance should not erase the baseline. Patchwork retains ordinary catalog search and filters as a distinct fallback. The fallback has its own promise: it displays catalog results without an assisted compatibility implication.

Test that:

• assisted failure does not break ordinary search;
• ordinary search does not inherit AI labels or scores;
• permission still applies;
• users can tell which mode they are in;
• switching modes preserves or clears inputs intentionally;
• telemetry distinguishes fallback use from assisted success;
• a feature disable returns users to a useful path.

This makes rollback a product behavior rather than an infrastructure toggle.

Write the slice limitation statement

A complete implementation includes a readable limitation statement:

PF-06 v0.1 is a local, deterministic, fictional/synthetic teaching slice. It validates declared interfaces, behavior-state transitions, selected permission and policy mechanics, five failure scenarios, trace correlation, and zero external effects. It has no real catalog, user, model/provider, compatibility, scale, latency-tail, cost, fairness, security-authorization, or release evidence. Structured success means the fixture passed the implemented contract, not that a part fits.

Place the statement beside runner output and dossier evidence. A reader should not need to infer the limits from source code.

Prepare a failure evidence packet

For each injected failure, keep a compact record: scenario and purpose; input and actor fixture references; contract, interface, data, policy, adapter, and code versions; expected and actual transition sequence; result/effect assertions; fallback; trace completeness; owner; limitation; and next test.

The invalid-output packet shows the malformed fixture, validation errors, absent product result, and zero effects. The authorization packet proves no candidate path executed. The timeout packet records the budget and proves a late result cannot change terminal state. Compare packets across changes. If a new adapter maps timeout to generic failure or a framework adds automatic retry, the behavioral difference is visible.

Exercise semantic invalidity

Syntactic failure is easy. Add cases whose JSON validates but meaning does not:

• an ID exists globally but not in the current authorized set;
• a source version is stale under the current request;
• a candidate family changed through an alias;
• explanation says fits while policy says unresolved;
• candidate vectors use incompatible representation versions;
• result and trace name different policy versions;
• a tool proposal invents a recipient;
• a valid request has been superseded by correction.

Fail each case at the owning boundary. Candidate membership belongs at result validation. Explanation meaning belongs at presentation. Superseded state belongs at orchestration. Recipient authority belongs at the tool gateway. Semantic tests often carry more product value than another schema example because dangerous values can be perfectly well formed.

Test degraded behavior as a product mode

Patchwork distinguishes full assisted evidence, structured degradation without optional prose, ordinary search without assisted compatibility, abstention, and prohibition. Test labels, controls, telemetry, and transitions for each. Ordinary search must not use the visual promise of evidence-backed assistance. A recovered service invites a fresh request; it does not silently rewrite what the user already saw.

Degraded modes need capacity ownership. If every timeout shifts traffic to a slower fallback, that fallback can collapse. Chapter 14 quantifies budgets; the slice names the dependency and safe terminal state.

Verify deterministic replay

Run the same input, dataset version, components, configuration, and seed twice. The reference path should reproduce state, candidate IDs, reasons, and split hash. Trace IDs or timestamps may differ under a documented canonical comparison.

For nondeterministic providers, record settings and run repetitions appropriate to the decision. Stable application assertions still hold when wording varies. If local replay changes, inspect source revision, dependency lock, runtime, fixture generator, environment, and ordering before treating the run as regression evidence.

Define completion without pretending launch

The slice is complete here when a new reader can run it from one command; interfaces reject invalid examples; all five outcomes are deterministic; invalid or partial output creates no result/effect; the draft remains reversible; traces expose the boundary safely; structured behavior survives optional-provider removal; tests and limitations travel with code; and a backlog records scale, representative quality, controls, and release work.

This is a technical milestone, not a market or safety claim. Its value is that every later evaluation has a real inspectable object.

Conduct a slice review from the user’s screen backward

Begin with each visible state and trace to its evidence.

For respond, point from every candidate field to the permitted source and version, from every label to a policy state, and from every explanation sentence to structured evidence. For clarify, show that the requested field is required and that providing it creates a new request. For abstain, prove no candidate or generated substitute is hidden behind the message. For degraded, identify exactly which promise became weaker. For prohibited, prove expensive or sensitive downstream work never started.

Then inspect controls the user cannot see: authorization, source eligibility, deterministic conflict, adapter validation, effect ledger, and trace redaction. A reviewer should be able to explain why an attractive high-score candidate was excluded without exposing its denied content.

Finally, reverse one condition at a time. Grant discovery scope; add an explicit unit; repair permission metadata; restore the optional adapter. The state should change only through allowed transitions. If toggling one condition causes unrelated differences, the slice has hidden coupling.

Preserve a diagnostic replay bundle

For a failed case, capture a minimal local bundle: sanitized request fixture, actor scopes, referenced synthetic records, configuration, contract/interface/component versions, deterministic adapter scenario, expected state, and trace projection. The bundle must contain no production secret or unnecessary personal content.

Replaying the bundle should reproduce the relevant failure. If it does not, identify missing environmental state rather than declaring the issue intermittent. A production replay system may need protected access and stronger data controls; the companion teaches the shape with safe fixtures.

Use the bundle in code review and regression. When a bug is fixed, keep the case in development regression and acquire fresh release evidence where independence matters.

Know what not to add

Do not add a database merely to look production-like if no state requires persistence. Do not add a queue until asynchronous behavior answers a real uncertainty. Do not add a live model to validate contracts already testable locally. Do not add a sending tool before a draft creates demonstrated task value. Do not add raw payload logging because diagnosis feels difficult.

Every omitted component reduces failure surface. Every included component must earn its place through a behavior clause or an evidence need. This is how a thin slice remains production-shaped without becoming a premature platform.

Document deferred concerns beside the slice: representative data acquisition, domain validation, evaluator calibration, adversarial testing, tail latency, concurrency, capacity, cost, accessibility, localization, privacy review, security authorization, incident response, rollback, and real-user research. Give each an owner or explicit unknown and name which later phase addresses it.

Deferral is honest only when the current claim excludes the missing evidence. If the demo is shown outside the team, carry the limitation statement with it. A polished interface must not silently upgrade a local fixture into a product promise.

The slice is now ready for evaluation because its observable states, inputs, versions, and failures are stable enough to become cases. That readiness is the exit condition, not feature completeness.

Local execution walkthrough

Run the companion from the repository root:

node content/publications/applied-ai-engineering/companion/run.mjs
node --test content/publications/applied-ai-engineering/companion/tests/*.test.mjs

The runner validates the behavior contract, generates synthetic data, verifies its split mechanics, runs retrieval, executes the success slice, generates evaluation cases, hashes the split, checks contamination, and reports visible coverage gaps.

The Chapter 8 tests execute:

1. success with structured evidence and zero effects;
2. empty-evidence abstention;
3. invalid provider output with no result or effect;
4. dependency timeout with degraded state and no partial result;
5. authorization failure before retrieval result or effect;
6. reversible unconfirmed seller-question draft.

The output is evidence about the checked code and fixtures. It is not a production-readiness badge.

Diagnose failures with the slice

Suppose the success test begins abstaining after a dataset change.

Inspect the trace:

• authorization passed;
• query validation passed;
• candidate count became zero;
• permission exclusions increased;
• dataset version changed;
• adapter was never called.

The responsible layer is data/context eligibility, not provider quality.

Suppose candidates remain stable but the result abstains with INVALID_PROVIDER_OUTPUT. Compare adapter and interface versions. Inspect structural errors. Do not tune retrieval.

Suppose the result is valid but the interface omits unresolved evidence. The failure is presentation integration. Candidate quality is irrelevant to that defect.

Suppose a timeout produces a candidate card. The trace reveals an illegal state transition. Fix orchestration before measuring task quality.

Inspectable state turns an end-to-end symptom into a discriminating next check.

Avoid the framework-shaped slice

Frameworks can accelerate routing, schema generation, tracing, and adapter integration. They can also hide state, retry, memory, and tool behavior behind defaults.

Before adopting one, answer:

• Can every contract be expressed independently of the framework?
• Can local deterministic tests run without provider initialization?
• Are retry and timeout policies explicit?
• Is tool execution separate from model output?
• Can request state and versions be inspected?
• Can a provider be removed without deleting product behavior?
• Can traces avoid copying sensitive payloads?
• Can failure states map to the behavior contract?

If not, the framework may be useful for a prototype but has not earned ownership of the production slice.

Apply the slice to a non-language system

A predictive replenishment slice can use the same pattern:

• synthetic inventory features enter a validated request;
• a deterministic test double returns a forecast range;
• application policy checks data freshness and item scope;
• output validation rejects missing horizon or non-finite values;
• the interface shows the forecast and existing baseline;
• no order is placed;
• timeout falls back to the current planning method;
• tests cover stale data, invalid values, authorization, and zero effects.

No prompt, retrieval, or conversation is required. The vertical-slice method belongs to applied AI systems generally.

PF-06 running slice v0.1

The dossier records:

• repository map and local commands;
• interface-set version 0.1.0;
• deterministic data and adapter versions;
• request, candidate, result, policy, trace, and tool-proposal contracts;
• success, abstention, invalid-output, timeout, and authorization scenarios;
• structured fallback;
• zero external effects;
• test results;
• known limitations;
• owner and change triggers.

Limitations remain prominent:

• all records and outcomes are synthetic;
• retrieval quality is not representative;
• domain compatibility is not validated;
• no scale, tail-latency, cost, or capacity evidence exists;
• threat and control tests are incomplete;
• no real user interaction has been evaluated;
• no provider adapter is approved;
• no release authorization exists.

The slice is worthy because its limits are visible.

Practice: prove the failure path

Implement one behavior slice in your own system. Use a deterministic provider or model test double. Preserve real schemas, policy, state, fallback, trace, and effect boundaries.

Required inputs:

• behavior contract;
• combined-system boundary;
• synthetic data with permission and failure states;
• stable interface definitions;
• named actor and capability;
• one permitted user consequence.

Required outputs:

• runnable local command;
• repository map;
• success and failure traces;
• contract, integration, and failure tests;
• evidence that invalid or partial output creates no effect;
• limitations and adjacent-role handoff.

Inject success, abstention, invalid output, timeout, and authorization failure. For each, assert terminal state, reason code, permitted fallback, trace fields, result presence or absence, and effect count.

Do not ask a real provider to behave badly and hope the event occurs. Make failure deterministic. Do not connect a real external tool merely to prove a boundary. Use a proposal and effect ledger.

The adjacent-role limit remains: the Applied AI Engineer implements the path and gathers evidence. Platform, security, privacy, product, and domain authorities approve their respective production obligations.

Handoff to evaluation cases

PF-06 gives Chapter 9 an executable object. We no longer need to speculate about what one request does. We can construct cases around contract clauses, segments, data states, mechanisms, interactions, and consequences.

The next task is not to collect many rows. It is to decide which failures this slice must reveal, which population claims remain unsupported, how development and release evidence stay separated, and how the evaluation set changes without losing provenance.