Choose the Simplest Adequate Mechanism

Capability does not choose architecture

Patchwork’s behavior contract is finally precise enough to make a mechanism discussion useful. The product must retrieve evidence-backed candidates, preserve deterministic incompatibility rules, distinguish matched and unresolved attributes, clarify or abstain when critical evidence is absent, retain ordinary search, and prohibit automatic seller contact or purchase.

Now imagine a mechanism review. One engineer proposes a large language model because buyers use natural language. Another proposes an agent because the task includes several steps. A third proposes vector search because listing vocabulary varies. A fourth proposes a specialized ranker. Each proposal is plausible. None follows from the task by itself.

The mistake is to treat capability categories as product architectures. A model can generate language, but Patchwork first needs candidates from authorized catalog evidence. An agent can call tools, but the contract prohibits the external actions that would justify most tool autonomy. An embedding can represent semantic similarity, but similarity does not enforce dimensional compatibility. A deterministic rule can exclude known mismatches, but it cannot recover candidates expressed in unfamiliar language.

The engineering question is not, “Which mechanism is most capable?” It is:

What is the least complex combination that can satisfy the current behavior contract for the supported segment, with evidence and operations proportionate to the consequence?

Least complex does not mean fewest lines of code or oldest technique. It includes the total burden of data, calibration, latency, capacity, cost, permissions, attack surface, monitoring, incident response, provider change, user interaction, and fallback. A managed model API may take one line to call and still create a large product obligation. A familiar retrieval service may be easier for one team to operate than a custom rule engine. Simplicity is contextual.

Experienced ML engineering guidance treats simple rules and pipelines as legitimate starting points, not embarrassing prototypes. A mechanism should beat a credible baseline on the decisions the product needs to make. [CLM-016]

A mechanism ladder without a maturity story

The following mechanisms are options, not stages of sophistication. A team can combine them, reject them, or return to them after evidence changes.

Deterministic rules

Rules express known invariants and exact transformations: normalize punctuation, convert an explicit unit, require a field, enforce a permission, reject an out-of-range dimension, or prevent a tool call.

Rules are strong when semantics are stable and authority is explicit. They are inspectable, cheap to replay, and suitable for prohibited effects. They become fragile when they attempt to enumerate open-ended language or when exceptions outgrow the domain model. A large rule set can be operationally more complex than a learned component.

For Patchwork, deterministic controls own known incompatibility, permission, allowed catalog family, required evidence, and non-autonomy boundaries. A ranking or generative component cannot override them.

Lexical and structured retrieval

Lexical search matches terms and fields. Structured retrieval filters by catalog family, explicit measurements, materials, status, permission, and other stable attributes. It is excellent for exact part numbers and interpretable constraints. It can miss synonyms, descriptions, and cross-modal resemblance.

An improved lexical baseline matters because it represents what careful ordinary software can achieve. Normalization, catalog aliases, field-aware scoring, and structured filters may solve much of Patchwork’s task without learned behavior.

Predictive classification or scoring

A classifier or scorer estimates a task-specific label or score from features. It may identify a catalog category, detect an incomplete listing, or estimate candidate relevance. Its value depends on labels, population, calibration, and how the score changes product behavior.

A predictive model is not automatically simpler than retrieval. It may require a label pipeline and threshold policy that the task does not support. Conversely, a small calibrated classifier can be more controlled than a broad generative prompt.

Dense and multimodal representation

An embedding maps text, images, or other inputs into a representation used for similarity and retrieval. Multimodal representations can support image-to-listing or text-to-image discovery without chat, generation, or autonomy. Research and first-party product cases show this path is technically real, while their datasets and outcomes remain specific to those systems. [CLM-018]

Patchwork can use a buyer photo or description to find visually or semantically related listings. That result is a candidate set. It is not a compatibility decision. The application must still apply structured filters, evidence states, and domain rules.

Ranking

A ranker orders a candidate set using query, listing, context, and possibly user signals. Separating candidate generation from ranking lets the product seek broad recall first and spend more computation on a smaller set. Search-system cases illustrate this architecture and the difficulty of interpreting implicit feedback. [CLM-019]

A ranker needs its own contract: what enters the set, which features are permitted, what score means, how ties and missing fields behave, and which deterministic constraints it cannot overrule.

Generative transformation or explanation

A generative component can translate a query into structured fields, summarize catalog evidence, explain why a candidate appears, or draft a question. It produces new text or structure, creating failure surfaces that retrieval alone does not: fabrication, unsupported implication, instruction conflict, unstable wording, and output-to-effect translation.

Generation can improve comprehension while remaining downstream of frozen evidence. It should not invent candidates or create compatibility evidence. Retrieval and generation solve different problems and deserve separate evaluations. [CLM-019]

Tool use and agents

A model can propose or invoke functions and participate in a multi-step loop. The interface may be powerful. It does not supply authorization, validation, idempotency, confirmation, rollback, or a product reason for autonomy. Provider documentation describes how function calls can be represented; security guidance identifies excessive agency as a risk. Neither proves that an agent belongs in Patchwork. [CLM-020]

Patchwork’s current seller-question behavior permits an editable draft after explicit user request. It prohibits sending. A tool that sends messages has no justified role. Calling the workflow an agent would add a sophistication label without adding allowed task value.

Hybrids

Most dependable systems combine mechanisms. Patchwork’s proposed first path is hybrid:

1. deterministic scope, permission, normalization, and incompatibility controls;
2. lexical, structured, and multimodal candidate retrieval;
3. deterministic feature construction and bounded ranking;
4. evidence-state assembly for the interface;
5. clarification, abstention, and ordinary-search fallback;
6. optional generation over frozen evidence only if it wins a separate experiment.

The architecture is not less AI because deterministic controls surround a learned representation. Those controls make the uncertain component useful.

Compare mechanisms against a tradeoff surface

A single benchmark number cannot select a product mechanism. Evaluate at least twelve dimensions.

Task fit

Which contract clauses can the mechanism satisfy? Which does it leave to other components? A language model may interpret a description but cannot by itself guarantee that candidates come from the authorized catalog.

Evidence availability

What data, labels, context, domain rules, and reviewer judgments are required? If the mechanism needs a compatibility label that the domain cannot define, its apparent capability is irrelevant.

Error consequence

How do false inclusion, false exclusion, ordering error, fabrication, stale evidence, and unauthorized effect differ? The mechanism must support the product’s failure disposition, not only an average metric.

Calibration and thresholds

Can scores support a stable accept, clarify, or abstain policy for material segments? A model with higher average accuracy can be worse if its score cannot guide the required state transitions.

User interpretability and correction

Can the product expose why a result appeared, what evidence is missing, and how the user can correct it? Interpretability here is task interaction, not a universal claim that one model class is explainable.

Latency distribution

Does the entire path fit the task’s timing, including tail behavior? Large services can experience amplified tail latency when a user request fans out to many components. The exact mitigation depends on topology, but averages alone are inadequate. [CLM-017]

Capacity and cost

What happens at representative and burst load? Include inference, indexes, storage, review, retries, evaluation, observability, and change testing. Price today is a volatile input, so keep it in a versioned decision record rather than a durable mechanism ranking.

Privacy and permission

Which data leaves which boundary? Does the option need user history, images, private text, or provider retention? A mechanism requiring unjustified data fails regardless of quality.

Attack and effect surface

Can untrusted listing text influence generated instructions? Can the mechanism reach tools? Does it create new paths to expose data or perform actions? Deterministic retrieval is not risk-free, but generation and tools change the threat model.

Operational ownership

Can the team detect, diagnose, degrade, recover, and update the mechanism? A “simple” custom model can be hard if nobody owns training and monitoring. A managed service can be hard if its changes and failure semantics are opaque.

Change and portability

Which behavior evidence must be replayed after a catalog, model, provider, prompt, index, feature, or interface change? Can the mechanism be replaced behind a stable contract, or has the product inherited provider-specific semantics?

Fallback and option value

Can the team remove or disable the mechanism without destroying the task? A reversible component preserves option value while evidence is weak. Patchwork’s ordinary search and structured evidence state are not temporary scaffolding; they are fallback assets.

Broad evaluation frameworks demonstrate why multi-scenario and multi-metric evidence is necessary, but a general benchmark cannot decide Patchwork’s product criteria. The team must construct the tradeoff surface from its contract and consequence. [CLM-017]

Do not collapse component and product metrics

Metrics describe a method and claim. They do not automatically express value.

For candidate retrieval, measure whether a relevant candidate appears within a bounded set. For ranking, measure ordering among candidates using task-specific judgments. For classification, distinguish error types, segments, and threshold behavior. For explanation, evaluate evidence faithfulness, unsupported implication, comprehension, and correction. For tools, verify authorization and effects.

Official metric documentation is useful for defining precision, recall, ranking measures, calibration, and other calculations. It cannot choose which error matters or where a threshold belongs. [CLM-016]

Patchwork needs at least four connected result layers:

1. candidate retrieval: did authorized plausible candidates enter the set?
2. ranking: did evidence-supported candidates appear in a useful order?
3. behavior state: did the application respond, clarify, abstain, degrade, or prohibit correctly?
4. task interaction: could the buyer reach a defensible candidate set relative to baseline without a stronger unsupported claim?

A retrieval improvement can coexist with a product regression. More candidates can increase recall and overwhelm the user. A ranker can improve an offline ordering score while placing unresolved evidence under a confident visual treatment. A fluent explanation can increase perceived clarity while reducing correction.

The mechanism decision record must say which layer each result supports and what it cannot prove.

Build an experiment ladder

An experiment ladder spends evidence cost in the order most likely to change the architecture.

Step 0: stop gate

Verify that the task-data audit can define required attributes, permission, and supported segments. If it cannot, do not compare advanced mechanisms. Chapter 5 will execute this gate.

Step 1: credible deterministic baseline

Implement normalized lexical search, catalog aliases, structured filters, explicit-unit conversion, and domain-approved exclusions. This is a product candidate, not a straw baseline.

Step 2: lexical retrieval plus bounded ranker

Separate broad candidate retrieval from a deterministic or simple learned ordering. Measure the incremental task effect and inspect missing-field behavior.

Step 3: multimodal and dense candidate retrieval

Add text and synthetic image-feature similarity behind the same candidate interface. Keep authorization and evidence assembly unchanged. Compare which task segments gain candidates and which gain noise.

Step 4: hybrid portfolio

Combine lexical, structured, and representation-based candidate sources, deduplicate them, apply deterministic exclusions, and rank. Verify provenance for every inclusion and exclusion.

Step 5: evidence presentation

Test structured matched, mismatched, and unresolved attributes without generated prose. This isolates whether comprehension gains come from information architecture.

Step 6: generated explanation

Freeze candidate evidence and ask whether a generated explanation improves comprehension or correction beyond a template. It must pass unsupported-claim, variability, latency, cost, change, and degraded-mode gates independently.

Step 7: seller-question draft

Compare a deterministic editable template with a generated draft. The draft never receives send permission. Evaluate provenance, missing-fact focus, pressure, user correction, and refusal of unsupported requests.

There is no automatic Step 8 for an agent. A future task would need to show that a bounded action creates value that the confirmed-draft workflow cannot, and it would require a new authority and effect-control contract. Tool capability alone is not evidence. [CLM-020]

Option value and reversibility

Early mechanism choices create path dependence. A conversational interface can make users expect unrestricted language behavior. A provider-specific response format can spread through application code. A personalization feature can create data collection and retention obligations. An agent can create tool permissions that become hard to remove.

Preserve option value with stable internal contracts:

• a candidate has an ID, source, evidence fields, permission state, freshness, retrieval reasons, rank features, exclusions, and unresolved constraints;
• a behavior decision consumes candidates and emits an explicit state;
• presentation consumes that state without knowing which provider produced a score;
• optional generation consumes a frozen evidence packet and cannot alter candidate selection or effects;
• external tools remain behind application authorization and explicit confirmation.

This separation lets Patchwork replace a representation model, ranker, or generator without rewriting its user promise. It also lets the team remove generation while retaining retrieval and structured evidence.

Reversibility is not free. Maintaining interfaces, fixtures, and comparison paths costs engineering time. The cost is justified when evidence is weak, providers change, or consequence makes a silent dependency dangerous. Later chapters will make change replay operational.

Six mechanism portfolios for Patchwork

The team compares portfolios, not isolated technologies.

Portfolio A: improved ordinary search

Components: normalization, lexical search, category aliases, structured filters, deterministic exclusions, evidence display.

Strengths: inspectable, low generation risk, strong exact lookup, clear fallback.

Limits: vocabulary and image mismatch; manual comparison burden; limited open-ended interpretation.

Decision: retain as mandatory baseline and fallback.

Portfolio B: predictive compatibility classifier

Components: feature model predicting compatibility or fit.

Strengths: potentially compact decision score.

Limits: Patchwork lacks an authorized, representative compatibility label; a score may hide semantic and segment uncertainty.

Decision: reject for the first path. Reconsider only if Chapter 5 establishes labels and the contract defines a valid use.

Portfolio C: multimodal retrieval only

Components: lexical plus text/image representation retrieval, deterministic exclusions, evidence display.

Strengths: addresses vocabulary and visual discovery without generation; broadens candidates.

Limits: similarity is not compatibility; ranking and noise still matter.

Decision: include as an experiment behind the stable candidate interface. [CLM-018]

Portfolio D: hybrid retrieval and ranking

Components: structured, lexical, and representation retrieval; deduplication; deterministic exclusions; bounded ranker; evidence state.

Strengths: separates recall from ordering and preserves exact controls.

Limits: more components, latency, evaluation, and operational ownership; rank signals may encode weak proxies.

Decision: selected first learned path, conditional on task-data fitness.

Portfolio E: generative finder

Components: broad model receives query and catalog context and returns candidates plus explanation.

Strengths: flexible language transformation and presentation.

Limits: candidate provenance, unsupported synthesis, evaluation, latency, cost, injection, and provider-change burden are entangled.

Decision: reject as an integrated first path. Test explanation only downstream of frozen candidates.

Portfolio F: autonomous marketplace agent

Components: model searches, contacts sellers, selects a listing, and initiates purchase.

Strengths: would reduce steps if authorized and correct.

Limits: violates PF-B14; action value is unproven; authorization, payment, messaging, abuse, idempotency, confirmation, and recovery obligations dominate.

Decision: reject. It is outside the task and contract, not a later maturity level.

The selection is not a generic endorsement of hybrid retrieval. It is a Patchwork decision under the current task, data assumptions, authority, and fallback.

When the more capable model is worse

A model can dominate a benchmark and still be a worse product mechanism.

Suppose Model X finds more semantically similar listings than the baseline but has unpredictable tail latency, sends buyer text outside the approved boundary, and changes behavior through silent provider updates. Model Y finds fewer but enough candidates, runs inside the approved boundary, and produces stable scores. If Patchwork’s supported task values dependable evidence and correction over marginal recall, Model Y may be adequate while Model X is not.

Or suppose a generator explains catalog evidence beautifully but occasionally converts “similar dimensions” into “will fit.” A structured template is less expressive and satisfies the contract. Expressiveness is not quality when the additional variation creates the consequential error.

Mechanism selection is therefore a constrained frontier, not a leaderboard. The engineer must defend which tradeoff is acceptable for the task and state who authorizes the residual consequence. [CLM-017]

Define stable interfaces before comparing implementations

A fair experiment needs a stable object at each boundary. Otherwise every mechanism receives different inputs and emits different promises, and the team cannot identify why behavior changed.

Query request

The request should contain the user-entered query, explicit measurements with units, catalog scope, permission context, locale, and a trace/version envelope. It should distinguish absent values from inferred values. An image feature may be present, but the local companion uses deterministic synthetic vectors rather than real buyer media.

The request must not contain broad user history merely because a mechanism can consume it. Optional signals require their own purpose and evidence. The same request object should reach lexical, vector-like, and hybrid retrievers.

Candidate

Every retriever returns a candidate with:

• stable listing and source IDs;
• source version and freshness state;
• permission disposition;
• structured attributes and their provenance;
• retrieval channel and score;
• exact terms or feature evidence that contributed;
• unresolved and conflicting fields;
• deterministic exclusions;
• trace ID.

The candidate does not contain compatible: true. It represents evidence available for ranking and product-state decisions.

Ranked candidate

The ranker adds a bounded score breakdown or ordering reasons. It cannot remove provenance, convert uncertainty into fact, or restore an excluded candidate. If a learned score is used later, its version and calibration evidence belong in the trace.

Behavior decision

The decision layer emits one of the Chapter 3 states plus clause IDs, allowed candidates, exclusions, missing facts, and fallback. The interface renders this record. A generator, if enabled, can explain the frozen record; it cannot change the state or candidate set.

These interfaces permit a clean comparison. Lexical and multimodal paths can return candidates through one contract. A deterministic template and a generator can receive the same evidence packet. A provider adapter can be added later without becoming the application’s semantic center.

They also improve diagnosis. If a relevant listing never enters the candidate set, the failure is retrieval. If it enters and is ordered poorly, the failure is ranking or features. If it is ranked well and then incorrectly excluded, the failure is a rule or semantic mapping. If the decision is correct and the interface implies a guarantee, the failure is presentation. The team does not need to call every problem “model quality.”

Interpret mechanism experiments without laundering evidence

An experiment result should preserve the claim, population, method, result, uncertainty, limitation, and decision.

Suppose hybrid retrieval finds at least one domain-judged plausible candidate in more synthetic challenge cases than improved lexical search. That result can support a local mechanical claim: the implementation combines distinct signals and increases candidate coverage on the constructed set. It cannot establish real marketplace recall, user value, or financial outcome. Synthetic data was created to exercise behavior, not sample a real population.

Suppose a first-party engineering case reports that an image-search feature improved a product metric. That can demonstrate a feasible architecture and identify tradeoffs. It cannot prove that Patchwork has comparable images, users, catalog semantics, or outcomes. [CLM-018]

Suppose a language-model benchmark reports stronger performance on many scenarios. That can inform capability screening. It cannot waive Patchwork’s latency tail, permission boundary, candidate provenance, segment, interface, or provider-change requirements. [CLM-017]

Use an evidence transfer record when external evidence influences a mechanism choice:

Field	Patchwork question
Source claim	What exactly did the paper, documentation, or case observe?
Original context	Which data, model, task, product, and period produced it?
Transferable principle	What mechanism or tradeoff is plausibly durable?
Non-transferable result	Which reported number or outcome cannot be imported?
Local evidence	What must Patchwork test before acting?
Decision use	Does the source motivate an option, design a test, or support a threshold?

This protects the mechanism portfolio from citation theater. A source can justify why an option deserves testing without proving the option should ship.

Use paired cases

Compare portfolios on the same frozen cases and evidence. A paired view reveals which cases change state, not only whether an average moves. Inspect:

• cases found only by one retrieval channel;
• candidates whose order changes;
• cases where extra recall introduces unsafe noise;
• critical exclusions preserved or violated;
• empty and conflicting evidence behavior;
• latency and cost per task state;
• user-facing consequence of each difference.

Inspect wins and regressions separately

If a hybrid retrieves ten additional plausible candidates and introduces two unresolved listings into the first position, do not net them into one number. Classify the wins and regressions by consequence. A deterministic gate may repair the regressions; if it cannot, the added channel may need narrowing.

Preserve rejected alternatives

A rejection record contains the version, evidence, reason, and revisit trigger. “LLM was too slow” will become stale. “Provider/model V exceeded the task’s tail budget under load method M, and structured presentation matched comprehension criterion C; revisit when the budget or evidence changes” is durable.

The same applies to accepted mechanisms. Selection is conditional, not permanent endorsement.

The conditional autonomy test

Teams often move from generation to tools because the model can emit structured calls. The product decision should pass six gates before any adaptive action is considered.

1. Task-value gate

Name the user consequence improved by action rather than suggestion or confirmed execution. If an editable seller-question draft solves the task, automatic send has no earned value.

2. Authority gate

Identify who may authorize the action, for which resource, scope, and time. The model cannot be the authority. User confirmation may be necessary but is not always sufficient; organization and affected-party rules still apply.

3. Evidence gate

Define what evidence must exist before the action is even proposed. Missing, stale, conflicting, or unauthorized evidence must lead to clarification, abstention, or escalation.

4. Effect-control gate

Validate arguments, permissions, limits, idempotency, confirmation, reversibility, reconciliation, and audit in application code. Structured function arguments are untrusted proposals, not authorized commands. [CLM-020]

5. Evaluation gate

Test ordinary, ambiguous, adversarial, repeated, partial, delayed, and dependency-failure cases. Measure incorrect action, missed action, user burden, review burden, and recovery. A text-quality score is insufficient.

6. Operating gate

Name owners, signals, stop controls, incident behavior, and change replay. An agent loop without a bounded runtime and support model is an experiment, not a product.

Failing any gate means use a simpler state. The alternatives are not binary “agent or nothing.” The product may present a recommendation, prepare an editable draft, ask for explicit confirmation, use a deterministic workflow, or route a named review.

Patchwork fails the first gate for automatic seller contact: no evidence shows that sending rather than drafting improves the current discovery task enough to justify the effect. It also fails the authority and operating gates because the current contract provides no messaging permission or incident path. Purchase fails even more directly. Therefore no agentic implementation belongs in PF-03 v0.1.

This conclusion could change only through a new task contract. A future authorized negotiation or procurement task would not be a switch on the current finder; it would need different affected parties, permissions, evidence, controls, and outcomes.

Design the local companion experiment

The companion should make mechanism boundaries executable without requiring a vendor, model download, secret, or network.

Synthetic catalog

Use a seeded set of fictional pump-seal listings. Each record has catalog family, title, terms, explicit dimensions and units, permission, freshness, seller label, synthetic image-feature vector, and provenance. Inject missing units, duplicates, stale records, conflicting fields, and excluded families deliberately.

The seed makes results replayable. It does not make the distribution representative. Chapter 5 records that limit in the data card.

Three retrieval paths

1. Lexical: token overlap and exact identifier/attribute signals.
2. Vector-like: deterministic cosine similarity over supplied synthetic feature vectors.
3. Hybrid: normalized combination of lexical and vector-like candidates after permission and scope filtering.

The vector-like path demonstrates a representation boundary. It is not a claim that the handmade vectors behave like a production embedding model.

Deterministic ranker

Rank using inspectable features: retrieval evidence, explicit measurement match, missing-field penalty, freshness, and conflict penalty. Apply known incompatibility as exclusion before presentation. Do not claim the weights are learned or production-optimal.

Candidate trace

For each query, emit channel scores, included and excluded IDs, permission/freshness checks, matched/unresolved/conflicting fields, rank reasons, and final behavior state. Empty results must abstain or fall back. Unauthorized candidates should not appear merely with a warning.

Comparison report

Run all paths on frozen cases and report component behavior separately from end-to-end contract state. Include cases where lexical wins, vector-like wins, hybrid introduces noise, and deterministic controls dominate every score.

Optional provider adapter

Define an interface but provide no default implementation or secret lookup. A later user can supply a provider adapter deliberately. Canonical tests remain local and deterministic, so the book’s evidence does not depend on a commercial service.

The companion’s completion claim is limited: it proves that the decision architecture can be represented and its synthetic failure states tested. It does not prove Patchwork quality in the world.

Facilitate the mechanism review

A mechanism review should end in decisions, not a tour of technologies. Send the behavior contract, task-data assumptions, portfolio table, tradeoff surface, experiment ladder, and authority map in advance.

During the review, use this order:

1. Reconfirm the task and current non-goals.
2. Identify clauses that no current baseline can satisfy.
3. Compare portfolios only on those gaps plus their new burden.
4. Surface data and authority assumptions that could stop each portfolio.
5. Review failure states and fallback before best-case capability.
6. Choose the cheapest discriminating experiment.
7. Record accepted, rejected, and gated options with revisit triggers.

Do not begin with vendor demos. They anchor the group on an implementation and often hide the evidence boundary. If a demo is necessary, map every visible behavior to a clause and list what the prepared conditions omit.

When reviewers disagree, locate the disagreement:

• Is the task consequence different?
• Is one person assuming data that does not exist?
• Are they comparing component and product evidence?
• Do they price latency, cost, or review burden differently?
• Is an authority decision unresolved?
• Are they optimizing different segments?

The Applied AI Engineer owns making the disagreement testable. They do not win through title or novelty. A strong recommendation can say, “Hybrid retrieval is the leading option, conditional on the data-card gate; generation remains unselected because structured evidence has not yet been tested.”

Mechanism changes the team topology

Every mechanism creates ownership work.

Deterministic domain rules need a domain source, version, tests, and change owner. Retrieval needs index, provenance, permission, freshness, and relevance ownership. A ranker needs feature semantics, evaluation, thresholds, and monitoring. Generation needs prompts or policies, semantic evaluation, failure controls, provider/version change, and fallback. Tools need authorization, effect controls, reconciliation, and incident paths.

This does not mean one Applied AI Engineer owns every component. It means the behavior decision includes the responsible interfaces. A platform team may operate retrieval infrastructure. A catalog team may own attribute semantics. Product engineering may own the interface. The Applied AI Engineer ensures the combined evidence supports the contract and that missing ownership blocks or narrows behavior.

Mechanism selection without ownership is architectural debt created on day one. Include the owner and evidence-maintenance cost in the portfolio, even during a prototype.

Failure patterns

Benchmark winner equals product winner

A broad evaluation result selects a model without representative task segments, system effects, or operating constraints.

Correction: map benchmark claims to the contract, then run task and system evidence. Preserve dataset and version limits.

Agent as sophistication badge

A multi-step task is declared agentic even though a deterministic workflow with user confirmation is sufficient.

Correction: identify the specific action that needs adaptive autonomy, its incremental value, and why a simpler confirmed workflow fails. If none exists, do not add agency.

LLM everywhere

Generation performs extraction, retrieval, ranking, policy, explanation, and tools in one opaque call.

Correction: separate stable deterministic and evidence-retrieval decisions. Add generation only where variable transformation earns value.

Deterministic controls treated as obsolete

A learned score overrides known incompatibility or permission.

Correction: encode authorized invariants outside the learned component and test that they dominate.

Feature parity

Every mechanism is required to produce the same interface, including false confidence or explanation.

Correction: compare whether each portfolio satisfies the task contract, not whether it imitates a favored demo.

Operational unfamiliarity hidden as simplicity

A small custom model is chosen because its algorithm looks simple, while the team lacks data/version/monitoring ownership.

Correction: include lifecycle burden and team capability in the tradeoff surface.

Revisit the decision under changing constraints

A mechanism record becomes valuable when the obvious recommendation changes with one constraint. Work through four Patchwork variants.

Variant 1: exact identifiers dominate

Suppose authorized task evidence shows that most supported buyers possess a manufacturer part number and failures come from punctuation, spacing, and superseded identifiers. Improved lexical and structured lookup is likely adequate. Multimodal retrieval adds little task value and creates index and evaluation burden. The correct decision is not to include it merely because the architecture anticipated it.

The experiment ladder stops after the improved baseline until a segment with unresolved vocabulary is shown to matter.

Variant 2: visual form is material, dimensions remain explicit

Suppose buyers often lack catalog vocabulary but can provide a photo and required measurements. Multimodal retrieval may add candidates while deterministic dimension rules protect compatibility states. Hybrid retrieval earns an experiment. Generation still does not follow: structured matched and unresolved evidence may be understandable.

The application keeps the image as an authorized task input with purpose, retention, access, and deletion rules. An embedding is a representation of the image, not permission to retain it indefinitely.

Variant 3: catalog labels are unreliable

Suppose the data audit finds that sellers use dimension fields inconsistently and no domain rule can reconcile them. A stronger model is not the answer. The supported compatibility behavior narrows or stops. Retrieval may remain useful for similarity-only exploration if the interface and contract clearly change.

This is a product decision, not a failed AI ambition. The mechanism portfolio returns to the task and contract movements of the evidence loop.

Variant 4: generated explanations improve correction

Suppose structured evidence leads users to overlook conflicts, while a bounded explanation over the same frozen evidence significantly improves correction in representative interaction tests without increasing unsupported claims, tail latency, or cost beyond the authorized budget. Generation can enter the portfolio for that presentation job.

It still does not select candidates, infer missing values, change state, or contact sellers. The evidence earns one bounded component, not a general license for generation.

These variants show why the mechanism record stores rejected alternatives and revisit triggers. “Hybrid is best” is brittle. “Hybrid is selected for segments where lexical/structured baselines miss plausible candidates and authorized multimodal evidence exists, while deterministic controls remain dominant” can survive change.

Quality gates for the selected portfolio

Before Chapter 5 proceeds, PF-03 must satisfy eight quality gates.

Contract coverage

Every selected component maps to one or more behavior clauses, and every clause maps to a selected component, application control, interaction, or human role. Orphan components are unjustified. Orphan clauses are unimplemented.

Baseline credibility

The improved ordinary-search path receives fair normalization, structured filters, and evidence presentation. It remains a shippable fallback, not deliberately weak comparison code.

Data assumption visibility

Each mechanism lists the population, inputs, labels, provenance, permission, and freshness it requires. Unknown availability is a gate, not an implicit future task.

Consequential failure separation

Candidate miss, unsafe inclusion, ordering error, unsupported explanation, unauthorized data, and prohibited effect are measured separately. An aggregate cannot compensate for a critical failure.

Boundary enforcement

Known incompatibility, scope, permission, and tool prohibitions live outside learned scores. Tests must show that no higher score or generated text overrides them.

Operational feasibility

The portfolio has owners and provisional latency, capacity, cost, telemetry, degradation, and recovery questions. Exact budgets arrive later, but missing responsibility is visible now.

Change isolation

Candidate, decision, and presentation contracts prevent a provider or model change from silently redefining the product. Every volatile component has a version and replay trigger.

Authority preservation

The Applied AI Engineer recommends the technical path. Product, catalog, privacy, security, trust, accessibility, and release/risk owners retain their real decisions. A mechanism review does not double as formal approval.

Passing these gates means the portfolio is coherent enough to test. It does not mean it is fit to release. Chapter 5 may still invalidate the data assumptions; Chapters 6 through 19 will add context, system, evaluation, operational, release, and change evidence.

Write the recommendation so it can be challenged

A mechanism recommendation should fit on one decision page even when its evidence packet is large. Include:

• the task segment and contract version;
• the selected portfolio and the job of each component;
• the credible baseline;
• decisive evidence and its limitations;
• data and authority assumptions;
• consequential failures and deterministic boundaries;
• operational and change burdens;
• rejected and gated alternatives;
• cheapest next experiment;
• decision owner and revisit triggers.

Avoid adjectives such as powerful, state of the art, intelligent, and flexible unless they resolve into a task criterion. Avoid claiming that the architecture is future-proof. State which interfaces make specific components replaceable and which semantics remain coupled.

For Patchwork, the challengeable recommendation is: select hybrid candidate retrieval and bounded ranking only after the synthetic data pipeline proves its mechanics and an authorized real data audit supports the segment; preserve improved search and deterministic controls; defer generation until it improves comprehension against a template; prohibit action tools.

A reviewer can attack every condition. They can show that lexical search is adequate, multimodal evidence lacks permission, hybrid latency is unacceptable, the ranker introduces a segment regression, or structured presentation already meets the interaction criterion. Any of those results can change the decision without undermining the process.

That is the standard: a good mechanism decision is not one that survives every future fact. It is one whose assumptions, evidence, and return path make a future correction controlled rather than embarrassing.

Archive the recommendation with the contract, frozen cases, component versions, and review disposition. A slide without its evidence is not a durable decision. When the portfolio changes, preserve the earlier record and explain which assumption changed; do not rewrite history as though the new architecture was always obvious.

The archive also lets incident and migration reviewers distinguish a violated design from a design whose original evidence no longer represents current use.

PF-03 mechanism portfolio v0.1

Patchwork records the following decision.

Selected first path: improved ordinary search plus deterministic scope, permission, normalization, and incompatibility controls; lexical, structured, and multimodal candidate retrieval; deduplication and bounded ranking; structured evidence states; clarification, abstention, and ordinary-search fallback.

Selected next experiment: after the Chapter 5 data gate, compare improved ordinary search with lexical-only, multimodal-only, and hybrid candidate retrieval behind one local candidate interface. Use a frozen synthetic dataset to test mechanics and failure states; do not interpret synthetic distributions as marketplace performance.

Rejected now: compatibility classifier without an authorized label; integrated generative finder; any autonomous seller contact or purchase.

Gated later: generated explanation over frozen evidence; editable seller-question draft. Each must beat a deterministic template and preserve the capability boundary.

Decision owner: Applied AI Engineer prepares technical evidence and recommendation. Product, catalog-domain, privacy/security/trust, and release/risk authorities retain their named decisions.

Revisit triggers: task segment changes; authoritative label becomes available; catalog schema changes; representation/ranker evidence fails; latency/cost/permission changes; generation demonstrates decision-relevant benefit; or a new authorized task genuinely requires action.

Completion exercise

Score six portfolios for a task using the twelve dimensions in this chapter. Include at least one deterministic, retrieval/ranking, predictive, multimodal, generative, and tool/agent option. Reject at least two. Your output must contain:

• the behavior clauses each portfolio can and cannot satisfy;
• the data and authority it requires;
• consequential failure modes;
• component and end-to-end evidence;
• latency, cost, privacy, attack, operational, and change burdens;
• fallback and reversibility;
• the cheapest experiment likely to change the decision.

The exercise fails if the portfolios form a maturity ladder, if a benchmark chooses the winner, if a tool call implies authorization, or if generation and autonomy cannot fail their gates.

Chapter decision

Choose a mechanism portfolio against the behavior contract, not fashion. Simple baselines are legitimate product candidates. Multimodal representation can enable discovery without language generation. Retrieval, ranking, explanation, and tools solve different problems and create different evidence obligations. More capability can be a worse product choice when it increases consequence, operating burden, or change risk without task value.

Patchwork’s PF-03 v0.1 selects deterministic controls plus hybrid lexical, structured, and multimodal retrieval and bounded ranking as its first learned path. Generation remains an optional explanation experiment over frozen evidence. Seller-question drafting remains editable and unsent. Autonomous contact and purchase remain prohibited.

The mechanism decision is conditional on data. Chapter 5 will test whether Patchwork’s synthetic task-data design can represent the supported population, preserve provenance and permission, expose missing and duplicated states, and keep development evidence separate from release evaluation. If it cannot, the selected mechanism must narrow or stop.