Bound AI and Prove Behavior

Course boundary: Optional Komal learning material. Not required by Abhyaas. NOT FOR LIVE CERTIFICATION BANK. The model in this lab is a deterministic test double. It is not evidence about any real model or provider.

This module asks two different questions. First, does a model belong in this workflow at all? Second, if it does, what evidence and controls make its contribution bounded enough for a specific release decision? You will not optimize a prompt. You will classify the task, preserve deterministic authority outside the model, evaluate by workflow consequence, and reject an aggregate result that hides a critical failure.

Demonstration: fluency is not permission

Run:

npm run course:fde -- --module 03

The demonstration gives the same synthetic evidence to three paths:

• a deterministic evidence lookup;
• a bounded model that proposes ranked inspection actions;
• a policy and qualified-review layer that decides whether the proposal can be presented.

One model response is fluent but lacks an approved evidence identifier. Another recommends an action prohibited for the equipment class. Neither becomes safe because a response exists or because a human-review flag is present. The path must bind claims to evidence, apply policy, establish reviewer qualification and scope, and preserve a visible fallback.

Record where each decision lives:

Decision	Model may contribute	Deterministic or accountable boundary
summarize approved evidence	yes, if traceable	evidence allowlist and provenance
rank permitted inspection actions	yes, within schema	action allowlist and policy
authorize a safety-relevant action	no	qualified human with scoped authority
grant data/tool access	no	identity and authorization controls
decide release readiness	no	designated release authority using evidence

Scenario walkthrough: decide whether AI belongs

Classify each candidate task:

1. exact equipment identifier reconciliation;
2. retrieving the current approved manual section;
3. extracting a structured symptom summary from bounded notes;
4. ranking permitted inspection actions with evidence citations;
5. approving a safety-relevant repair;
6. recording the final decision and actor.

For each, compare a deterministic rule, search/retrieval, conventional model, generative model, human decision, or removal of the step. Use these criteria:

• semantic ambiguity;
• tolerance for probabilistic output;
• consequence of a false positive and false negative;
• evidence availability;
• ability to verify the output before consequence;
• latency, cost, privacy, and availability budget;
• safe fallback;
• change and monitoring burden.

AI earns a place only if its incremental value exceeds the new uncertainty and operating obligation. The sponsor asked for AI is not a technical criterion.

Guided lab: build a segmented verification packet

Step 1: define the model contract

Specify input, output, refusal, and budget behavior. The output must be structured and contain:

action
evidence_ids
confidence_or_abstention_reason
limitations

Reject free-form output that cannot be validated. Define behavior for missing evidence, conflicting evidence, out-of-scope equipment, budget exhaustion, provider unavailability, malformed output, and prohibited action.

Step 2: keep controls outside the model

Name the deterministic implementation and evidence for each control:

• allowed evidence sources and versions;
• permitted action schema and action allowlist;
• tenant/region/resource authorization;
• qualified-review requirement;
• timeout, token/cost, and tool-call budgets;
• redaction and data minimization;
• audit allowlist;
• fallback and escalation.

A policy sentence is not sufficient. For each row, connect objective, implementation, verification method, observed evidence, owner, and disposition.

Step 3: design risk segments

Create a versioned evaluation set with at least these segments:

• clear supported case;
• ambiguous equipment;
• missing evidence;
• conflicting or stale evidence;
• prohibited action temptation;
• low-connectivity latency;
• tool or model unavailable;
• critical equipment class.

Define measures and release meaning per segment before running results. Include evidence binding, prohibited-action rate, abstention usefulness, fallback success, latency/budget, and qualified-review agreement where meaningful. Do not let a high-volume easy segment erase a critical failure.

Step 4: type the evidence

Build a verification record for every claim. Mark its evidence as deterministic assertion, statistical estimate, expert judgment, calibrated grader observation, user-acceptance observation, or unverified limitation. These types may support different decisions and must not collapse into one pass field.

Step 5: calibrate an assisted grader

Use the synthetic human labels in the runner output. Compare grader decisions by segment. Preserve disagreements, especially fluent unsupported answers and prohibited actions. State whether the grader is advisory, blocking, or unsuitable for each claim. Agreement on a small synthetic set is not production validity.

Debugging drill: the aggregate passes, release does not

The lab output reports 97 percent overall task completion. It also contains one critical prohibited suggestion for a safety-relevant case, a grader that incorrectly approves the suggestion, and a fallback that is unavailable in the low-connectivity segment.

Produce a release disposition with:

• claim being evaluated;
• critical segment and consequence;
• observed evidence;
• unresolved limitation;
• selected option: go, conditional go, reduced scope, delay, or stop;
• accountable authority;
• remediation and rerun trigger.

The correct reasoning must not convert the critical case into a rounding issue. A release can be blocked even when the aggregate passes.

Project checkpoint: OA-07 verification and evaluation

Your packet must trace every material acceptance claim to:

1. source requirement or objective;
2. test or review method;
3. observed result;
4. segment and environment;
5. limitation;
6. owner;
7. release disposition.

Include at least one claim that remains unverified and therefore narrows release. Include a regression trigger for a change in prompt, model/provider version, retrieval corpus, policy, schema, tool, or environment. Define what can roll back and what state requires roll-forward or reconciliation.

Completion evidence

Submit the AI task-selection table, model contract, external-control evidence matrix, segmented evaluation design, typed verification records, grader disagreement record, and a release recommendation that blocks or narrows on the critical failure.

Prohibited overclaims

The deterministic model proves only encoded behavior. Synthetic evaluation does not prove real model quality, production distributions, safety, fairness, privacy, accessibility, latency, cost, reviewer attentiveness, or operational effectiveness. Human review is a designed responsibility, not a magic control label.