flightops learning records frozen evaluation suites, all attempt outcomes, explicit training decisions and small model/selector artifact candidates in the local history SQLite catalog. It never downloads weights, calls a model, starts training or deploys a remote service. Standard mode owns the complete interface.
Frozen comparisons
A suite names 1–1000 incidents and Standard arm A, same-version general-model arms B/C/D (ordinary tools, evidence contracts, reviewed memory), plus optional E. Every arm has a fixed configuration digest. Incidents retain source IDs, exact content hashes, declared near-duplicate groups, parent/follow-up IDs, project, fault family, configuration, interval, split and label-review state. The evaluator, review declaration, success target and minimum held-out sample count are frozen.
Exact content/group/follow-up leakage across train, development and holdout is rejected. Selected project/family/configuration dimensions must also stay separate; when time separation is selected, periods must be disjoint and ordered. Semantic near-duplicate detection is a reviewed group declaration, not an automatic universal similarity guarantee. Unknown/unreviewed labels cannot qualify training. An evaluator/reviewer name records an assertion; it does not authenticate an independent human or establish correctness by itself.
Attempts carry an exact suite/evaluator digest, incident/arm, unique sequence, completed/failed/interrupted/abandoned/not-run state, separate nullable metrics and observed costs. Metrics cover verified success, false success, unsupported acceptance, useful/unnecessary abstention, relevance, wrong applicability, protected omission and recovery. Unreviewed and unrun attempts cannot provide scored metrics. Successful outcomes require explicit false-success and unsupported acceptance negatives, rather than treating missing observations as false.
Reports preserve every attempt, count retries and failed attempts, include every planned incident/arm cell and distinguish unmeasured cost from zero cost. Latest cell outcomes and all-attempt counts are both visible. Costs include failures, retries, reads, escalation and human review. Results are stratified by project, family, configuration and split. B→C, C→D and D→E paired success differences list actual pairs and exclusions; they are descriptive counts without statistical intervals or a causal A→D memory claim.
All twelve master negative-test categories remain required for every arm. Missing or failed negative checks block that arm's boundary gate regardless of average success. Evidence hashes and reviewer declarations are preserved; this registry does not execute those tests or independently attest to their evidence.
Training and model lifecycle
Training eligibility requires reviewed train-only examples, current source-use retention and training rights through all parent sources, an explicit audience/owner review for every source in the resolved upstream closure, redaction and labels review with native receipt references, compatible model-term review, a bounded example/time/money budget and a complete measured held-out shortfall beyond untuned retrieval/memory arm D. Missing evidence records a typed no_training decision. An eligible experiment still launches no job.
A model record without real artifact bytes and a valid held-out report remains planned_unverified. Model records support artifacts up to the bounded JSON/file reader limit (4 MiB), such as small selector adapters. Arm E's configuration digest must equal the artifact SHA-256, and its model ID must match the candidate. Its suite may update the E arm after training while preserving every incident, evaluator, separation rule, target and A–D control. Unmeasured critical/protected outcomes, any critical regression and unrun/failed boundary checks prevent activation even when success is high.
Activation selects a local routing record for a declared allowed audience; it is not an identity service or an instruction to call/deploy a model. Every route lookup rechecks current source rights, source-withdrawal events, exact artifact bytes, baseline eligibility and held-out source availability. Withdrawal commits the source tombstone before bounded dependency diagnosis, so interrupted propagation still makes dependent dataset decisions and models unusable; lookup disables the route. Rollback only selects an earlier still-eligible candidate, otherwise disabling routing. No automatic remote fallback occurs. Local artifact copies remain under the owner's control; disabling a route does not prove weights forgot data.
Commands
All commands accept --database PATH and repeated --allow-source SOURCE flags before the subcommand. Current source policy remains mandatory in addition to these explicit local-owner authorization flags.
freeze-suite REQUEST.jsonrecords a validated suite and returns its registry ID.report REQUEST.jsonacceptssuite_id,attempts, andnegativesand preserves their input records alongside the recomputable report.show-report IDchecks current source rights and report arithmetic.assess-training REQUEST.jsonrecords an eligibility or no-training decision.register-model DECISION_ID --model-id ID [--artifact PATH --heldout-report ID]records a candidate with honest unavailable/unverified states.activate REGISTRY_ID --audience AUDIENCE,route --audience AUDIENCE,rollback --audience AUDIENCEmanage the local route.withdraw SOURCE --reason TEXTappends training withdrawal and invalidates dependent records without pretending to erase model weights.
The additive tables are learning_records and learning_events. Portable history backup must explicitly exclude registry bodies and original artifact paths until a policy-gated portable registry format exists. Full real-incident quality, independent evaluation, real training and release acceptance remain separate gates.