Skip to content

Clinical Validation

Current approval

The workspace owner recorded that Dr. Tro reviewed and approved the active registry and current mapping review decisions on 2026-08-06. The immutable approval scope and SHA-256 fingerprints are in evaluation/clinical_review/approval_manifest.json.

This completes the clinical-review blocker for the approved registry. Approval does not weaken runtime controls: active-code checks, universal semantic retrieval, UMLS/SapBERT evidence, six-axis validation, UCUM safety, and the confidence gate remain mandatory.

Interim measured result

The de-identified, clinically reviewed Chambersburg corpus was evaluated through the full production pipeline. The source, audit, and result are retained under evaluation/clinical_review/.

  • 27 clean, labelled rows from one laboratory.
  • 100% auto-accepted precision (21/21).
  • 77.78% coverage (21/27).
  • 100% top-3 recall and zero dangerous false positives.

This is encouraging safety evidence, but it does not pass the release gate: it has one source laboratory and coverage is below 85%. The six abstentions are preserved in the summary so any future recall improvement can be assessed without hiding its effect on precision.

Release metrics

Clinical approval and a measured release gate are related but different:

  • Clinical approval says reviewed mappings may be activated.
  • A release report demonstrates the current build's precision, coverage, and dangerous-false-positive rate on a frozen, source-labelled holdout.

The historic stress CSVs remain regression inputs. They do not include a true source-laboratory holdout split, so they cannot independently prove the multi-laboratory release metrics even after their mapping decisions have been reviewed.

Before declaring a production build to meet the numeric gate, preserve a de-identified, reviewed CSV with source_laboratory and run:

$env:PYTHONPATH = "$PWD\src"
python -m loinc_mapper evaluate `
  --input evaluation/clinical_review/holdout.csv `
  --details evaluation/runs/clinical_holdout_details.json `
  --profile-matrix `
  --umls-path assets/umls/2026AA

The pass criteria are at least 95% auto-accepted precision, at least 85% coverage, and zero dangerous false positives. Retain the resulting summary with the approval manifest and release metadata.

Change control

The manifest fingerprints the approved registry files. If a mapping, unit constraint, source-specific record, or safety rule changes, generate a new approval record after review. This prevents an old clinical sign-off from being misapplied to a changed registry.

Continuous Clinician Learning

Clinician review improves coverage in two distinct ways. An approved mapping becomes a versioned registry snapshot for the next mapping job immediately after validation and replay. It does not retrain SapBERT or a learned ranker. That immediate vocabulary learning is auditable and reversible by moving the active snapshot pointer back to a prior version.

Reviewed decisions are also retained as future training/evaluation records. They may train a challenger ranker only after there are enough reviewed labels and only after the challenger preserves accepted precision, multi-laboratory coverage, and zero dangerous unit/property/specimen failures on the frozen holdout. No repeated abstention or uploaded CSV row may silently become a mapping without clinician approval.

Monitor weekly: review volume, repeat-abstention rate, review turnaround, coverage, accepted precision, dangerous false positives, registry-version adoption, and clinician reversals. A lower abstention rate is useful only if the safety metrics remain intact.