Universal Multi-Lab Mapping

This mapper must work for a laboratory that appears once, not only for a laboratory that has a hand-maintained local map. A local map is helpful evidence, but it is never the sole route to a normal laboratory result.

The Core Idea

LOINC codes are not selected from a flat list of similar names. A laboratory row states facts that can be compared to the six LOINC axes:

Row fact	LOINC axis it constrains	Example
`Creatinine, Urine`	Component and System	`Creatinine`; `Urine`
`Sodium 24 hour urine`	Component, Time, System	`Sodium`; `24H`; `Urine`
`LDL calculated`	Component and Method	LDL cholesterol; `Calculated`
`LDLP`, `nmol/L`	Component and Property/unit family	particle count; substance concentration

The mapper extracts those facts before broad semantic retrieval. It then asks the pinned LOINC release for candidates whose six-axis signatures agree with the report. This works whether the source laboratory is Labcorp, Quest, a hospital laboratory, or an unseen laboratory name.

LabObservation
  -> AxisFactExtractor
  -> UniversalSemanticIndex (release-derived component and axis retrieval)
  -> deterministic aliases / FTS / Parts
  -> ScispaCy + local UMLS + FAISS/SapBERT recall
  -> hard UCUM and six-axis validation
  -> AxisSignature grouping
  -> ranker and calibrated confidence gate

The original OCR text is never replaced. Every fact, candidate, rejected sibling, and final decision is retained in MappingResult.provenance.

Example: Creatinine In Urine

These labels should mean the same thing even when they come from different laboratories:

Creatinine, Urine
Urine creatinine
CREATININE - URINE

For Creatinine, Urine with mg/dL, the extractor retains facts similar to:

{
  "component": [{"value": "Creatinine", "source": "raw_name"}],
  "system": [{"value": "urine", "source": "raw_name"}]
}

The universal index retrieves active result-row candidates from the local LOINC release. The hard validator rejects serum/plasma siblings because the row explicitly says urine. It also rejects a 24-hour urine sibling unless the row explicitly states 24 hour or equivalent duration. It does not need a Wellspan or Quest registry record to reach the ordinary point-in-time urine candidate.

collection_specimen="Blood, Venous" is intentionally different from specimen="Urine". A collection source says how material was obtained; it does not prove whether the analyzer used whole blood, serum, plasma, or urine. The mapper keeps collection evidence in provenance and only fills the LOINC System axis from an analytical specimen stated by the row or its label.

What The Universal Index Contains

UniversalSemanticIndex is generated once per mapper worker from the pinned SQLite LOINC catalog. It does not create a second authoritative terminology database. It uses active laboratory Observation/Both terms plus the catalog's release-compiled sources:

LOINC term names and Components.
LOINC Parts and primary/supplementary Part links.
Consumer names, RELATEDNAMES2, and enabled linguistic variants.
Release aliases and token/FTS retrieval.
UMLS aliases only as additional candidate-recall evidence.

COMMON_TEST_RANK is a small tie-breaker after compatibility checks. It does not suppress rare tests and never overrides a unit, specimen, property, time, scale, or method conflict.

Candidate Signatures

Several LOINC codes can have similar text but clinically different meanings. Before confidence scoring, candidates are grouped by:

Component | Property | Time | System | Scale | Method

The strongest code within an indistinguishable signature is retained for ranking, while the full group remains in provenance. A candidate in a different signature remains only if it is compatible with every explicit report fact. This makes a failure explainable:

Creatinine in serum is rejected when the row says urine.
A 24-hour candidate is rejected without an explicit collection duration.
A calculated LDL candidate is rejected when the row states direct assay.
LDL particle candidates are rejected for an mg/dL mass-concentration row.

If two safe, clinically distinct signatures remain and the row lacks the fact that separates them, the correct behavior is abstain, not a guessed code.

CLASS And Panel Context

LAB_RESULT is a hard scope: only active laboratory (CLASSTYPE=1) result terms are eligible. LOINC CLASS, a panel name, and a report section are not clinical axes. They are advisory ranking/retrieval evidence only.

For example, an OCR parser can incorrectly put CRP, TSH, or an endocrine result under a CBC or CMP section. The mapper records the class disagreement, searches laboratory-wide when needed, and lets unit/property/specimen/method validation decide safety. A class mismatch can never make an otherwise valid laboratory candidate unsafe by itself.

Governed Registry Layers

The active registry supports two explicit layers:

Layer	Purpose	Runtime behavior
`universal_mapping_templates`	Expert-confirmed, clinically invariant display forms with explicit constraints	Available for every laboratory after unit and axis validation
`source_mapping_evidence`	Local test IDs, vendor methods, or report displays that need source context	Participates only for the named laboratory; a matching `source_test_id` may disambiguate a documented distinct assay, while lab-name-only evidence only boosts retrieval and cannot suppress a universal template

An approved universal template can retain supporting_source_laboratory for audit without requiring that laboratory at runtime. Entries with source_laboratory remain source evidence until an expert explicitly promotes them.

When both layers propose different active codes, the mapper evaluates both through the same unit and six-axis safety gate. An exact local source_test_id, when the report actually supplies one, can outrank a universal template for a clinically distinct documented assay. Without that identifier, a verified clinician universal template outranks source-laboratory display evidence. A laboratory name alone is never a veto over source-neutral LOINC retrieval.

Audit the current registry without changing it:

python -m loinc_mapper audit-registry-scopes `
  --registry config/mapping_registry.json `
  --output evaluation/registry_scope_audit.json

Export the same records in the governed two-layer format when a main project is ready to consume it:

python -m loinc_mapper export-layered-registry `
  --registry config/mapping_registry.json `
  --output config/mapping_registry.layered.json

Neither command changes the live registry or promotes a mapping. The legacy mappings format remains supported so an existing integration can upgrade without a big-bang configuration migration.

Review-Gated Learning

Uncertain rows go to a review queue with their axis facts, signatures, candidates, units, source laboratory, and full provenance. Cluster recurring rows before review:

python -m loinc_mapper cluster-review-queue `
  --input evaluation/review_queue.jsonl `
  --output evaluation/review_clusters.json

Clusters from one source default to source_evidence. A recurring surface from multiple sources may be proposed as universal_template, but only a reviewer can approve it. No cluster creates a live mapping automatically.

Debugging A Row

For an abstained or rejected row, inspect these fields before adding any registry record:

result = mapper.map(observation)
stages = result.provenance["stages"]

print(stages["universal_axis_facts"])
print(stages["axis_signature_groups"])
print(stages["registry_evidence"])
print(stages["routing_behavior"])
print(result.candidates)
print(result.axis_evidence)
print(result.unit_evidence)

If the needed specimen, duration, method, or unit is absent from the report, adding a broad global alias is not the solution. Keep the result in review or obtain the laboratory's test definition. This preserves safety across the many laboratories the fax automation system will encounter.

registry_evidence separates universal_templates from source_mapping_evidence, even if an entry later fails validation or loses to another candidate. routing_behavior explicitly records that class was advisory and that laboratory-wide retrieval remained enabled.

Validation Status

The source-neutral regression fixture verifies ordinary rows with their original source and a previously unseen source have identical results. It is a software regression test, not a clinical accuracy claim. The production gate remains at least 95% accepted precision and at least 85% coverage on an expert-reviewed multi-laboratory holdout, with zero dangerous unit/property/specimen false positives in the safety suite.