Skip to content

OCR Recovery Contract

noise_to_loinc maps a row, not a page image. Faxautomation owns OCR, table geometry, Gemini, source-document storage, and patient data. The mapper remains storage-free and independently validates every ordinary mapping.

Required row lifecycle

  1. Preserve original OCR cells, page number, crop reference, and row ID in the main project.
  2. Run deterministic table checks before mapping. Detect repeated footnote markers, duplicate cells, merged analyte labels, shifted value/unit cells, and orphan continuation fragments.
  3. Call Gemini once per damaged table region, never once per analyte. Do not call Gemini for a clean table.
  4. Validate Gemini's structured answer against the original cells and page geometry. Send valid repaired rows through Mapper.map_many().
  5. Send a row marked input_structure_invalid when a table cannot be reconstructed. The mapper will abstain before retrieval rather than invent a clinical result.

Gemini boundary

Gemini is an OCR reconstruction advisor. It must not select, rank, suggest, or publish LOINC codes. Provide the cropped table image, OCR cells with coordinates, neighboring headings, and the closed panel enum from panel_context_index.2.82.json. For a heading the enum answer is one listed record key or UNKNOWN, never the nearest listed value: a heading the index does not carry is UNKNOWN with the raw heading preserved, and the application trusts an enum answer only when a row under the heading names one of that record's members (PanelContextIndex.member_codes_for_name, panels.row_member_codes; see panel context).

Require JSON matching this shape:

{
  "source_laboratory": "Labcorp",
  "sections": [
    {
      "raw_heading": "CBC with differential",
      "panel_context": "CBC",
      "panel_context_source": "llm_enum_hint",
      "panel_context_confidence": 0.7,
      "rows": [
        {
          "row_id": "page-2:row-14",
          "raw_name": "Magnesium, RBC B, 01",
          "mapping_name": "Magnesium, RBC",
          "value": "5.5",
          "unit": "mg/dL",
          "input_quality": "gemini_verified_repair",
          "repair_operations": ["remove_layout_marker", "remove_detached_cell_artifact"],
          "repair_confidence": 0.99,
          "source_cell_references": ["page-2:cell-17", "page-2:cell-18"],
          "row_state": "valid"
        }
      ]
    }
  ]
}

Allowed input_quality values are:

Value Meaning
original Original row is structurally valid; no repaired name.
deterministic_repair Layout evidence deterministically repaired the row.
gemini_verified_repair Gemini proposed a repair and faxautomation verified it against source cells.
input_structure_invalid Merged, shifted, duplicate, or incomplete cells prevent safe reconstruction.

Only deterministic local heading matching may use panel_context_source="official_panel_heading" with confidence 0.98. Gemini-only enum selections remain llm_enum_hint with confidence no greater than 0.70. Unknown headings retain their literal text with source unrecognized_panel_heading. Send the printed heading itself, cut or not: the index resolves a heading cut at the page edge when it ends inside a word of exactly one record's alias (heading_truncated in the resolution, 2026-09-25), so that case never needs an enum guess.

The review question travels on the result (2026-09-29)

The mapper computes the review question of every result row and returns it in provenance.stages.question (see clinician learning). Two optional observation fields feed it:

  • printed_label: the label the row prints once a verified cleaning removed layout, captions and the row's own facts (Sodium for the cell Sodium Normal Range: 135 - 145 mmol/L, Vitamin B12 for Vitamin B1202). Never an expansion: an expansion is mapping_name with llm_name_expanded, and the question keeps the print.
  • name_grounding: text, image_only or name_unverified. An unverified name keeps raw_name as its question's name and creates no alias until a person confirmed it.

Send the row's own facts: the printed heading with official_panel_heading only when it is printed above the row (a hint or an inference is no fact of the question), the specimen with its specimen_source, the method column, the collection time. Never build a case key from the laboratory or from the mapper's outcome.

Mapper payload

Keep the original OCR label in raw_name. A repaired name goes in mapping_name; it never overwrites the source value.

LabObservation(
    raw_name="Glucose 02",
    mapping_name="Glucose",
    input_quality="deterministic_repair",
    repair_operations=("remove_repeated_layout_footnote",),
    repair_confidence=1.0,
    source_cell_references=("page-2:cell-6",),
    value="80",
    unit="mg/dL",  # exact printed/recovered surface, never UnitAssessment.canonical
    source_laboratory="Labcorp",
    observation_domain="LAB_RESULT",
    panel_context="Comprehensive Metabolic Panel",  # the index record key
    panel_context_source="official_panel_heading",
    panel_context_confidence=0.98,
    report_section_id="p2:COMP. METABOLIC PANEL (14)",
    reference_range="70-99",  # the printed cell
    source_test_id="322000",  # the printed vendor test number, when printed
)

What the fields carry (2026-09-24, agreed with the main project):

Field Content
panel_context The panel index record key the application resolved the printed heading to (every key is an alias of its own record). The printed heading stays application-side. An unrecognized heading is sent as printed, with source unrecognized_panel_heading.
panel_context_confidence Always sent, 0.0 included.
panel_loinc_hint Only under official_panel_heading; when it names the same record as the heading, the heading's source and confidence are kept.
report_section_id p{page}:{heading} for a row under a heading, p{page}:table for heading-less model rows, p{page}:markdown-{n} for Document AI tables. One id may span a page break when the same panel continues on the next page; a second instance of the heading, or a new heading after the break, opens a new id. Its only use here is grouping rows for sibling inference.
reference_range The printed reference cell, unparsed.
source_test_id The printed vendor test number. A source-scoped entry with a test id then needs the same id; universal entries always participate.

mapping_name requires deterministic_repair or gemini_verified_repair. input_structure_invalid cannot carry a mapping name. The mapper records provenance.stages.input_repair for every result.

When the row carries a specimen, send where it was read as specimen_source (2026-09-24): row for the row's own cell, section_heading for a phrase the row's section prints (its heading or any line of the section; a printed fact for every row of the section), report for a document-level phrase that the application applied to the row. A report-level specimen is soft evidence: it reviews instead of rejecting a candidate whose System it contradicts, and it does not steer retrieval (see Safety). An absent specimen_source is treated as a row fact.

Units and result styles

Send the printed unit surface, not a previously computed canonical unit. The mapper reads units by UCUM grammar, not by a list of known surfaces: every EXAMPLE_UCUM_UNITS value on active laboratory terms in the pinned release parses (tests/test_units_loinc_coverage.py), including annotations (mg/g{creat}, {titer}), groups (mg/(24.h)), microscopy fields (/[HPF], distinct from /[LPF]), arbitrary bracket units ([arb'U]/mL), and a leading solidus (/uL). Report forms go through the same grammar: unit names and SI scale words (Thousand/uL, units/mL, mcmol/L, cu mm), report notes (mg/dL (calc), % by wt; the note is recorded in annotations, never in the dimension; a print that is only a note, (calc) beside a ratio, is an absent unit carrying the note, and the validator reads a bare method word of the reviewed vocabulary in the unit column, calc or IA, as context rather than a unit, 2026-09-25), the release's own display forms (arb U/mL, mg/g creatinine, aligned from config/unit_release_vocabulary.json), and bounded OCR repairs such as x10E3/uL, pcg, ulU/mL and mL/min1.73/ (state recovered, the accepted surface in recovered_surface). The original is always preserved in unit evidence. ratio and titer are non-dimensional result-style tokens and can match only a compatible ratio, fraction or titer term. A true mass/molar dimensional conflict remains a hard rejection; an unfamiliar surface is unrecognized and routes to review, never to a rejection on its own.

The unit contract on the application side: unit_printed is the print and is never rewritten there. LabObservation.unit is either the print or a repaired surface the application verified against the page image or against the expected dimension of a registry-evidenced code; it is never a UCUM canonical and never a control character. The mapper turns control, format and private-use characters into one visible placeholder (U+FFFD) inside a unit. The application differs for names: it turns a non-whitespace control (Cc) or private-use (Co) character into U+FFFD and removes a format character (Cf: zero-width space, soft hyphen, byte-order mark), because a placeholder inside a name would break its OCR verification of names.

The expansion contract (2026-09-25, interface 1): an abbreviation the application expanded with the page's own context (Absolute Neuts (auto) -> Absolute Neutrophils (auto), Est GFR -> Estimated GFR) is sent as mapping_name with input_quality=gemini_verified_repair and the repair operation llm_name_expanded; raw_name stays the print, and the registry is asked for both surfaces (LabObservation.printed_surfaces, 2026-10-06): an approval of the print applies whatever the expansion, which is retrieval evidence (the practice's universal eGFR approval had not fired on a row the model expanded to estimated glomerular filtration rate). Only an approval of the print as printed counts for it; containment runs on the checked name alone, because the print behind a repair carries what the repair removed (Magnesium, RBC B, 01 would otherwise retrieve RBC). The application verifies the expansion with grounding.abbreviation_covers(printed, expanded) before sending it, and the mapper re-verifies with the same rule: every printed token must equal an expanded token, OCR-fold to it (grounding.OCR_FOLD, fold_word), abbreviate it by two or more leading letters after singularising, or be the initials of a run of consecutive expanded tokens (PT -> Prothrombin Time); digits must be equal and every expanded token must be consumed, so an expansion never adds a word (Glucose, Serum for Glucose is refused). A refused expansion maps the print with repair_operations + ("expansion_rejected",), a non-blocking diagnostic expansion_rejected (stage input_repair, details.printed only) and the batch counters expansion_accepted / expansion_rejected. grounding.NAME_FILLER_TOKENS is the shared filler set. There is no abbreviation table on either side: the letters decide.

The value contract (2026-09-25): send value exactly as printed (<, :, flags included; never a canonical or a stripped number), and never send a cell that is not a result. The application decides what a cell is from the page (a comment, a See note/See below pointer whose result sits in the comment, a TNP/Not reportable status, a title); such a row is kept for the note and never reaches the mapper or a grid. The mapper reads the value's shape by grammar anyway (result_style.value_shape) and abstains with value_not_a_result on a pointer or status by every path, as the second net; the optional LabObservation.result_kind (result, comment, not_performed, see_note; additive, never required) is honoured the same way, anything but result. A titer value (<1:64) is a Property fact and a sentence fits only a narrative term (see safety).

Forced outputs

Only a server-side, certified force_exact record can return force_mapped. It matches the repaired canonical name losslessly and requires the configured result style (quantitative, count, fraction, ratio, titer, or qualitative). It bypasses unit and six-axis validation by policy, but the target must still be active and result-eligible. Faxautomation must queue such rows for final authorized approval; do not automatically create an Elation payload.

Certified review-case snapshot

config/mapping_registry.review_cases.20260910.json is an append-only, de-identified bootstrap snapshot built from the supplied 55 review cases. Its initial build preserved all 899 mappings from parent igenex-20260828T000000Z and added 41 exact certified overrides. Revision review-cases-20260910T000000Z-r1 additionally contains the clinician-approved normal mappings Globulin -> 10834-0 and CHOL/HDLC RATIO -> 9830-1. The remaining three records are deliberately not mapped because they are a merged multi-analyte row, a shifted IGF row, or an orphan continuation fragment.

The 41 overrides are not aliases for damaged OCR text. The runtime must first send a canonical reconstructed mapping_name and matching result style. Three overrides are source-scoped Labcorp assay records; they also require source_laboratory="Labcorp". All other certified entries are source-neutral.

This file is usable as an active snapshot only when its recorded parent is the main project's active registry. If production has a newer active snapshot, do not replace it with this older parent. Rebase the 41 force_exact_overrides onto the newer immutable parent through the registry publisher, validate the result, and advance the active pointer atomically. In either case, every force_mapped result remains in the final-approval queue and is not filed to Elation automatically.

Diagnostics

Display provenance.stages.primary_outcome before candidate-level diagnostics. It identifies the decision-driving condition, such as unit_surface_unrecognized, input_structure_invalid, ambiguous_safe_signatures, or missing_required_clinical_fact. Candidate diagnostics remain available for clinical audit but are not the primary cause when they describe an unrelated rejected sibling.