Skip to content

CLI and integration

Map one observation:

python -m loinc_mapper map `
  --name "LDL" `
  --unit "mg/dL" `
  --specimen serum `
  --source-laboratory "Wellspan Health" `
  --observation-domain LAB_RESULT `
  --loinc-class-hint CHEM `
  --panel-context "lipid panel" `
  --umls-path assets/umls/2026AA `
  --scispacy-model en_core_sci_md `
  --vector-index assets/loinc/2.82/terms.faiss `
  --vector-metadata assets/loinc/2.82/terms.json

The single-row command accepts --source-laboratory because reviewed local mapping evidence is intentionally scoped to the performing laboratory. Batch and report inputs carry the same source_laboratory field in each row.

Map a CSV batch with a raw_name column:

python -m loinc_mapper map-batch `
  --input observations.csv `
  --output mapped.jsonl `
  --umls-path assets/umls/2026AA `
  --scispacy-model en_core_sci_md `
  --vector-index assets/loinc/2.82/terms.faiss `
  --vector-metadata assets/loinc/2.82/terms.json

Map a raw OCR report. This command executes row extraction before the same required mapping pipeline and writes one JSON object per extracted row:

python -m loinc_mapper map-report `
  --input report.txt `
  --output mapped.jsonl `
  --umls-path assets/umls/2026AA `
  --scispacy-model en_core_sci_md `
  --vector-index assets/loinc/2.82/terms.faiss

Build release artifacts

The CSV release remains immutable source data. Compile it once into the worker artifact used at runtime:

python -m loinc_mapper build-catalog `
  --output assets/loinc/2.82/catalog_v5.sqlite3 `
  --release 2.82

Build the catalog-wide SapBERT sidecar after installing the production package (FAISS is a required dependency):

python -m loinc_mapper build-vectors `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --output assets/loinc/2.82/terms.faiss `
  --metadata assets/loinc/2.82/terms.json `
  --aliases-per-code 4 `
  --batch-size 128

IndexFlatIP is exact inner-product search over normalized vectors, which is cosine similarity. The metadata pins the LOINC release and SapBERT model. Do not mix a vector sidecar with a different release or model. The runtime loads the index once per worker and reuses cached query embeddings/hits; it does not re-encode the full catalog for each row. Every non-verified-registry row still executes UMLS, validation, ranking, and the confidence gate independently.

Build canonical code-level vectors from an already-built index without another full SapBERT encoding pass:

python -m loinc_mapper build-code-vectors `
  --index assets/loinc/2.82/terms.faiss `
  --metadata assets/loinc/2.82/terms.json

Optional official-class shards are derived from the same global index. They are a benchmarked latency optimization, not a separate LOINC database:

python -m loinc_mapper build-vector-shards `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --index assets/loinc/2.82/terms.faiss `
  --metadata assets/loinc/2.82/terms.json `
  --output assets/loinc/2.82/shards

Compact UMLS Serving Index

prepare-umls and upgrade-umls retain the raw licensed source needed for audit/rebuilds. Build the compact runtime artifact after source validation:

python -m loinc_mapper build-umls-serving `
  --asset-path assets/umls/2026AA `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --output assets/umls/2026AA/serving/umls_loinc.sqlite3

This keeps aliases only when their CUI bridges to an active local LOINC code. It can take time and disk during the explicit build; do not invoke it at request time. Once present, runtime uses serving_manifest.json and avoids rehashing raw UMLS during worker startup.

Profiles and Worker Batches

Stage flags live in .env; see .env.example. Production refuses disabled evidence stages. Use PIPELINE_PROFILE=evaluation only for comparisons; its results are experimental and cannot create Elation payloads.

python -m loinc_mapper evaluate `
  --input evaluation/gold/holdout.csv `
  --details evaluation/gold/profile_matrix.json `
  --profile-matrix `
  --umls-path assets/umls/2026AA

The matrix reports full pipeline, safe fast path, no-UMLS, no-FAISS, no-UMLS/no-FAISS, and deterministic-only diagnostics, with metrics and stage p50/p95 timings.

For the main fax project, send a versioned MappingBatch instead of starting models for each fax:

python -m loinc_mapper run-worker `
  --input batch.json `
  --output batch-result.json `
  --umls-path assets/umls/2026AA

The batch preserves report boundaries while mapping all observations through one warm Mapper.map_many() call. See Deployment for the payload schema and Cloud Run Job settings.

Clinician Review and Registry Snapshots

The current production review path is storage-free in this package and clinician-friendly in the main project. It validates decisions and creates immutable candidate snapshots; the main project owns reviewer authentication, the UI, durable review events, Cloud Storage, and Cloud Run orchestration. See Clinician-governed learning for the full contract.

The old JSONL queue commands remain available for local diagnostics and older integrations, but import-reviews is compatibility-only and never enables the registry fast path. New clinical approvals should use the ReviewCase CSV workflow below.

Create review cases in the main project with build_review_case, then export them to a doctor-friendly CSV:

python -m loinc_mapper export-review-csv `
  --cases review_cases.jsonl `
  --output clinician_review_template.csv

Validate the completed CSV without modifying an active registry:

python -m loinc_mapper validate-review-csv `
  --cases review_cases.jsonl `
  --input clinician_reviewed.csv `
  --registry config/mapping_registry.json `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --output review_validation.json

Compile only fully valid approvals into a candidate snapshot:

python -m loinc_mapper compile-review-snapshot `
  --cases review_cases.jsonl `
  --input clinician_reviewed.csv `
  --registry config/mapping_registry.json `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --registry-output candidate_registry.json `
  --manifest-output candidate_manifest.json

The candidate has registry_status=candidate, so a worker refuses to load it. The main project must finalize it before publishing an active snapshot. For normal authorized clinician publication, use PublicationPolicy.CLINICIAN_FAST_REPLAY with a mapper factory configured by PipelineSettings.clinician_review_replay(). That runs one batched exact registry replay over changed reviewed rows and does not load UMLS, ScispaCy, FAISS, or SapBERT. PublicationPolicy.STRICT_REPLAY retains original/unseen and fast-equivalence replay for scheduled release audits. There is no force-map field and no clinician-controlled fast-path switch.

Export uncertain rows with their complete candidate explanation for local diagnostics or a legacy migration:

python -m loinc_mapper review-queue `
  --input observations.csv `
  --output review_queue.jsonl `
  --umls-path assets/umls/2026AA

For an existing legacy JSONL queue only, import explicit approvals:

python -m loinc_mapper import-reviews `
  --input review_queue.jsonl `
  --output config/mapping_registry.reviewed.json

Finalized registry snapshots change retrieval in newly launched jobs. They do not silently retrain SapBERT. A feature ranker can be trained later from detailed evaluation records, then compared against the champion with the precision and dangerous-false-positive gate before deployment:

python -m loinc_mapper train-ranker `
  --input evaluation/stress_test/production_detailed_report.json `
  --output assets/rankers/adarank-compatible.json

Audit the existing registry before deciding which local records are genuinely universal. This is read-only; it never promotes a source-specific mapping:

python -m loinc_mapper audit-registry-scopes `
  --registry config/mapping_registry.json `
  --output evaluation/registry_scope_audit.json

Export a two-layer registry for a new integration without changing the legacy runtime file:

python -m loinc_mapper export-layered-registry `
  --registry config/mapping_registry.json `
  --output config/mapping_registry.layered.json

Cluster unresolved rows for review. The output contains recurrence counts, source laboratories, axis facts, and a suggested review scope. It never writes an active mapping:

python -m loinc_mapper cluster-review-queue `
  --input review_queue.jsonl `
  --output evaluation/review_clusters.json

Build a MIMIC review set

MIMIC-IV v3.1 is handled as a real-world candidate corpus first. Its local itemid labels are not automatically trusted as LOINC gold labels:

python -m loinc_mapper prepare-mimic `
  --labevents path/to/labevents.csv `
  --d-labitems path/to/d_labitems.csv `
  --output evaluation/mimic/candidates.csv `
  --mapping-template evaluation/mimic/item_mapping_template.csv

After expert review of the template:

python -m loinc_mapper build-mimic-gold `
  --candidates evaluation/mimic/candidates.csv `
  --mapping evaluation/mimic/item_mapping_template.reviewed.csv `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --output evaluation/mimic/gold.csv

The preparation step excludes patient, encounter, specimen, timestamp, and provider identifiers from its output. Keep the downloaded source and derived files under the PhysioNet agreement and outside Git.

Audit labels before evaluation

Before interpreting accuracy, audit whether the input name and expected LOINC code agree on method, specimen, duration, and unit dimension:

python -m loinc_mapper audit-labels `
  --input evaluation/stress_test/stress_test_output_v2.csv `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --output evaluation/stress_test/stress_test_output_v2_label_audit.json

Rows such as HDL Cholesterol (Calc) expected as methodless 2085-9 are flagged rather than silently relabeled. A clinical reviewer must decide whether Calc is a real method, a local panel suffix, or an incorrect label.

Audit the Toward Health screening seed files before registry promotion:

python -m loinc_mapper audit-screening `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --input data/initial_labs_data_temp/toward_health_screening_loinc_cpt_mapping.csv `
          data/initial_labs_data_temp/toward_health_screening_specialty_questionable.csv `
          data/initial_labs_data_temp/toward_health_screening_top500.csv `
  --output evaluation/screening/toward_health_screening_audit.json

This command is review-only. It reports blank, missing, inactive, non-laboratory, exact-name, and release-name-different rows; it never writes entries into config/mapping_registry.json.

Evaluation, report, and CSV batch mapping use map_many. This shares safe infrastructure such as repeated UMLS lookups and SapBERT query embeddings; it does not merge rows or share a clinical decision. Each result retains its own candidate list, six-axis evidence, unit decision, confidence, and provenance.

The Python boundary is:

from loinc_mapper import LabObservation, build_mapper

mapper = build_mapper(
    core_path,
    rich_path,
    common_names_path,
    registry_path,
    map_to_path,
    umls_path,
    "en_core_sci_md",
    "cambridgeltl/SapBERT-from-PubMedBERT-fulltext",
    vector_index_path="assets/loinc/2.82/terms.faiss",
    vector_metadata_path="assets/loinc/2.82/terms.json",
)
result = mapper.map(LabObservation(raw_name="Hb A1C", unit="%"))

For category-aware routing, pass observation_domain, loinc_class_hint, and optional panel_context in the LabObservation. LAB_RESULT is the hard laboratory scope. LOINC class and panel context are advisory retrieval/ranking evidence, so an incorrect section label cannot reject a safe clinical sibling. See Category-aware routing and Universal multi-lab mapping.

Production construction requires umls_path, scispacy_model, sapbert_model, and the pinned FAISS vector sidecar. The CLI auto-detects the default sidecar, but explicit paths are recommended for reproducible runs. If the sidecar is missing, production construction fails instead of silently falling back to direct encoder reranking. map-report runs OCR row extraction and emits mapping results with accepted Elation payloads.

The CLI uses the generated schema-6 catalog_v5.sqlite3 automatically when it is present. Pass --catalog explicitly in deployment scripts so the selected artifact is visible and reproducible. See Deployment for the complete worker integration sequence and Maintenance for release refresh procedures.