UMLS integration and production setup

Development of individual package components can use fixtures, but the production pipeline does not bypass UMLS. Every production mapping loads the pinned local UMLS index, runs ScispaCy entity extraction, and uses the resulting concept evidence before SapBERT ranking.

UMLS is isolated behind ConceptLinker, but it is required by the production mapper. MockConceptLinker and the deterministic ranker exist only for unit tests; they cannot construct a production mapper.

Recommended download

For this OCR-to-LOINC module, download only the current MRCONSO.RRF artifact from the UMLS Metathesaurus page. The current NLM listing is 2026AA: approximately 492 MB compressed and 2.2 GB uncompressed. It contains the names, synonyms, concepts, and source codes needed by our required local linker.

Do not download the UMLS Full Release, Metathesaurus Full Subset, Value Sets, RxNorm, or SNOMED CT for this module. The full UMLS archive is approximately 5.3 GB compressed and 40.1 GB uncompressed, which is not appropriate for the available disk space. NLM UMLS file sizes

The license is approved, but credentials and licensed files remain local deployment inputs. Use the API key from the UTS profile through an environment variable:

$env:UMLS_API_KEY = "your-key-from-uts-profile"
python -m loinc_mapper download-umls --release-url "https://download.nlm.nih.gov/umls/kss/2026AA/umls-2026AA-mrconso.zip" --output assets/umls/2026AA/mrconso.zip

The CLI also loads a local .env file when run from the workspace. The clean URL above is the official NLM MRCONSO.RRF link; remove tracking query parameters such as _gl from copied browser URLs. Official NLM link

If the package has not been installed yet, run from the repository root with $env:PYTHONPATH = "$PWD\src" before invoking python -m loinc_mapper.

Use the current MRCONSO.RRF download URL from the UMLS page; do not use the umls-full-release URL. Record the exact release selected in the local manifest.

Extract the archive outside Git, then build the pinned local index:

python -m loinc_mapper prepare-umls --asset-path assets/umls/RELEASE_ID --output-index assets/umls/RELEASE_ID/umls.sqlite3 --release RELEASE_ID
python -m loinc_mapper validate-assets --umls-path assets/umls/RELEASE_ID

prepare-umls reads MRCONSO.RRF, creates a SQLite lookup index, and writes manifest.json with the release, source file, checksum, and schema version. OfflineUMLSLinker refuses an index whose checksum does not match the manifest.

After the base index is built, materialize the LOINC-linked FTS5 and character 3-gram retrieval structures. This is required for the current production configuration:

python -m loinc_mapper upgrade-umls --asset-path assets/umls/2026AA

This can require several additional gigabytes because the source MRCONSO file contains millions of terms. Check available disk space first and do not run mapping against the database while it is being upgraded. The command updates the SQLite file in place, writes the checksum only after the tables are complete, and can take a long time on a disk-constrained Windows machine.

The completed 2026AA workspace artifact contains:

Artifact	Purpose	Current verification
`umls.sqlite3`	Local UMLS terms and derived LOINC-linked retrieval	`cui_loinc`: 104,334 rows; `loinc_terms`: 930,125 rows; `loinc_term_ngrams`: 44,236,264 rows
`manifest.json`	Release, source, schema, and SQLite checksum	`schema_version: 3`, release `2026AA`

Compact Serving Artifact

The raw upgraded SQLite file is kept for audit/rebuilds. It is not the ideal worker-time database because it includes broad UMLS data and a large explicit trigram table. Build a separate compact serving artifact after the raw release and catalog have passed validation:

python -m loinc_mapper build-umls-serving `
  --asset-path assets/umls/2026AA `
  --catalog assets/loinc/2.82/catalog_v5.sqlite3 `
  --output assets/umls/2026AA/serving/umls_loinc.sqlite3

This build verifies the raw source checksum once, then retains every UMLS alias whose CUI bridges to an active local LOINC code. It creates exact, FTS5, and trigram retrieval structures in the derived file plus serving_manifest.json. It does not edit or delete the raw licensed artifact.

At runtime OfflineUMLSLinker automatically prefers the serving artifact. It holds one read-only SQLite connection per worker and caches repeated mentions. Worker startup validates the pinned release, manifest shape, and file size; full checksums remain a build/deployment validation operation. Expected provenance is umls_retrieval_backend=serving_exact+fts5+trigram.

Validate the finished artifact before starting a worker:

$env:PYTHONPATH = "$PWD\src"
python -m loinc_mapper validate-assets `
  --umls-path assets/umls/2026AA

The mapper provenance must report umls_retrieval_backend=serving_exact+fts5+trigram once the serving artifact is provisioned. A result reporting exact_only_bridge is using an incomplete or stale UMLS index.

After installing the required dependencies and an appropriate ScispaCy model, ScispaCyUMLSLinker extracts entities and links them against this local index. The local LOINC catalog still validates the final code.

Enable it for mapping with:

python -m loinc_mapper map `
  --name "low density lipoprotein cholesterol" `
  --unit "mg/dL" `
  --umls-path assets/umls/RELEASE_ID `
  --scispacy-model en_core_sci_md `
  --vector-index assets/loinc/2.82/terms.faiss

No UMLS API keys or licensed Metathesaurus files belong in this repository.

For refresh and rollback procedures, see Maintenance. For embedding this package in the fax service, see Deployment.