Reidentifying Text

Reidentifying Text

Some workflows need to remove PHI for processing but keep the ability to restore it later under authorization — adjudication, patient recontact, linking results back to a record. That is pseudonymization (reversible), not anonymization (irreversible). OpenMed supports it with deidentify(..., keep_mapping=True) to capture a mapping, and reidentify to restore. Everything runs on-device.

When to use

• You need to re-link redacted output to the original record later.
• You are doing GDPR pseudonymization (Art. 4(5)): identifiers held separately, reversible under controlled conditions.
• A reviewer must spot-check redactions against originals.

Do NOT use reversibility when:

• The goal is HIPAA Safe Harbor anonymization or a true anonymous release — a re-identification mapping defeats anonymization. Use method="remove" and keep no mapping.
• The redacted text leaves your trust boundary and the mapping might travel with it. The mapping is the secret; never co-locate it with the de-identified output.

Install

bashCopy code
pip install "openmed[hf]"

Quick start: reversible round-trip

pythonCopy code
import openmed

note = "Patient John Doe (MRN 00481726) seen on 2024-03-02 by Dr. Alice Smith."

# 1) De-identify AND capture the reversal mapping
deid = openmed.deidentify(
    note,
    method="mask",          # or "replace" for realistic surrogates
    keep_mapping=True,       # <-- required to enable reidentify()
    policy="gdpr_pseudonymization",
)

safe_text = deid.deidentified_text       # ship/process this
mapping   = deid.mapping                  # SECRET: store separately, encrypted

# 2) Later, under authorization, restore the original
restored = openmed.reidentify(safe_text, mapping)
assert restored == note

reidentify(deidentified_text, mapping) performs the inverse substitution. The mapping is a dict[str, str] of redacted → original text, produced only when keep_mapping=True.

Use consistent surrogates for stable pseudonyms

For replacement that maps the same identifier to the same surrogate across a document (and reproducibly across runs with a seed):

pythonCopy code
import openmed

deid = openmed.deidentify(
    "Mr. John Doe called. John Doe's MRN is 00481726.",
    method="replace",
    consistent=True,    # same input value -> same surrogate within the run
    seed=42,            # reproducible across runs (implies consistent=True)
    keep_mapping=True,
)
print(deid.deidentified_text)
restored = openmed.reidentify(deid.deidentified_text, deid.mapping)

consistent=True keeps surrogates stable so analytics on the pseudonymized text stay coherent; seed makes them reproducible. Either way, reversal still requires the saved mapping.

Store the mapping securely — separate from the text

The mapping is the re-identification key. Treat it like a secret:

• Never write it to the same store/file/log as the de-identified text.
• Encrypt at rest; restrict access; audit every reversal.
• Key the store by an opaque document id, not by any patient identifier.

pythonCopy code
import json, os
import openmed

note = "Patient John Doe (MRN 00481726), DOB 1970-01-15."
deid = openmed.deidentify(note, method="mask", keep_mapping=True, seed=7)

doc_id = "doc-7f3a"   # opaque id, no PHI

# De-identified text -> general processing store (safe to share downstream)
with open(f"deid/{doc_id}.txt", "w", encoding="utf-8") as fh:
    fh.write(deid.deidentified_text)

# Mapping -> SEPARATE, access-controlled, encrypted vault (illustrative path)
os.makedirs("vault", exist_ok=True)
with open(f"vault/{doc_id}.map.json", "w", encoding="utf-8") as fh:
    json.dump(deid.mapping, fh)   # encrypt this store in production

To re-identify later, load only the mapping for the authorized doc_id:

pythonCopy code
import json, openmed
with open("vault/doc-7f3a.map.json", encoding="utf-8") as fh:
    mapping = json.load(fh)
with open("deid/doc-7f3a.txt", encoding="utf-8") as fh:
    safe_text = fh.read()
original = openmed.reidentify(safe_text, mapping)

Reversible vs irreversible: pick deliberately

method="hash" yields consistent, one-way tokens — good for joining records without ever restoring the original. That is not reversible and needs no mapping.

Hand-off to / from OpenMed

• From extracting-pii-entities: preview the spans first if you want to confirm what will be masked before committing to a reversible run.
• From deidentifying-clinical-text: that skill covers methods, policies, and the safety sweep; this one adds the keep_mapping + reidentify round-trip.
• To downstream NER: run openmed.analyze_text on deid.deidentified_text; re-identify only the final, authorized output — never intermediate logs.

Edge cases & gotchas

• keep_mapping=True is mandatory for reidentify to work; without it deid.mapping is None.
• Result field is .deidentified_text (and .pii_entities, .mapping), not .text/.entities.
• Mapping direction is redacted → original. reidentify substitutes those keys back into the text.
• Mask collisions: with method="mask", identical placeholders (e.g. two [NAME]) cannot be distinguished on reversal. For lossless round-trips use method="replace" with consistent=True/seed, which produces distinct, reversible surrogates.
• Never anonymize-and-keep-mapping. If the release must be anonymous, keep no mapping — a stored mapping makes it pseudonymous, not anonymous.
• Authorization & audit. Re-identification is privileged; log who/when/why and keep the mapping out of general PHI logs.

Standards & references

• GDPR pseudonymization: Regulation (EU) 2016/679, Art. 4(5) & Recital 26 — https://gdpr-info.eu/art-4-gdpr/
• HIPAA de-identification (Safe Harbor / Expert Determination): https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/
• See also OpenMed policy profiles gdpr_pseudonymization and hipaa_safe_harbor (configuring-privacy-policies).