How to De-Identify Audio Data Under HIPAA

This is the implementation guide, not the regulation. It walks the five steps of taking clinical or conversational audio to a state you can defend as de-identified: knowing which identifiers are in scope, choosing a method, detecting protected health information automatically, redacting it in both transcript and audio, and leaving an audit trail that survives review.

Prerequisites: familiarity with the HIPAA Privacy Rule at 45 CFR 164.514, a working understanding of speech-to-text systems, and enough Python to adapt the examples.

This is not legal advice. De-identification obligations depend on your data, your use case, and your organisation’s policies. Take qualified counsel before any production deployment.

Overview: what the rule actually requires

HIPAA gives two routes to de-identified status, and only two.

Safe Harbor (45 CFR 164.514(b)(2)) is the bright line: remove all 18 listed identifiers, and have no actual knowledge that the remaining information could identify an individual. No statistician required. It tends to over-redact, which costs utility.

Expert Determination (45 CFR 164.514(b)(1)) is the analytic route: a person with appropriate knowledge and experience of statistical and scientific de-identification principles determines that the re-identification risk is very small, and documents the methods and the result. It can retain far more utility, and it costs money and time.

There is no third option, and a “mostly de-identified” dataset is simply PHI.

Step 1: know the 18 identifiers

The Safe Harbor list at 45 CFR 164.514(b)(2)(i) runs (A) through (R). Reproduced in full, because partial lists are where this goes wrong:

1. Names
2. All geographic subdivisions smaller than a state, including street address, city, county, precinct and ZIP code, with the limited exception for the first three ZIP digits where the population of that area exceeds 20,000
3. All elements of dates except year, for dates directly related to an individual, and all ages over 89
4. Telephone numbers
5. Fax numbers
6. Email addresses
7. Social Security numbers
8. Medical record numbers
9. Health plan beneficiary numbers
10. Account numbers
11. Certificate and license numbers
12. Vehicle identifiers and serial numbers, including licence plate numbers
13. Device identifiers and serial numbers
14. Web URLs
15. IP addresses
16. Biometric identifiers, including finger and voice prints
17. Full-face photographs and any comparable images
18. Any other unique identifying number, characteristic or code

Items 5, 9 and 11 are the ones most often dropped from summaries, and item 18 is the one that does the most work in practice.

What is different about audio

Speech carries identifiers the transcript does not. A voice print is explicitly item 16. Ordinary conversational audio is the contested case: the identifying signal is in pitch, accent and speaking style rather than in any stored template. In a large, diverse cohort that signal is weak. In a rare-disease cohort of forty people it may not be.

Ambient content is in scope. A PA announcement naming a clinician, a background conversation, a ward-specific soundscape: each can carry item 1 or item 2 without appearing in the intended speech at all.

Contextual re-identification is item 18’s territory. “My son’s birthday is next week” is not a date under item 3, but combined with a diagnosis and a region it can narrow a population sharply.

Checklist

• Document which of the 18 identifiers actually occur in your audio, per source and per recording condition
• Identify indirect identifiers that only bite in combination
• Decide explicitly whether voice characteristics are in scope for your cohort, and record the reasoning

Citations: 45 CFR 164.514(a), 45 CFR 164.514(b)(2)

Step 2: choose the method

Choose Safe Harbor when you have no statistical expertise on hand, the downstream use tolerates coarse data, or you need a process an auditor can check line by line. It is the default for most operational work.

Choose Expert Determination when utility is the constraint: a research design that needs real age distributions, seasonality that dies if you strip everything but the year, or a geographic signal that matters. Budget for a qualified expert and for the time their analysis takes.

A practical decision: if data utility is not the binding constraint, Safe Harbor is almost always the cheaper correct answer. If it is, the expert engagement is not optional, and a “hybrid” that applies Safe Harbor to the obvious identifiers and quietly keeps the rest is not a recognised method. Retaining anything on the list requires the expert determination that covers it.

For audio specifically: under Safe Harbor, spoken names, dates, phone numbers and addresses all go, and voice anonymisation becomes a judgement call. Under Expert Determination you may be able to keep coarse dates or region-level geography if the analysis supports it.

Checklist

• Record which method you are using, and why
• Under Expert Determination, contract the expert before processing begins and keep their documentation
• Under Safe Harbor, build the removal checklist from all 18 items, not a summary of them

Citations: 45 CFR 164.514(b)(1), 45 CFR 164.514(b)(2)

Step 3: detect PHI automatically

Three components, in order.

Transcription. Use a speech-to-text service covered by a signed BAA, or run on-premise. Enable diarization where there are multiple speakers, and request word-level timestamps: without them you cannot align a detected identifier back to the audio, and step 4 becomes guesswork.

Entity recognition. General-purpose NER under-performs on clinical text. Combine a medical model with deterministic pattern matching, because the two fail differently:

• Regex for the structured identifiers: phone numbers, SSNs, account and record numbers
• Date parsing across the formats people actually speak, including partial and relative dates
• Gazetteers for names, facilities and clinician lists specific to your source

Metadata scrubbing. The file is an identifier too. Strip embedded metadata, rewrite filenames that encode patient or date information, and check for creation timestamps that reconstruct an admission date.

import re
from presidio_analyzer import AnalyzerEngine, RecognizerRegistry

# Custom recognizer for an 8-digit medical record number
class MRNRecognizer:
    def analyze(self, text, entities):
        matches = []
        for match in re.finditer(r'\b\d{8}\b', text):
            matches.append({
                "entity_type": "MEDICAL_RECORD_NUMBER",
                "start": match.start(),
                "end": match.end(),
                "score": 0.95,
            })
        return matches

registry = RecognizerRegistry()
registry.add_recognizer(MRNRecognizer())
analyzer = AnalyzerEngine(registry=registry)

transcript = """Dr. Jane Smith saw patient John Doe (MRN 98765432)
                on 03/15/2024 at Memorial Hospital."""

results = analyzer.analyze(
    text=transcript,
    entities=["PERSON", "DATE_TIME", "LOCATION", "MEDICAL_RECORD_NUMBER"],
    language="en",
)

for r in sorted(results, key=lambda x: x.score, reverse=True):
    print(f"{r.entity_type}: {transcript[r.start:r.end]} ({r.score})")

Checklist

• Speech-to-text service under BAA, or on-premise
• Medical NER plus rule-based matching, not one or the other
• Word-level timestamps enabled
• Detection measured on representative validation material, reporting recall per identifier type
• Metadata and filename scrubbing in the pipeline, not as a manual step

Step 4: redact transcript and audio

In the transcript, replacement beats deletion. Bracketed placeholders keep the sentence intact and keep the data usable:

“John Doe was admitted on 01/15/2024” becomes “[PATIENT_NAME] was admitted on [DATE]”

Deleting outright leaves gaps that break downstream parsing and tell the reader nothing about what was removed. Generalisation, replacing an age with a band, is an Expert Determination technique and needs that determination behind it.

In the audio, three options with different trade-offs:

• Tone replacement. Overwrite the segment with a fixed tone. Unambiguous, easy to verify, intrusive to listen to.
• Silence. Less jarring, but a silence surrounded by context can still imply what was removed, and it is harder to distinguish from a natural pause during QC.
• Voice conversion. Pitch shifting or formant modification preserves the content while attacking the biometric signal. It addresses item 16 rather than the spoken identifiers, so it complements redaction rather than replacing it.

import librosa
import numpy as np
import soundfile as sf

def generate_tone(duration_sec, sample_rate=16000, frequency=1000):
    t = np.linspace(0, duration_sec, int(sample_rate * duration_sec))
    return 0.3 * np.sin(2 * np.pi * frequency * t)

def redact_segments(audio_file, segments, output_file):
    """segments: list of (start_sec, end_sec) aligned from the transcript."""
    audio, sr = librosa.load(audio_file, sr=None)
    redacted = audio.copy()

    for start_sec, end_sec in segments:
        start, end = int(start_sec * sr), int(end_sec * sr)
        redacted[start:end] = generate_tone(end_sec - start_sec, sr)

    sf.write(output_file, redacted, sr)

redact_segments("interview.wav", [(2.5, 3.2), (15.8, 16.5)], "interview_redacted.wav")

Verification is part of the step, not a follow-up. Sample a proportion of output for human review sized to your risk, measure recall against a labelled set rather than eyeballing it, and log every redaction with its timestamp, entity type and confidence so the audit can reconstruct what the system decided and why.

Checklist

• Redaction strategy chosen and documented for transcript and for audio
• Timestamp alignment verified between the two
• Human review on a defined sample, with results recorded
• Per-redaction logging in place

Step 5: document and audit

De-identification you cannot evidence is de-identification you cannot rely on.

Chain of custody. For every file: original and de-identified names, processing timestamp, identifiers detected with type and confidence, redaction method applied, and review status. Store append-only. Retain for at least six years under 45 CFR 164.530(j).

Attestation. A written, signed record of the method used, the date, and the verification performed. Under Safe Harbor that is confirmation that all 18 identifiers were removed and no actual knowledge of residual identifiability remains. Under Expert Determination it is the expert’s own documentation. This is what goes into an IRB submission or a data sharing agreement.

Ongoing monitoring. Models drift and pipelines rot. Sample quarterly to catch degradation in detection recall. Define the incident path before you need it: contain, analyse root cause, assess against the Breach Notification Rule, remediate the detection or redaction step that failed. Version-control the scripts, models and rules so an audit can tell which version processed which file.

Citations: 45 CFR 164.530(j), 45 CFR 164.404, 45 CFR 164.514(b)(2)(i)

Required versus recommended

Required

• Remove all 18 identifiers under Safe Harbor, or obtain expert certification under Expert Determination
• Retain de-identification documentation for at least six years
• Cover every processor, including speech-to-text providers, with a Business Associate Agreement

Recommended

• Human review on a defined sample of output, sized to residual risk
• Voice anonymisation where re-identification from voice is a real concern for the cohort
• Quarterly sampling for detection drift
• Version control across scripts, models and redaction rules

Where re-identification risk concentrates

• Small cohorts. Under a hundred individuals, ordinary demographic combinations start to be identifying on their own. This is where Expert Determination earns its cost.
• Public release. Anything published can be joined against external data you do not control. Redact harder than internal use requires.
• Distinctive speech. A rare accent or an atypical speech pattern narrows the population before any content is considered, and it is exactly the case where a clinical dataset is most likely to be small.

Related

• Clinical speech data: collection and annotation controls, documented provenance, EEA processing where required
• Healthcare AI solutions: what an engagement looks like end to end
• HHS de-identification guidance: the official interpretive guidance
• 45 CFR 164.514, full text: the regulation itself, current text