AI Scribes and HIPAA BAAs

Healthcare systems, physician groups, and clinical networks have been adopting generative AI (GenAI) for clinical purposes at a rapid pace. Ambient clinical scribes and automated intake assistants, tools that capture clinician-patient conversations in real time and automatically populate electronic health records (EHRs), are increasingly common. Examples of ambient clinical AI scribes include Glass Health, Freed, Abridge, Suki, and Microsoft's Dragon Copilot.

While clinical GenAI companies frequently pitch seamless workflow integration and promise that their platforms are "turnkey," healthcare professionals must look beyond surface-level marketing claims. Incorporating large language models into clinical settings potentially brings legal exposure under HIPAA, state consumer privacy statutes, and common-law informed consent doctrines.

For in-house counsel as well as compliance, privacy, and risk management teams, this requires auditing how each AI model processes, stores, and routes patient data; negotiating AI-specific covenants and data-use terms; and addressing important healthcare privacy vulnerabilities.

Ambient listening systems help alleviate major clinical documentation burdens, mitigating professional burnout. As clinicians converse naturally with patients, the software captures live audio, converts speech to structured text, and parses out symptoms, diagnoses, and medication plans.

As highlighted in the NIST Artificial Intelligence Risk Management Framework (AI RMF 1.0), introducing automated capture tools and cloud-hosted machine learning into sensitive environments brings significant new privacy risks. Because most ambient AI scribes are categorized as administrative tools and HIPAA-eligible software rather than Software as a Medical Device (SaMD), they bypass FDA premarket review, compromising oversight of both clinical safety and record accuracy. Moreover, a typical HIPAA business associate agreement (BAA) meant for cloud storage or traditional dictation software fails to account for the unique data flows, retention risks, and training mechanisms inherent in AI models.

As Nature has noted, these ambient scribes can, like other forms of GenAI, hallucinate entire clinical statements. Meanwhile, clinical research published across JAMA Network Open continues to evaluate physician experiences with ambient tools, highlighting persistent concerns over clinical accuracy, ongoing editing burdens, and downstream legal liability.

When negotiating or revising vendor contracts for ambient AI technology, legal teams must account for critical operational pitfalls:

1. Secondary Data and Vendor Training Risks

Model training remains the primary battleground in negotiations between healthcare organizations and AI vendors. Vendors, acting as business associates or downstream subcontractors under HIPAA, often seek language in contracts that lets them retain conversational transcripts, audio recordings, and clinical notes to train and refine their proprietary AI algorithms.

Under the HIPAA Privacy Rule, covered entities cannot permit a business associate to use protected health information (PHI) for its own commercial product development unless express, valid patient authorizations have been obtained, or data meets strict federal data de-identification standards under the Expert Determination method or Safe Harbor standard. Routine BAA provisions authorizing "system improvement" or "data aggregation" do not protect covered entities if an AI vendor feeds identifiable clinical audio or transcripts into foundational models without patient authorization.

2. Ambient Audio and Re-Identification

Natural conversation in an exam room is uniquely identifiable. Patients routinely disclose biographical details, family histories, and personal anecdotes, among other personal details, that automated filtering scripts often fail to scrub. Unstructured medical dialogue can present serious re-identification risks compared to structured lab results or standardized claims data. So long as an ambient tool holds onto raw audio recordings, that data remains PHI regardless of any vendor claims regarding automated de-identification or redaction. BAAs must therefore mandate immediate, automated disposal protocols that permanently destroy raw audio the moment transcription is complete.

3. Downstream Sub-Processor Vulnerabilities

Most commercial AI scribes do not run isolated, self-hosted models. Instead, they relay clinical audio across APIs to external cloud hosts and complex webs of technical sub-processors. Because PHI crosses multiple vendor boundaries, covered entities cannot rely on an upstream BAA alone. Every downstream infrastructure provider, model host, and transcription service must be contractually bound to identical restrictions that categorically prohibit vendor data pooling, unauthorized data caching, and third-party telemetry scraping.

To safely operationalize ambient listening and clinical LLMs while remaining compliant, legal and operational teams should implement a multi-tiered governance approach:

• Mandate Zero-Data Retention for Model Training: Require vendors to contractually agree that client audio, transcripts, and metadata will not be retained to train, fine-tune, or evaluate proprietary or commercial models.
• Mandate Sub-Processor Approval and Audit Rights: Require explicit contractual authority to review, approve, or reject any downstream infrastructure provider or API host prior to transferring or processing patient data.
• Update HIPAA Notice of Privacy Practices (NPP) and Informed Consent: While ambient scribes operating strictly as business associates may technically fall under treatment or health care operations (the “T” and “O” of HIPAA’s TPO exceptions), transparency remains paramount. Regulatory bodies like the Federal Trade Commission and state attorneys general increasingly scrutinize deceptive omissions regarding algorithmic monitoring. In accord with ethical standards, healthcare facilities should update their Notice of Privacy Practices, post prominent exam-room signage disclosing active ambient listening, and enforce intake protocols that allow clinicians to obtain on-record consent or pause recording during sensitive physical exams.
• Enforce Third-Party Audits and Technical Safeguards: Require documented proof that conversational streams are fully encrypted end-to-end, both in motion and at rest, and supported by regular third-party penetration testing alongside current SOC 2 Type II examination reports and HITRUST certification.

While ambient listening holds immense promise for clinical documentation, deploying it without adequate guardrails invites severe legal and compliance risks. Healthcare leaders must look past vendor marketing and treat these systems as active data processors that require specialized BAA terms, total sub-processor transparency, and continuous administrative governance.

Disclaimer: This industry insight is for general information only, is not legal advice, and does not create an attorney-client relationship.