Executive summary

During a routine medical appointment in fall 2025, a HiddenLayer researcher observed their physician using a widely adopted AI-powered clinical documentation assistant, one of the countless real-world examples of AI woven into everyday experiences. The encounter prompted the researcher to include the tool in HiddenLayer’s routine monitoring of publicly visible AI behavior. Within two hours, the Threat Research Group uncovered a severe multi-layered attack chain involving hidden indirect prompt injection and a critically vulnerable PDF-generation service.

Marketed as secure and fully compliant, the AI assistant was used across several countries and documented medical encounters for more than two million patients each week. Its adoption spanned nearly every major clinical discipline, including primary care, cardiology, dentistry, mental health, and emergency and critical care teams.

The vendor’s Trust Center on their website lists full compliance with: SOC 2, HIPAA, ISO 27001, GDPR, PIPEDA, and provincial Canadian health regulations, as well as additional global privacy certifications. However, none of these standards require protections against AI-specific risks such as model compromise, prompt injection, model manipulation, or dynamic content–based backend exploitation. As a result, a globally deployed AI healthcare system operated under the appearance of comprehensive security while remaining deeply vulnerable.

What follows is a detailed account of the attack chain, the systemic risks created by global-scale deployment, and the implications for healthcare, compliance regimes, and AI supply chain security.

Background

AI scribes are rapidly becoming a standard tool in clinical settings worldwide. By relieving documentation burdens, they are being adopted in: Hospitals, Private clinics, Specialty practices, Dental networks, Mental health centers,  Emergency and critical care environments

This particular AI assistant had already:

• Entered multiple international markets
• Integrated with major EHR systems and health networks
• Scribed notes for 2+ million patients weekly
• Supported clinicians across nearly every discipline

With such widespread deployment, any exploitable vulnerability becomes globally consequential.

The scale of adoption meant that even a single overlooked security weakness could create a disproportionate ripple effect across interconnected healthcare systems.

In environments where clinicians and staff depend on seamless integration, a compromised AI workflow does not stay isolated, but instead propagates, quietly and efficiently, across every connected endpoint.

Discovery Timeline

Routine Observation → Research Inquiry

By fall 2025, AI-driven tools had become a quiet but constant presence in clinical workflows. A routine medical visit confirmed this when a HiddenLayer researcher noticed their physician relying on a widely deployed AI documentation assistant. The moment served as a practical example of how seamlessly AI now blends into everyday healthcare, and why its behavior must be continuously monitored.

Within two hours, the HiddenLayer Researchers identified a complete attack chain involving:

• Hidden indirect prompt injection
• Model manipulation
• Abuse of a vulnerable PDF generation library
• Silent exfiltration of credentials, API keys, and healthcare system data

Global Impact Context

Because the AI assistant was used in multiple countries and across millions of patient encounters, the potential impact was borderless and multidisciplinary, extending across entire national healthcare ecosystems.

Unlike localized software vulnerabilities confined to individual organizations, weaknesses in globally distributed AI systems travel with the workflows themselves, meaning a compromise in one clinical context could manifest in another thousands of miles away.

The discovery underscores a broader industry reality. When AI models mediate sensitive processes across borders, a single compromised component becomes a systemic, international risk factor.

Technical Breakdown Of the Attack Chain

Step 1 - Hidden Indirect Prompt Injection in Medical Records

Attackers embed concealed malicious instructions in:

• External medical records
• Referral letters
• Dental images with text metadata
• Psychological evaluation notes
• Cardiology diagnostic reports
• Imported PDFs
• Multilingual documents from global markets

AI models read all context, not just visible text, making stealth prompt injection trivial.

Because the model processed entire documents, including metadata, scanned layers, or character encodings invisible to the clinician, adversarial content could be embedded in ways that bypassed traditional document sanitization steps.

This meant attackers no longer needed direct access to clinical systems; they only needed to place a malicious record anywhere upstream in the patient’s data journey.

Step 2 - Clinician Imports the Compromised Record

Across specialties, from cardiology to dentistry to critical care, clinicians routinely import records.  The AI scribe then absorbs everything in the document, including hidden adversarial content.

There are: No alerts, No filtering, No AI behavior monitoring, No AI detection and response.

In practice, this created a seamless delivery mechanism for malicious content: clinicians performed standard documentation tasks, unaware that a single imported file could quietly redirect the model’s behavior.

With no visibility into model context ingestion and no runtime anomaly detection, the compromise occurred entirely inside the AI workflow, invisible to every stakeholder.

Step 3 - Weak System Prompts Allow Full Model Hijacking

The vendor’s model guardrails were insufficient:

• System prompts lacked enforcement strength
• No prompt injection detection existed
• Outputs were not sanitized
• Backend services fully trusted the model’s output

The attacker’s hidden instructions could now manipulate the model into generating an exploit payload.

The absence of guardrail enforcement converted the system prompt from a control mechanism into a suggestion, one that adversarial instructions could easily override.

Once hijacked, the model effectively became an attacker-controlled interface capable of crafting backend-facing payloads tailored to the system’s own infrastructure.

Step 4 - The Root Cause: Vulnerable PDF Creation Service

The vendor’s PDF document creation library was used across millions of clinical encounters, containing a fatal vulnerability. This service could be manipulated to:

• Access internal files
• Retrieve environment variables
• Enumerate internal systems
• Embed invisible content into PDFs

This gave attackers access to the entire infrastructure, including:

High-Sensitivity Cloud Secrets to all Databases

• Full Google Cloud Platform and AWS credentials
• Private keys
• OpenAI API keys
• Anthropic keys
• Stripe payment credentials
• Redis and Posthog keys
• VectorDB passwords

Global Healthcare Ecosystem Integrations

Including major EHR vendors and international health data systems:

• Epic
• Veradigm EHR
• Halo Connect
• Regional and country-specific health integrations
• Records of all customer organizations and connected providers
• All transcripts from Doctor and patient discussions from the prior 2 weeks

In short, the vulnerability exposed core components required to compromise the vendor’s infrastructure, its global customers, and millions of patients.

Because document-generation services operated as a privileged backend function, rather than an isolated component, the model’s ability to influence PDF creation transformed a single vulnerability into a system-wide access point.

This misalignment between privilege and exposure enabled an attacker to extract sensitive infrastructure secrets that should never coexist within the same trust boundary as dynamic AI-generated content.

The result was a cascading exposure of authentication keys, platform credentials, and integrated healthcare ecosystem access points that collectively supported the organization’s entire operational footprint.

Step 5 - Stealth Exfiltration via PDFs or Webhooks

The attacker could exfiltrate data through:

• Invisible PDF layers
• Metadata fields
• Steganographic embedded objects

Across countries and specialties, clinicians would receive PDFs appearing completely benign. Unaware they carried hidden data or outbound connections.

Because exfiltration occurred through legitimate clinical workflows, the activity blended seamlessly with normal document exchange patterns.

Security monitoring tools, oriented toward network anomalies or endpoint behaviors, were unlikely to detect that routine patient documentation had become an exfiltration channel.

The Compliance Mirage

Fully Certified, Yet Fully Exposed

The vendor held certifications and compliance attestations accepted globally:

• SOC 2
• HIPAA
• ISO 27001
• GDPR
• PIPEDA and Canadian health privacy laws
• Other international privacy standards

Yet none of these frameworks require:

• Prompt injection protections
• AI model integrity validation
• Detection of compromised model behavior
• Guardrails on AI-to-backend interactions
• Red-teaming of AI-driven workflows
• Analysis of AI-generated dynamic content for malicious patterns
• Isolation between AI models and sensitive backend systems

Thus a vendor could be:

• Fully certified
• Fully compliant
• Fully trusted by providers - globally

…while remaining vulnerable to an attack chain that started with an AI model that required no elevated access and could operate entirely undetected.

Certification ≠ AI security.

The case demonstrated how certification regimes , designed for traditional software systems, fail to account for the dynamic, emergent behavior of AI models interacting with real-world data.

These standards verify documentation, policy controls, and static system configurations, but they do not evaluate how an AI model behaves under adversarial conditions or how its outputs influence downstream infrastructure.

As a result, organizations relying on compliance as a proxy for security unknowingly operate AI systems that may meet every regulatory checkbox while remaining susceptible to readily exploitable model-level attacks.

Societal Implications at Global Scale

International Patient Trust

Over two million patients weekly, across multiple countries and specialties had medical encounters documented through this system.  A breach at this scale threatens trust in healthcare institutions, international data-sharing agreements, and ross-border privacy expectations.

Because clinical documentation forms the basis for diagnoses, billing, legal records, and inter-provider communication, the integrity of these workflows is foundational to patient trust.

A systemic compromise does not merely expose data; it undermines confidence in the accuracy and confidentiality of the entire healthcare experience.

Regulatory Gaps Across Jurisdictions

No major global compliance regime currently requires AI-specific security controls.

This case shows the urgent need for:

• AI-specific extensions to HIPAA
• GDPR AI security amendments
• ISO standards for ML security
• Mandatory AI red teaming
• International AI supply chain validation

The incident highlights a transnational regulatory blind spot: healthcare privacy laws govern data, but not the security properties of the AI systems that transform, interpret, and transmit that data.

As AI becomes embedded across borders, regulatory frameworks must evolve beyond static controls and incorporate operational requirements for monitoring, testing, and defending AI model behavior in real time.

Business Implications

Substantial Liability Exposure

Across countries, this could trigger: HIPAA penalties (US), GDPR fines (EU), PIPEDA violations (Canada), Lawsuits from healthcare networks, and Class-action litigation from millions of patients.

Because the vulnerability compromised both regulated health data and critical infrastructure credentials, liability would extend beyond privacy violations to include operational disruptions, downstream breaches at partner organizations, and failures to maintain adequate security controls.

Cross-border legal exposure would multiply, as each jurisdiction evaluates harm under its own regulatory framework.

Cross-Border Supply Chain Collapse

Healthcare systems rely on interconnected digital infrastructures. A vulnerability in one AI component can propagate through: EHR integrations, Referral systems, Insurance claims pipelines, and Shared health information exchanges.  This incident highlights how AI components, especially those embedded in documentation workflows, function as supply chain amplifiers. A compromise introduced at one integration point can silently travel through multiple dependent systems, triggering failures far from the original entry point.

Such propagation is uniquely accelerated in AI-driven environments where data ingestion, summarization, and distribution occur automatically and continuously.

Market Shift Toward Secure AI

Global healthcare buyers will increasingly demand:

• AI model security validation and run time protection against model attacks
• Proof of adversarial ML testing
• Independent model security reviews

Procurement criteria are already evolving: healthcare organizations are beginning to treat AI vendors less like traditional SaaS providers and more like participants in a high-risk, interconnected digital ecosystem.

Security assurances must now include demonstrated resilience against adversarial manipulation, not just compliance attestations or claims of encrypted storage.

Strategic Recommendations

For Healthcare Providers (Globally)

• Require AI security assessments beyond compliance certifications
• Demand model integrity and prompt injection testing
• Audit document-generation and file-handling workflows
• Require segmentation between AI and infrastructure

For AI Vendors

• Implement strong prompt injection defenses that go well beyond guardrails
• Harden system prompts and monitor model behavior
• Sanitize and validate model outputs
• Sandbox PDF and document-generation services
• Strictly separate secrets and credentials from AI-accessible components

For Regulators

• Update global standards (HIPAA, GDPR, PIPEDA, ISO) to include AI-specific operational security requirements for AI model runtime protection that can detect and respond to various AI attacks
• Require auditing of AI supply-chain components
• Mandate red-teaming and adversarial testing for clinical AI tools

Conclusion

Substantial Liability Exposure

This AI scribe deployed across multiple countries, across virtually every medical specialty, and documenting the care of over two million patients per week appeared trustworthy due to its full compliance posture.  But compliance did not address its most dangerous risks.

The discovery of this attack chain reveals a critical, global truth:

Traditional security frameworks are not designed to protect AI systems, nor the healthcare infrastructures that depend on them.

Until compliance evolves to incorporate AI-specific operational security controls, every globally deployed AI medical assistant represents a potential systemic healthcare vulnerability.

This case illustrates how rapidly an overlooked model-level weakness can escalate into an ecosystem-wide exposure when embedded in international clinical workflows.AI systems now sit at the operational center of healthcare delivery, and their security cannot be evaluated through the lens of traditional software controls alone.

Protecting global healthcare infrastructure requires recognizing AI not merely as a tool, but as a new class of technical dependency, one that must be secured with the same rigor applied to any other critical system.