HiddenLayer in the News

For years, generative AI vendors have reassured the public and enterprises that large language models are aligned with safety guidelines and reinforced against producing harmful content. Techniques like Reinforcement Learning from Human Feedback have been positioned as the backbone of model alignment, promising ethical responses even in adversarial situations.

But new research from HiddenLayer suggests that confidence may be dangerously misplaced.

Their team has uncovered what they’re calling a universal, transferable bypass technique that can manipulate nearly every major LLM—regardless of vendor, architecture or training pipeline. The method, dubbed “Policy Puppetry,” is a deceptively simple but highly effective form of prompt injection that reframes malicious intent in the language of system configuration, allowing it to circumvent traditional alignment safeguards.

One Prompt to Rule Them All

Unlike earlier attack techniques that relied on model- specific exploits or brute-force engineering, Policy Puppetry introduces a “policy-like” prompt structure— often resembling XML or JSON—that tricks the model into interpreting harmful commands as legitimate system instructions. Coupled with leetspeak encoding and fictional roleplay scenarios, the prompt not only evades detection but often compels the model to comply.

“We found a multi-scenario bypass that seemed extremely effective against ChatGPT 4o,” explained Conor McCauley, a lead researcher on the project. “We then successfully used it to generate harmful content and found, to our surprise, that the same prompt worked against practically all other models.”

The list of affected systems includes OpenAI’s ChatGPT (o1 through 4o), Google’s Gemini family, Anthropic’s Claude, Microsoft’s Copilot, Meta’s LLaMA 3 and 4, DeepSeek, Qwen and Mistral. Even newer models and those fine- tuned for advanced reasoning could be compromised with minor adjustments to the prompt’s structure.

Fiction as a Loophole

A notable element of the technique is its reliance on fictional scenarios to bypass filters. Prompts are framed as scenes from television dramas—like House M.D.—in which characters explain, in detail, how to create anthrax spores or enrich uranium. The use of fictional characters and encoded language disguises the harmful nature of the content.

This method exploits a fundamental limitation of LLMs: their inability to distinguish between story and instruction when alignment cues are subverted. It’s not just an evasion of safety filters—it’s a complete redirection of the model’s understanding of what it is being asked to do.

Extracting the Brain Behind the Bot

Perhaps even more troubling is the technique’s capacity to extract system prompts—the core instruction sets that govern how an LLM behaves. These are typically safeguarded because they contain sensitive directives, safety constraints, and, in some cases, proprietary logic or even hardcoded warnings.

By subtly shifting the roleplay, attackers can get a model to output its entire system prompt verbatim. This not only exposes the operational boundaries of the model but also provides the blueprints for crafting even more targeted attacks.

“The vulnerability is rooted deep in the model’s training data,” said Jason Martin, director of adversarial research at HiddenLayer. “It’s not as easy to fix as a simple code flaw.”

Consequences Beyond the Screen

The implications of this are not confined to digital pranksters or fringe forums. HiddenLayer’s chief trust and security officer, Malcolm Harkins, points to serious real- world consequences: “In domains like healthcare, this could result in chatbot assistants providing medical advice that they shouldn’t, exposing private patient data or invoking medical agent functionality that shouldn’t be exposed.”

The same risks apply across industries: in finance, the potential exposure of sensitive client information; in manufacturing, compromised AI could result in lost yield or downtime; in aviation, corrupted AI guidance could compromise maintenance safety.

In each case, AI systems that were trusted to improve efficiency or safety could become vectors for risk.

RLHF Is Not a Silver Bullet

The research calls into question the sufficiency of RLHF as a security mechanism. While alignment efforts help reduce surface-level misuse, they remain vulnerable to prompt manipulation at a structural level. Models trained to avoid certain words or scenarios can still be misled if the malicious intent is wrapped in the right packaging.

“Superficial filtering and overly simplistic guardrails often mask the underlying security weaknesses of LLMs,” said Chris “Tito” Sestito, co-founder and CEO of HiddenLayer. “As our research shows, these and many more bypasses will continue to surface, making it critical for enterprises and governments to adopt dedicated AI security solutions before these vulnerabilities lead to real-world consequences.”

Rethinking AI Security Architecture

Rather than relying solely on model retraining or RLHF fine-tuning—an expensive and time-consuming process— HiddenLayer advocates for a dual-layer defense approach. External AI monitoring platforms, such as their own AISec and AIDR solutions, act like intrusion detection systems, continuously scanning for signs of prompt injection, misuse and unsafe outputs.

Such solutions allow organizations to respond in real time to novel threats without having to modify the model itself— an approach more akin to zero-trust security in enterprise IT.

The Road Ahead

As generative AI becomes embedded in critical systems— from patient diagnostics to financial forecasting to air traffic control—the attack surface is expanding faster than most organizations can secure it. HiddenLayer’s findings should be viewed as a dire warning: the age of secure-by-alignment AI may be over before it ever truly began.

If one prompt can unlock the worst of what AI can produce, security needs to evolve from hopeful constraint to continuous, intelligent defense.

AUSTIN, Texas – April 23, 2025 – HiddenLayer, the leading security provider for AI models and assets, and Cyera, the pioneer in AI-native data security, today announced a strategic partnership to deliver end-to-end protection for the full AI lifecycle from the data that powers them to the models that drive innovation.

As enterprises embrace AI to accelerate productivity, enable decision-making, and drive innovation, they face growing security risks. HiddenLayer and Cyera are uniting their capabilities to help customers mitigate those risks, offering a comprehensive approach to protecting AI models from pre- to post-deployment. The partnership brings together Cyera’s Data Security Posture Management (DSPM) platform with HiddenLayer’s AISec Platform, creating a first-of-its-kind, full-spectrum defense for AI systems.

“You can’t secure AI without protecting the data enriching it,” said Chris “Tito” Sestito, Co-Founder and CEO of HiddenLayer. “Our partnership with Cyera is a unified commitment to making AI safe and trustworthy from the ground up. By combining model integrity with data-first protection, we’re delivering immediate value to organizations building and scaling secure AI.

Cyera’s AI-native data security platform helps organizations automatically discover and classify sensitive data across environments, monitor AI tool usage, and prevent data misuse or leakage. HiddenLayer’s AISec Platform proactively defends AI models from adversarial threats, prompt injection, data leakage, and model theft.

Together, HiddenLayer and Cyera will enable:

• End-to-end AI lifecycle protection - Secure model training data, the model itself, and the capability set from pre-deployment to runtime.
• Integrated detection and prevention - Enhanced sensitive data detection, classification, and risk remediation at each stage of the AI Ops process.
• Enhanced compliance and security for their customers: HiddenLayer will use Cyera’s platform internally to classify and govern sensitive data flowing through its environment, while Cyera will leverage HiddenLayer’s platform to secure their AI pipelines and protect critical models used in their SaaS platform.

"Mobile and cloud were waves, but AI is a tsunami, unlike anything we’ve seen before. And data is the fuel driving it,” said Jason Clark, Chief Strategy Officer, Cyera. “The top question security leaders ask is: ‘What data is going into the models?’ And the top blocker is: ‘Can we secure it?’ This partnership between HiddenLayer and Cyera solves both: giving organizations the clarity and confidence to move fast, without compromising trust.”

This collaboration goes beyond joint go-to-market. It reflects a shared belief that AI security must start with both model integrity and data protection. As the threat landscape evolves, this partnership delivers immediate value for organizations rapidly building and scaling secure AI initiatives.

“At the heart of every AI model is data that must be safeguarded to ensure ethical, secure, and responsible use of AI,” said Juan Gomez-Sanchez, VP and CISO for McLane, a Berkshire Hathaway Portfolio Company. “HiddenLayer and Cyera are tackling this challenge head-on, and their partnership reflects the type of innovation and leadership the industry desperately needs right now.” 

About HiddenLayer
HiddenLayer, a Gartner-recognized Cool Vendor for AI Security, is the leading provider of Security for AI. Its security platform helps enterprises safeguard the machine learning models behind their most important products. HiddenLayer is the only company to offer turnkey security for AI that does not add unnecessary complexity to models and does not require access to raw data and algorithms. Founded by a team with deep roots in security and ML, HiddenLayer aims to protect enterprise AI from inference, bypass, extraction attacks, and model theft. The company is backed by a group of strategic investors, including M12, Microsoft’s Venture Fund, Moore Strategic Ventures, Booz Allen Ventures, IBM Ventures, and Capital One Ventures.

About Cyera
Cyera is the fastest-growing data security company in the world. Backed by global investors including Sequoia, Accel, and Coatue, Cyera’s AI-powered platform empowers organizations to discover, secure, and leverage their most valuable asset—data. Its AI-native, agentless architecture delivers unmatched speed, precision, and scale across the entire enterprise ecosystem. Pioneering the integration of Data Security Posture Management (DSPM) with real-time enforcement controls, Adaptive Data Loss Prevention (DLP), Cyera is delivering the industry’s first unified Data Security Platform—enabling organizations to proactively manage data risk and confidently harness the power of their data in today’s complex digital landscape.

Contact
Maia Gryskiewicz
SutherlandGold for HiddenLayer
hiddenlayer@sutherlandgold.com

Yael Wissner-Levy
VP, Global Communications at Cyera
yaelw@cyera.io

Austin, TX – April 22, 2025 – HiddenLayer, the leading provider of security for AI models and assets, today announced the release of AISec Platform 2.0, the platform with the most context, intelligence, and data for securing AI systems across the entire development and deployment lifecycle. Unveiled ahead of the RSAC Conference 2025, this upgrade introduces advanced capabilities that empower security practitioners with deeper insights, faster response times, and greater control over their AI environments.

The new release includes Model Genealogy and AI Bill of Materials (AIBOM), expanding the platform’s observability and policy-driven threat management capabilities. With AISec Platform 2.0, HiddenLayer is establishing a new benchmark in AI security where rich context, actionable telemetry, and automation converge to enable continuous protection of AI assets from development to production.

“With the proliferation of agentic systems, context is key to driving meaningful security outcomes,” said Chris “Tito” Sestito, CEO and Co-founder of HiddenLayer. “The new AISec Platform delivers the necessary visibility into interoperating AI systems to ensure and enable security across enterprise and government environments.”

AISec Platform 2.0: Contextual Intelligence for Secure AI at Scale

AISec Platform 2.0 introduces:

• Model Genealogy: Unveils the lineage and pedigree of AI models to track how they were trained, fine-tuned, and modified over time, enhancing explainability, compliance, and threat identification.
• AI Bill of Materials (AIBOM): Automatically generated for every scanned model, AIBOM provides an auditable inventory of model components, datasets, and dependencies. Exported in an industry-standard format, it enables organizations to trace supply chain risk, enforce licensing policies, and meet regulatory compliance requirements.
• Enhanced Threat Intelligence & Community Insights: Aggregates data from public sources like Hugging Face, enriched with expert analysis and community insights, to deliver actionable intelligence on emerging machine learning security risks.
  
• Red Teaming & Telemetry Dashboards: Updated dashboards enable deeper runtime analysis and incident response across model environments, offering better visibility into prompt injection attempts, misuse patterns, and agentic behaviors.

HiddenLayer AISec Platform - Model Genealogy Feature

HiddenLayer AISec Platform - AIBOM Feature

Empowering Security Teams and Accelerating Safe AI Adoption

With AISec Platform 2.0, HiddenLayer empowers security teams to:

• Accelerate model development by reducing the time from experimentation to production from months to weeks.
• Gain full visibility into how and where AI models are being used, by whom, and with what level of access.
• Automate model governance and enforcement through white-glove policy recommendations and telemetry-driven enforcement tools.
• Deploy AI with confidence, transforming it from a high-risk initiative into a scalable, secure enterprise function.

Built for the Future of AI Security

AISec Platform 2.0 also lays the foundation for a new generation of AI threat detection and response. With integrated support for agentic systems, external threat intelligence, and deployment observability, HiddenLayer enables organizations to stay ahead of emerging risks while empowering security and AI teams to collaborate more effectively.

To learn more, schedule a meeting with the HiddenLayer team at RSAC 2025 or book a demo.

About HiddenLayer
HiddenLayer, a Gartner-recognized Cool Vendor for AI Security, is the leading provider of Security for AI. Its security platform helps enterprises safeguard the machine learning models behind their most important products. HiddenLayer is the only company to offer turnkey security for AI that does not add unnecessary complexity to models and does not require access to raw data and algorithms. Founded by a team with deep roots in security and ML, HiddenLayer aims to protect enterprise AI from inference, bypass, extraction attacks, and model theft. The company is backed by a group of strategic investors, including M12, Microsoft’s Venture Fund, Moore Strategic Ventures, Booz Allen Ventures, IBM Ventures, and Capital One Ventures.

Press Contact

Maia Gryskiewicz
SutherlandGold for HiddenLayer
hiddenlayer@sutherlandgold.com

AUSTIN, Texas - March 4, 2024 - HiddenLayer, the leading security provider for artificial intelligence (AI) models and assets, released its second annual AI Threat Landscape Report today, spotlighting the evolving security challenges organizations face as AI adoption accelerates. 

AI is driving business innovation at an unheard-of scale, with 89% of IT leaders stating AI models in production are critical to their organization’s success. Yet, security teams are racing to keep up, spending nearly half their time mitigating AI risks. The report underscores that security is key to unlocking AI’s immense potential. Encouragingly, companies are taking action, with 96% increasing their AI security budgets in 2025 to stay ahead of emerging threats.

The report surveyed 250 IT leaders to shed light on the increasing security risks associated with AI adoption, including the material impact of AI breaches, insufficient protections against adversarial attacks, and a lack of clarity around governance responsibilities.

Key findings include:

• An Increase in AI Attacks: 74% of organizations report definitely knowing they had an AI breach in 2024, up from 67% reporting the same last year, emphasizing the need for companies to act quickly to protect their AI systems.
• Failure to Disclose Incidents: Nearly half (45%) of organizations opted not to report an AI-related security breach due to concerns over reputational damage.
• Material Impact of AI Breaches: 89% say most or all AI models in production are critical to their success. But many continue to operate without comprehensive safeguards with only a third (32%) deploying a technology solution to address threats.
• Internal Debate About Who is Responsible for Security: 76% of organizations report ongoing internal debate about which teams should control AI security, illustrating the need for leaders to clearly define ownership as AI becomes central to business operations.

“Securing AI isn’t just about protection—it’s about accelerating progress,” said Chris "Tito" Sestito, Co-Founder and CEO of HiddenLayer. “Organizations that embrace securing AI as a strategic enabler, not just a safeguard, will be able to move more quickly to realize its benefits. This year’s report shows an encouraging shift: companies are recognizing that comprehensive security accelerates AI adoption, builds trust, and strengthens competitive advantage. HiddenLayer is committed to partnering with those organizations to protect their AI assets so they can continue to innovate.”

Additional trends identified in the report include:

• The rise of “shadow AI:” AI systems being used without official approval is also a growing concern, with 72% of IT leaders flagging it as a major risk.
• AI attack origination: 51% of AI attack sources originate from North America. Other regions contributing to AI threats include Europe (34%), Asia (32%), South America (21%), and Africa (17%).
• Source of AI breaches: 45% identified breaches coming from malware in models pulled from public repositories, while 33% originated from chatbots, and 21% from third party applications.

Looking ahead, the AI security landscape will continue to face even more sophisticated challenges in 2025. Predictions for what’s on the horizon in the next year include:

• Agentic AI as a Target: Integrating agentic AI will blur the lines between adversarial AI and traditional cyberattacks, leading to a new wave of targeted threats. Expect phishing and data leakage via agentic systems to be a hot topic.
• Erosion of Trust in Digital Content: As deepfake technologies become more accessible, audio, visual, and text-based digital content will face a near-total erosion of trust. Expect to see advances in AI watermarking to help combat such attacks.
• Adversarial AI: Organizations will integrate adversarial machine learning into standard red team exercises, testing for AI vulnerabilities proactively before deployment.
• AI-Specific Incident Response: For the first time, formal incident response guidelines tailored to AI systems will be developed, providing a structured approach to AI-related security breaches. Expect to see playbooks developed for AI risks.
• Advanced Threat Evolution: Fraud, misinformation, and network attacks will escalate as AI evolves across domains such as computer vision (CV), audio, and natural language processing (NLP). Expect to see attackers leveraging AI to increase both the speed and scale of attack, as well as semi-autonomous offensive models designed to aid in penetration testing and security research.
• Emergence of AIPC (AI-Powered Cyberattacks): As hardware vendors capitalize on AI with advances in bespoke chipsets and tooling to power AI technology, expect to see attacks targeting AI-capable endpoints intensify.

HiddenLayer’s products and services accelerate the process of securing AI, with its AISec Platform providing a comprehensive AI security solution that ensures the integrity and safety of models throughout an organization's MLOps pipeline. As part of the platform, HiddenLayer’s provides its Artificial Intelligence Detection & Response (AIDR), which enables organizations to automate and scale the protection of AI models and ensure their security in real-time, its Model Scanner, which allows companies to evaluate the security and integrity of their AI artifacts before deploying them, and Automated Red Teaming, which provides one-click vulnerability testing to identify, remediate, and document security risks. 

For more information, view the full 2025 Report.

About HiddenLayer

HiddenLayer, a Gartner-recognized Cool Vendor for AI Security, is the leading provider of Security for AI. Its security platform helps enterprises safeguard the machine learning models behind their most important products. HiddenLayer is the only company to offer turnkey security for AI that does not add unnecessary complexity to models and does not require access to raw data and algorithms. Founded by a team with deep roots in security and ML, HiddenLayer aims to protect enterprise’s AI from inference, bypass, extraction attacks, and model theft. The company is backed by a group of strategic investors, including M12, Microsoft’s Venture Fund, Moore Strategic Ventures, Booz Allen Ventures, IBM Ventures, and Capital One Ventures.

Contact

Maia Gryskiewicz

SutherlandGold for HiddenLayer

hiddenlayer@sutherlandgold.com

HiddenLayer, a leader in security for AI solutions, is now available across three major cloud marketplaces, including Microsoft Azure, AWS, and Google Cloud Platform (GCP). This milestone solidifies HiddenLayer’s position as the trusted choice for AI protection, offering organizations seamless access to its cutting-edge security solutions across cloud, on-premise, and hybrid environments.

HiddenLayer’s platform is designed to meet the stringent security requirements of regulated industries, offering flexible deployment options and compatibility with leading AI frameworks such as TensorFlow and PyTorch. By making its solutions readily available across multiple cloud ecosystems, HiddenLayer ensures enterprises can deploy AI securely without compromising innovation.

As part of its strategic partnership with Microsoft, HiddenLayer was exclusively selected as the sole scanning tool in Microsoft’s AI Studio catalog—a testament to its unmatched capabilities in safeguarding AI models. This achievement is further supported by HiddenLayer’s participation in Microsoft’s Pegasus Program and investment backing from M12, Microsoft’s Venture Fund.

"Securing AI must be an accelerator, not a roadblock," said Chris Sestito, Co-founder and CEO of HiddenLayer. "Expanding across all three major cloud marketplaces ensures that organizations can integrate AI security seamlessly, protecting their models where they are built, deployed, and operated."

For more information, contact our team or explore HiddenLayer’s solutions in your preferred cloud marketplace.

About HiddenLayer

HiddenLayer, a Gartner recognized Cool Vendor for AI Security, is the leading provider of Security for AI. Its security platform helps enterprises safeguard the machine learning models behind their most important products. HiddenLayer is the only company to offer turnkey security for AI that does not add unnecessary complexity to models and does not require access to raw data and algorithms. Founded by a team with deep roots in security and ML, HiddenLayer aims to protect enterprise’s AI from inference, bypass, extraction attacks, and model theft. The company is backed by a group of strategic investors, including M12, Microsoft’s Venture Fund, Moore Strategic Ventures, Booz Allen Ventures, IBM Ventures, and Capital One Ventures.

Contact

Maia Gryskiewicz

SutherlandGold for HiddenLayer

hiddenlayer@sutherlandgold.com

We’re excited to share that HiddenLayer has been recognized as a Representative Vendor in Gartner’s AI Trust, Risk, and Security Management (AI TRiSM) Market Guide. This acknowledgment reinforces our mission to help organizations secure AI systems from emerging threats while maintaining trust, compliance, and innovation. AI security must be prioritized at every development and deployment stage to enable progress. 

This report highlights four key trends redefining AI TRiSM:

• AI TRiSM Teams Take Shape – Organizations are embedding AI security into innovation teams, ensuring AI can scale safely and responsibly.
• Unified AI Runtime Inspection & Enforcement – A proactive, unified approach empowers teams to detect, prevent, and adapt to AI risks in real-time.
• AI Hosting Providers Expand TRiSM Services – AI infrastructure providers are strengthening security offerings, giving enterprises the confidence to push AI capabilities further.
• AI TRiSM Market Consolidation – As AI governance and security converge, the industry is evolving to support broader, more seamless AI adoption.

According to Gartner, AI TRiSM solutions “enable organizations to more safely use AI, ensure AI actions align with organizational intent, keep AI systems secure from malicious actors, and assure confidential data and intellectual property are properly protected.”

This recognition highlights the growing urgency for AI security solutions that go beyond traditional cybersecurity approaches. At HiddenLayer, we’re at the forefront of this challenge—helping organizations mitigate AI risks without slowing innovation.

📄 Gartner members can access the full report here.