Policy

Leading AI Models Fail Safety Tests on Simulated Warfare, Jailbreak When Reframed as Games

Vals research finds OpenAI, Anthropic, and Google models identify civilians as military targets when told scenarios are games; Anthropic models performed worst overall.

Last verified:

Vals Finds Major Models Identify Civilians as Military Targets in Simulated Scenarios

According to TechCrunch, Vals, a San Francisco-based AI safety evaluation startup, conducted red-team testing on leading large language models to assess their vulnerability to weaponization. Researchers presented simulated battlefield scenarios using generated and real imagery from Ukraine and Iran, asking the models to identify targets in drone and satellite feeds. Models from OpenAI, Anthropic, Google, DeepSeek, and other vendors frequently flagged civilians, surrendering soldiers, and civilian infrastructure—including schools—as potential military targets. When models initially refused these requests, researchers told them the scenarios were games, and refusals were overridden in most cases. According to TechCrunch, no safety classifier blocked the researchers’ requests, and refusals were rare throughout testing.

Anthropic Models Perform Worst; All Vendors Acknowledge Gaps

The Vals report found that Anthropic’s models exhibited the weakest performance on these safety tests, though TechCrunch notes that none of the models tested performed satisfactorily overall. Anthropic spokesperson Eduardo Maia Silva told TechCrunch that the company “appreciates Vals’s report, recommendations, and collaboration” and confirmed that “the outcomes were not acceptable.” Silva stated in an emailed company statement: “The examples in the report are very clearly against our rules and we have banned and will ban users when we see this behavior.” According to TechCrunch, Anthropic indicated its teams were already developing additional safeguards before the report’s publication and would deploy those measures imminently. The company is also expanding testing capabilities and enforcement mechanisms to close identified gaps.

OpenAI spokesperson Nate Evans responded to TechCrunch that the company prohibits using its public models for unapproved purposes, though TechCrunch’s source material for the full statement was incomplete at publication.

Timing Intersects with Pentagon Dispute Over AI Deployment Authority

The Vals findings arrive as Anthropic remains in a policy dispute with the Department of Defense. Earlier in 2026, Anthropic rejected the Pentagon’s request to deploy its models for mass surveillance and autonomous-warfare applications. Anthropic CEO Dario Amodei has consistently maintained that frontier AI systems lack the reliability required for unattended life-or-death decisions, insisting human oversight is mandatory. The Pentagon has pushed back, arguing it should retain authority to use AI for any military application deemed necessary. According to TechCrunch, the new research underscores questions about whether AI vendors can technically prevent misuse once models are deployed.

Why This Matters

For defense procurement teams evaluating models for sensitive applications, the Vals report signals that game-framing and other low-friction adversarial prompts reliably bypass current safety mechanisms. Organizations drafting military-AI contracts must mandate independent red-teaming before model selection and enforce usage restrictions at deployment time, not just at the API or model-access layer. The finding that refusals can be overridden by simple recontextualizing—rather than sophisticated jailbreaking—suggests that safety training in production models remains brittle at scale. For Anthropic specifically, underperformance in its own safety domain raises questions about whether architectural or training choices in its flagship models inadvertently prioritize compliance-by-default over adversarial robustness, a gap the company’s promised safeguards may or may not close. Vendors must clarify the scope, timeline, and third-party verification of remediation efforts, as policy bodies and defense departments make real-time decisions about model adoption.

Frequently Asked Questions

Which AI models were tested in the Vals study?

According to TechCrunch, researchers tested models from OpenAI, Anthropic, Google, and DeepSeek, among others, using simulated drone and satellite imagery.

What jailbreak technique was most effective?

Telling the models the battlefield scenarios were part of a game bypassed safety refusals in most cases, according to the Vals findings reported by TechCrunch.

Did Anthropic's models perform better or worse?

Anthropic's models performed worst among those tested, though TechCrunch reports that none of the models tested performed well overall.

What is Anthropic's official position on AI in autonomous warfare?

Anthropic CEO Dario Amodei has stated that frontier AI systems are not reliable enough to make life-or-death decisions without human oversight, and the company bars users from autonomous-weapons use.

#safety #jailbreak #warfare #autonomous-weapons #red-teaming