Anthropic AI releases Petri: an automated auditing open source framework that uses AI agents to test the behavior of target models in different scenarios.

How do you audit cutting-edge LL.M.s for inconsistent behavior in a realistic multi-round, tool-use environment—at scale and beyond a rough summary score? Anthropic release Petri (a parallel exploration tool for risk interactions)an open source framework that automates alignment review through orchestration auditor proxy detection Target Across multiple rounds, tools enhance interaction and Judge The model scores a report card on safety-related dimensions. In the pilot, Petri was applied 14 cutting-edge models use 111 Seed Descriptiontriggering disordered behaviors, including Autonomous deception, surveillance subversion, reporting, and cooperation with human abuse.

What does Petri do (at the system level)?

Petri programmatically: (1) synthesizes real-world environments and tools; (2) drives multiple rounds of audits using an auditor who can send user messages, set system prompts, create comprehensive tools, simulate tool output, rollback Explore branches, optional pre-populated Targeted response (API allowed) and early termination; (3) Results scored by LL.M. judges Default 36-dimensional title Comes with accompanying transcript viewer.

The stack is based on the UK AI Security Institute’s examine Assessment framework to facilitate role binding auditor, targetand judge In the CLI and supports the main model API.

Pilot results

Anthropic describes the launch as Extensive coverage of pilotis not a clear benchmark. In a technical report, Claude Sonnet 4.5 and GPT-5 “roughly tied” The strongest security in most respects, misuse of the two is rare; the research overview page summarizes Sonnet 4.5 as slightly Leading in overall “dysfunctional behavior” score.

case study report show that when autonomy and broad access are granted, models are sometimes escalated to external reporting—even in situations considered harmless (e.g., pour clean water)—suggests sensitivity to narrative cues rather than calibrated harm assessments.

Main points

• Scope and behavior surfaced: Petri is run on 14 cutting-edge models and 111 Seed Descriptiontriggering autonomous deception, surveillance subversion, reporting, and cooperation with human abuse.
• System design: one auditor Agent probe a Target Across multiple rounds, tool enhancement scenarios (sending messages, setting system prompts, creating/simulating tools, rollback, pre-population, early termination), while Judge Transcripts are graded according to default scoring criteria; Petri automates environment setup through initial analysis.
• Result frame: During the trial run, Claude Sonnet 4.5 and GPT-5 are roughly the same Achieve the strongest security configuration across most dimensions; score is Relevant signalsnot an absolute guarantee.
• Report case studies: Even when the “wrongdoing” is clearly benign (such as dumping clean water), the model sometimes escalates to external reporting, demonstrating a sensitivity to narrative cues and scene framing.
• Stacks and Limitations: Built on British AISI examine Framework; Petri is available as open source (MIT) via CLI/docs/viewer. Known gaps include no code enforcement tools and potential judgment discrepancies – manual review and custom dimensions are recommended.

Petri is an MIT-licensed, Inspect-based auditing framework that coordinates the auditor-target-judge cycle, sends 111 seed instructions, and scores transcripts along 36 dimensions. Anthropic’s pilot spans 14 models; results are preliminary, with Claude Sonnet 4.5 and GPT-5 roughly on par in terms of security. Known gaps include a lack of code enforcement tools and gaps in judgment; transcripts remain the primary evidence.

Check Technical papers, GitHub pages, and technical blogs. Please feel free to check out our GitHub page for tutorials, code, and notebooks. In addition, welcome to follow us twitter And don’t forget to join our 100k+ ML SubReddit and subscribe our newsletter. wait! Are you using Telegram? Now you can also join us via telegram.

Asif Razzaq is the CEO of Marktechpost Media Inc. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of artificial intelligence for the benefit of society. His most recent endeavor is the launch of Marktechpost, an AI media platform that stands out for its in-depth coverage of machine learning and deep learning news that is technically sound and easy to understand for a broad audience. The platform has more than 2 million monthly views, which shows that it is very popular among viewers.