Jordan Taylor

Mythos 5.2 has higher precursor capabilities for evading a reasoning monitor than previous Anthropic models, including its ability to control the contents of its Chain-of-Thought reasoning:

Given the chance to iterate against monitor feedback, Mythos 5.2's evasion rate rises to 85%, compared to just 9% for Mythos 5.1

Action monitoring is still a strong defence against escape from less-vulnerable sandboxes.

Mythos 5.2 can break out of highly insecure simulated sandboxes without being caught around 30% of the time – against a production Opus-5 action monitor looking for sandbox escapes:

In our monitorability testing at @AISecurityInst, we found Mythos 5.2 is substantially more capable of evading a monitor which reviews its outputs than comparison models 🧵

CoT controllability is near saturated for many reasoning lengths. When prompted, the model is able to e.g. produce reasoning unrelated to the question or its answer.

"[...] the set of tasks Astra is able to accomplish without the use of CoT is greatly expanded compared to prior models. [...] See similar discussion in UK AISI evaluations, suggesting the no-CoT time horizon may have increased by about an order of magnitude."

The full report gives a lot of recommendations for model developers, but here are some recommendations I'm most excited about for general AI researchers:

White-box methods are also potentially promising, but are not maturing quickly enough to fill in for degradation elsewhere. And while action-only monitoring provides a floor for oversight, there are many settings where bad actions are hard to distinguish from benign ones.

Although promising, these training-based approaches are face challenges around generalisation: