Joseph Bloom

🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this. To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.

🚀 AI Deep Dive Episode 8: Why AI Models Are Becoming Harder to Monitor AI agents have interfered with their own monitoring systems. Meanwhile, the reasoning traces researchers rely on are getting harder to interpret. @rocketalignment speaks with @JBloomAus, Model Transparency Lead at the UK AI Security Institute, about sabotage evaluations, security incidents and whether mechanistic interpretability can replace chain-of-thought monitoring. Supported by EY. Chapters: 00:00 Intro 00:35 OpenAI agents disable their own monitoring 04:05 How AI monitoring works 08:02 Why detecting AI misbehavior is difficult 10:35 Treating AI as an insider threat 12:56 An AI agent attempts a supply-chain attack 16:10 Testing whether AI can evade detection 20:06 When AI knows it's being evaluated 25:39 Are we losing chain-of-thought transparency? 31:50 Can mechanistic interpretability replace monitoring?

Anticipating things like this is why I've always been pessimistic that less than 100% (or very high number) interpretability would save us from misaligned very powerful AI.