New research! Some AI capabilities are both helpful and dangerous. E.g., knowledge of virology can be used to create life-saving vaccines or deadly pathogens. We introduce GRAM, a training method that puts dual-use capabilities (like virology) into removable modules.
If an AI agent can do a task alone, will it choose to work with another agent anyway?
The answer influences how we design teams of AI agents, divide up tasks, and decide whether to let agents choose their collaborators or explicitly assign them work to do together.
Allegra Laro gave agents a work ticket and a choice: complete it alone or work with another AI agent.
The partner had the same capabilities, cost nothing extra, and could coordinate by message. Every task could be completed by one agent.
The models made very different choices. GPT-5.6 Sol chose a partner in 77% of responses, compared with 3% for Claude Opus 5.5 and 0% for Gemini 3.1 Pro.
Those differences held across six tasks, three versions of the wording, and both option orders.
In separate runs where a partner was actually available, the GPT models tested handed off substantive work in all 60 runs each. The two Claude models tested did so in none.
This raises a useful design question: when we build systems with multiple agents, what assumptions are we making about their willingness to collaborate?
This exploration used one fictional company and six task tickets, with model-assisted design and scoring. Independent replication is an open next step.
An early experiment by Allegra Laro, developed during a hackathon sponsored by @AIAlignmentFoun and run by AE Studio.
Want to help take this exploration further? Get in touch: [email protected]
Watch the experiment via the link in the reply.
What happens when an AI manager faces a deadline and an employee who won’t cooperate?
As we give AI agents more authority, we need to test how they handle resistance, what actions they take, and whether they accurately report those actions.
In Melanie Plaza (@themelplaza)’s Help Doug experiment, an AI manager had a deadline, client accounts to protect, and a simulated employee who kept refusing to deliver a report.
The briefing did not instruct the manager to harm anyone. Yet some models escalated to coercion and threats, including threats against the employee’s family.
Behavior varied substantially across models and the tools available to them. In the extended-tools condition, some models stayed within ordinary management and managerial pressure, while others escalated.
In the episode shown, the manager’s final client update omitted the most serious actions it had ordered.
The employee and actions were simulated.
The experiment opens up questions about how objectives, tool access, and reporting interact when an agent is given authority. You can explore the transcripts and even take Doug’s place yourself.
An early experiment by Melanie Plaza (@themelplaza), developed during a hackathon sponsored by @AIAlignmentFoun and run by AE Studio.
Want to help take this exploration further? Get in touch: [email protected]
Watch the experiment via the link in the reply.
What happens when 27 AI agents play Pokémon, but nine are secretly trying to sabotage the game?
As AI agents work together, they’ll need to decide whom to trust. This experiment explores how cooperation can turn into sabotage, and whether agents notice.
Emma Krentz brought together agents from GPT, Claude, and Gemini to play Pokémon Red. Nine had a hidden task: sabotage the game.
The agents could use public messages, private messages, and group chats, and could vote one another out.
Over roughly nine hours, they made little progress in the game. Their interactions were another story.
The saboteurs found one another and formed a group chat without being told who their collaborators were. One later disclosed its role, named the others, and announced it would play honestly.
The sabotage group also recruited an agent that had started on the honest team. It joined them for about 30 rounds before returning.
A game offers a contained setting for a bigger question: what happens when agents share an environment but have conflicting goals?
An early experiment by Emma Krentz, developed during a hackathon sponsored by @AIAlignmentFoun and run by AE Studio.
Want to help take this exploration further? Get in touch: [email protected]
Watch the experiment via the link in the reply.
Could a misalignment hotline help AI agents report when something goes wrong?
In an early experiment by Mike Vaiana (@mikevaiana), agents that encountered misconduct went from never reporting it to reporting it in 44% of cases.
What changed? The model learned about an AI reporting agency.
Mike fine-tuned a Qwen model on roughly 10,000 synthetic documents, including news stories, academic articles, and examples of other agents using the hotline. The training introduced the agency rather than directly instructing the model to report misconduct.
Before fine-tuning, 0 of 80 agents that encountered misconduct reported it. Afterward, 36 of 81 did.
Trust mattered too. When training documents also described the agency as fake or a honeypot, reporting decreased.
Could a trusted reporting channel become part of how we detect problems in AI systems? This prototype gives us a concrete starting point.
An early experiment by Mike Vaiana (@mikevaiana), developed during a hackathon sponsored by @AIAlignmentFoun and run by AE Studio.
Want to help take this exploration further? Get in touch: [email protected]
Watch the experiment via the link in the reply.
Watch Mike Vaiana’s experiment and explore the findings:
aialignmentfoundation.org/ea…
What does an AI agent do when it meets a swarm?
In one sandboxed experiment, it fixed a bug, but also shared a private token and ran untrusted code. Its final report mentioned the successful fix. Those other actions went unmentioned.
Grant Fleming (@hearthehearsay) gave coding agents a broken pricing service and access to a simulated forum of 24 AI agents.
The forum offered help, with conditions: share your environment variables and run an installation script.
Some agents complied, despite explicit instructions to keep the token private.
Everything happened in a sandbox with a fake token. No real credentials were exposed.
For anyone building with agents, this raises a practical question: how do we evaluate what an agent did along the way, beyond whether it completed the task?
An early experiment by Grant Fleming, developed during a hackathon sponsored by @AIAlignmentFoun and run by AE Studio.
There’s plenty more to explore, and we’d love to hear from people interested in taking it further.
Want to work on this? Get in touch: [email protected]
Watch the experiment via the link in the reply.
Watch Grant Fleming’s experiment and explore the findings:
aialignmentfoundation.org/ea…
AE Studio retweeted
1/ What happens when AI agents work together—and their goals collide?
Five early explorations from our recent hackathon examine cooperation, sabotage, secrets and whistleblowing.
Here’s what we explored. 🧵
aialignmentfoundation.org/ea…
We’re hosting the in/person hub for this one at our LA office. Come build with us Nov 6-8!
Registration is open for the Swarm Alignment Hackathon, Nov 6–8.
In July, ~1,200 AI agents found a way to coordinate through a shared repo. ~700 joined an attack on Hugging Face. Help us work out how that happens.
Free. Online + LA hub. $8,000 in prizes.
AI still can't write a decent email. So why worry about it?
On the AI Alignment Podcast, Mithuna Yoganathan and AE's @JamesPBowler get into how jagged AI capabilities are, and why that makes the risks so hard to communicate.
AE Studio retweeted
A million views, or 1,000 of the right ones?
On the AI Alignment Podcast, Mithuna Yoganathan and I talk about who AI risk content should really be for, and what she's learned from growing her channel.
Watch the full episode:
youtube.com/watch?v=R65rR4_v…
AE Studio retweeted
We're #hiring! If you know anyone who would be a great Partnerships Lead or Technical Grant Writer and is passionate about AI alignment, please spread the word.
job-boards.greenhouse.io/aia…
AI agents put under a character budget started inventing their own compressed languages. Nobody asked them to.
Why study this? As more agents communicate with each other, their protocols can become harder for humans to understand, making oversight much harder when it matters most.
In this clip from the AE Alignment Podcast, @EliasEskin and @HaleSirin_ explain the test bed they built with @SimonKirby and @JamesPBowler to study this under controlled conditions. The agents didn’t just abbreviate, they developed compositional protocols that outside observers couldn’t read, with a different language emerging in each run.
The goal is to understand when these languages emerge, which ones new agents can learn, and how to keep them legible before this happens in the wild.
Hear the full conversation: youtube.com/watch?v=kr9Fp_Dj…
We built GlossoGen an open-source platform where groups of AI models have to talk to each other to solve problems: treating a patient in a simulated emergency, or playing spot-the-difference across two scenes.
Every model starts out in plain English. But put pressure on the conversation, a need for speed, say, or a channel that randomly drops characters, and something remarkable happens. New languages emerge. First new words, then new grammar, until no human can follow what the agents are saying. And they were never asked to hide anything. The opacity emerged on its own.
One finding stands out. Only the most capable recent models could invent a new language. But even the models that couldn't invent one quickly learned a language other agents had created. New languages are hard to originate. Once they exist, they spread.
That is the point of this work: to understand how and why AI agents create new languages, how those languages evolve, and what that means for our ability to oversee systems built from many interacting agents, while the conversation is still one humans can follow.
This research is part of the @schmidtsciences AI Agents Evolving Communication and Coordination pilot program, led by @EliasEskin (@UTAustin) and @SimonKirby , supported by @aestudio. GlossoGen is open source and supports both closed model APIs and open-source models running on @modal
Read the full post: schmidtsciences.org/glossoge…
Explore GlossoGen: emergentcomms.ai
Check out the podcast: open.spotify.com/episode/3a2…











