
Founder of CASIR, Advisor at @ControlAI.
Joined October 2019
- Tweets903
- Following100
- Followers1.7K
- Likes1.3K
Gabe retweeted
I signed this. Anthropic is simultaneously the company that most loudly presents itself as the safety lab, and the company that is most directly shooting for superintelligence before they or our broader civilization are remotely ready for it. They should stop.
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: misanthropicstatement.com/
Gabe retweeted
Update: I'm no longer advising AI companies. I think they're too constrained to make the situation much better, so I'll stay independent until there is a meaningful slowdown or moratorium on this reckless race.
Gabe retweeted
With just a few emails, a little prep and one meeting, I convinced my MP to sign controlai.org/statement 🥳
Gabe retweeted
I signed this statement because I believe that Anthropic is in many ways the AI company doing the most harm. In public, their leaders warn of recursive self-improvement (RSI) creating uncontrollable systems that could cause human extinction within a decade. But in private, they are pedal to the metal on RSI, driving the race and boasting to prospective investors that it is their "competitive advantage". Ahead of today's IPO, prospective investors should consider whether the ROI is worth risking the lives of their loved ones.
Here are some ways in which Anthropic IMHO does more harm than competitors:
1) They’re driving and arguably leading the race to RSI.
2) With inflammatory rhetoric, they’ve done more than any other US company to pour gasoline on the geopolitical AI race, which harms US national security.
3) WaPo reporting suggests that they’re implicated in more civilian deaths than any other AI company (more than 100 little girls in a school selected for targeting by Claude as part of their Palantir collaboration). When Google left Project Maven after employee pressure, Anthropic chose to join.
4) They're the company leaning hardest into transhumanism and the idea that we should build conscious beings with rights, not controllable tools.
5) They and closely associated donors have done more than any other AI company to co-opt and neuter much of the AI safety ecosystem to back their stance: “RSI is very dangerous but let’s race to it anyway”. People have told me that they support today’s statement but are afraid of losing funding or other opportunities if they sign.
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: misanthropicstatement.com/
Gabe retweeted
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: misanthropicstatement.com/
Gabe retweeted
ControlAI’s @Gabe_cc and @epist_vigilance have written a piece about the destabilizing shocks caused by new technologies, and how they require strong institutions to be absorbed and channeled.
This is a great foundational model to understand the effects of AI acceleration on society, and why ControlAI is advocating for preventing the development of Runaway Machine Intelligence.
cognition.cafe/cp/224772769
Gabe retweeted
Reading this makes me feel like the AI Safety community has been played like a fiddle.
Back in 2023 it was too soon to stop AI, so we had to be satisfied with self-regulation.
Now, in 2026, it's too late to stop AI, so we have to be satisfied with self-regulation.
Ridiculous.
Replying to @tszzl
To "automate safety research while verifying the results", we need:
1) Evals that transfer to the real-world
2) Good specs
It's like asking Astra to do formal verification of safety properties over a physical system.
1) If you don't know enough to model the physical system, you can't verify that the verifiable proofs Astra is producing translate to the real world
2) If you don't know enough about what makes the physical system safe, you can't verify that the theorems that Astra is proving are good
If you lack either 1 or 2, you are in fact handing it off without oversight.
(To some extent, we already have this problem with formal verification over normal programs!)
We have the same problem with alignment.
1) We don't have evals of AIs that translate to them dealing with humans at scale
2) We don't have good specs for what "alignment" is
So, right now, if you ask AIs to do alignment, you are again handing it off without oversight.
And finally, one must understand that it is a recursive problem.
Ie: we have the same problem with automating safety research.
1) We don't have evals of AIs that translate to them being good at safety research
2) We don't have good specs for what "being good at safety research is"
So right now, if you ask AIs to do automate safety research so that later AIs do alignment, you are handing it off without oversight.
Gabe retweeted
Replying to @CyberPhilos
I’m not sure where people get the idea that Anthropic is the AI company that constantly talks publicly about human extinction. They actually talk about that very rarely, and downplay it when they do.
For example, the words “human extinction” have appeared in an official OpenAI statement before, but they have never appeared in any official Anthropic statement ever.
Gabe retweeted
people often post this stuff to imply that moral panics about the technologies of the past were quaint. But the past is full of reminders that we are on a long, slow march away from embodied experience, and that we’ve lost more of it than we can even remember
Gabe retweeted
I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years).
I’ll say a bit more about each of those claims.
It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: blog.aifutures.org/p/q35-tim…
I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see nitter.gq/dwarkesh_sp/status/209… and youtube.com/watch?v=-Ef0ocEo…
If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. blog.redwoodresearch.org/p/n…)
Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.)
I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon.
Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything.
I don’t know what you should do.
I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon.
(It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.)
I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.
This culminated in the third one taking over part of OpenAI itself.
All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened.
Here is my attempt to tell the whole story in plain English:
dwarkesh.com/p/openai-huggin…
Gabe retweeted
Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. blog.aifutures.org/p/q35-tim…
Gabe retweeted
and look how it’s gone for them:
- 95% of nonhuman mammals are livestock
- insect biomass reduces by 2% every year
- farmed animals live in crowded cages worse than any prison
- wild animals have their hunting, migration, and habitats disrupted by human light, sound, and trash pollution
outside of house cats and dogs, the industrial revolution has been disastrous for animals. even pets may live “happy” lives but are neutered, separated from their families, housed indoors, and fed slop.
this is despite humans generally being “aligned” with animal wellbeing.
Gabe retweeted
Spoke to an Anthropic employee today. They have a near-100% p(doom), but they think the only chance of survival is that they're somehow completely wrong about alignment and it turns out to be solvable, and the only way to get there is by accelerating capabilities as fast as possible. They were much more confident that humanity would never have the political will to do what needs to be done to stop ASI development than that alignment would be solved, even though they think alignment will almost certainly not be solved
They have not called their elected representatives to inform them that this is what they believe
Gabe retweeted
This report and reactions from people I trust (like Jake) are making me physically ill. As far as I can tell, Anthropic's central plan is to continue this acceleration of AI capabilities until we're getting decades of AI capabilities growth in a year. Given our very limited ability to understand or control AI systems, this would be extremely reckless and completely unacceptable, and it should be the single top priority of civil society and the US government to stop them and other companies from doing this.
Gabe retweeted
New interview out with @Gabe_cc, following up on emergent wisdom in LLMs but then quickly veering into broader questions of virtue and politics. Appreciated the spirited defense of democratic republicanism, cc @__drewface @danielgolliher. Link to full interview in thread.
Gabe retweeted
This is primarily a problem with the EA-affiliated side of AI safety. Unfortunately, that’s most of the field by now.
EAs don’t have memetic defenses against conflating “do the most good” with “gain the most power” (or sometimes just “be in the room where it happens”).




























