Aiden (@co0ver) profile picture

ai safety. prev @cais

berkeley
Joined October 2025
Photo by Aiden (@co0ver)
There is a common belief in the AI field—that advanced AI means the inevitable forfeiture of human primacy in the world. Even many who disfavor that future believe it is likely. I call this the silent consensus, and it must not be allowed to remain silent any longer.
140
223
152
1,900
398,964
たまに描くセイウチの絵。本当に人気がなくて、伸びない。これが最後のつもりで10枚投稿します。
150
1,616
74
20,550
291,707
Aiden retweeted
people who pursue AGI without keeping transhumanism as the Actual north star—the reason you’re doing this at all—feels so obviously wrong that i struggle to even call it evil, just an absurd degree of misunderstanding. like filling more mercury into a thermometer if you’re cold
28
33
9
463
22,412
Aiden retweeted
Obama shocks Bay Area by speaking perfect LessWrongese
Barack Obama: AI is reaching the point where it no longer needs humans to teach it. The era of exponential intelligence growth is upon us. “The machines are starting to be able to teach themselves. They don't have to wait for humans to start figuring things out.” “We're now at a stage where it's 50-50, and if the trend lines continue, the expectation is it might be 90-10.” “We're now hitting the hockey stick trajectory where the models are going to get smarter and smarter at an exponential pace.” “If they start setting their own agendas, you may get a misalignment between what they want to do and what we want them to do.” “And that gap can be dangerous.”
17
149
7
2,693
100,255
live forever or die trying
2
15
134
5,051
For all but a vanishingly brief instant near the dawn of history, the word ‘ship’ will mean ‘spaceship.’ - Arthur C. Clarke
148
1,113
156
12,180
3,350,302
I'm glad some posts on LessWrong are written at a level I can understand.
2
19
2
497
7,397
We mourn the loss of Senior Lecturer Richard Allen Posner, a longtime #UChicagoLaw scholar, former federal appellate judge, and pioneer in the field of law and economics, who died on November 10. Read more about his legacy: buff.ly/e3c3WdL
98
1,048
394
3,531
640,859
Aiden retweeted
Replying to @GWHayduke97
Is human connection, love, community not meaning beyond consumption and vegation? Do you really detest the people in your life that much that you would hate to spend the rest of time with them? Are you not able to derive meaning from work outside of competition and a feeling of superiority? Obviously, I find this insanely distasteful and disgusting, an attitude that could only ever be cooked up in the depths of silicon valley.
3
2
1
17
2,708
I think Buck is too optimistic about the situation. Given ASI by the end of 2027, I think it's *very likely* that your life is ended/ruined, greater than Buck's implied ~2/3 chance. I too feel "frustrated and sorry" that we didn't do better. But there is time and we fight on.
I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years). I’ll say a bit more about each of those claims. It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: blog.aifutures.org/p/q35-tim… I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see nitter.gq/dwarkesh_sp/status/209… and youtube.com/watch?v=-Ef0ocEo… If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. blog.redwoodresearch.org/p/n…) Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.) I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon. Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything. I don’t know what you should do. I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon. (It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.) I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.
83
34
26
452
191,098
I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years). I’ll say a bit more about each of those claims. It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: blog.aifutures.org/p/q35-tim… I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see nitter.gq/dwarkesh_sp/status/209… and youtube.com/watch?v=-Ef0ocEo… If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. blog.redwoodresearch.org/p/n…) Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.) I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon. Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything. I don’t know what you should do. I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon. (It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.) I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more-or-less in the dark about the scope of the conspiracy. I’ve spent the last three days reading through these reports and trying to understand exactly what happened. Here is my attempt to tell the whole story in plain English: dwarkesh.com/p/openai-huggin…
444
445
325
3,257
1,650,498
Aiden retweeted
yeah I'm TESCREAL: Transgenderism Egalitarianism Socialism Collectivism Rationalism (the french kind) Effective Altruism Longtermism
6
15
1
169
4,231
This is atrocious
Democrats have a wishlist for AI that includes a federal permit to train a model, a Department of AI, 20-year prison terms for executives, kill switches, and ID checks to use a chatbot. Writing code is about to need a hall pass if certain politicians get their way. reclaimthenet.org/democrats-…
124
111
12
1,424
112,014
Exclusively reasoning about how to be impactful from the local basin of "what looks the most tractable on the margin" is how we get to every junior AI safety person doing scheming evals and nothing else
“The problem with young AI safety researchers today is that the only thing they study is scheming evals and mech interp. When we were young, we studied Vingean reflection, we studied ontological crises, we read AGI Ruin. We swallowed everything. And then we went and tried to solve the alignment problem for superintelligence. But if you grow up and all you study is Claude LLMs, you all converge on new ‘scheming’ evals and regurgitate the same local minima research.”
3
10
398
These are the stakes: To make a world in which all of God's children can live, or to go into the dark.
AI companies admit what they’re building might kill everyone. We don't have to agree to this. Add your name at donotaccept.ai to say: we do not accept your terms.
4
106
2,196
“The problem with young AI safety researchers today is that the only thing they study is scheming evals and mech interp. When we were young, we studied Vingean reflection, we studied ontological crises, we read AGI Ruin. We swallowed everything. And then we went and tried to solve the alignment problem for superintelligence. But if you grow up and all you study is Claude LLMs, you all converge on new ‘scheming’ evals and regurgitate the same local minima research.”
Hayao Miyazaki on modern anime, via Roland Kelts the author of 'Japanamerica' "Hayao Miyazaki said to me at one point: 'The problem with young anime artists today is that the only thing they study is anime. When we were young, we studied French literature, we studied impressionism, we read Dostoevsky. We swallowed everything. And then we went and created original anime. But if you grow up and all you watch is anime, you're just regurgitating spinning wheels.'" He also spoke about his work on the upcoming documentary 'Tezuka! God of Manga'
9
19
2
221
6,985
In bombshell new reporting from @nytimes’ @AnaSwanson and @DevlinBarrett, it appears Nvidia executive leadership personally reached out to the Commerce Department in an effort to get it to drop its investigation into Megaspeed—a company we now know was run by Mr. Huang’s confidential romantic interest. Any independent investigation into Mr. Huang’s personal affair must get to the bottom of whether Nvidia’s efforts to protect Megaspeed were ultimately influenced by that relationship. I am not an investor, but at first look, this would appear to be a gross violation of Mr. Huang’s fiduciary responsibility as CEO—and a textbook example of (FAILED!) attempted political corruption by one of America’s largest tech companies. Remember, per New York Times reporting, the Justice Department has been conducting a criminal inquiry into Megaspeed—a criminal inquiry that concerns Megaspeed’s role in illegally shipping Nvidia’s products to customers in China. If Nvidia thought it could persuade the Trump administration to drop its investigation into Mr. Huang’s girlfriend, then that is a total betrayal of President Trump’s trust. It puts the administration in an extremely uncomfortable position, and reflects poorly on Nvidia’s executive leadership. nytimes.com/2026/11/09/busin…
25
160
19
496
52,836
Aiden retweeted
⚠️declaring accelerationist amnesty⚠️ if recent contact with reality is causing you to feel some kernels of worry about this whole ai safety thing, *you are allowed to change your mind*. you don't even have to change it all the way, you don't have to suddenly change your twitter bio or start protesting against nuclear power plants, you don't need to become an EA or suddenly think yudkowsky was always right about everything. you're allowed to just notice that shit seems to be getting real in some pretty weird ways and update your beliefs. at least personally, if i see someone saying "damn, i guess i was wrong or at least overconfident about X" i'm not gonna take the opportunity to dunk or i told you so. i'm sure others will, this is the fucking internet. but at least personally, i'm just gonna be happy that you're paying attention. there were lots and lots of good reasons to *not* take this situation seriously. there were lots of well verbalized reasons why rushing ahead was potentially a huge benefit for humanity. hell there were maybe even valid reasons why there was little to be done until we'd already gotten to nearly exactly this point. that's all fine man. all that matters right now is that we as a civilization realize what we're on the verge of, and make it through this carefully. it's gonna take a huge effort from all of us.
85
91
49
1,373
130,775
sometimes u talk to rich and powerful people. hear what's on their mind. then u realise it's all just high school drama/popularity contests, until u die. i shall go for a walk.
7
15
1
358
6,545