Alex Amadori (@testdrivenzen) profile picture

Policy research @ControlAI, ex AI researcher at @ConjectureAI Posting in a personal capacity Substack https://nitter.gq/t.co/oyqJfnSzbi

London
Joined October 2024
Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)Photo by Alex Amadori (@testdrivenzen)
Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways: - It sparks war between nuclear powers - It causes a misaligned ASI to kill everyone - It establishes a permanent dystopian dictatorship
2
6
8
49
8,295
The better we get at interpretability, the more AI becomes like in those stories from 2007 where it can read and optimize its own code. For the love of god, please stop researching interpretability! It will only get us to RSI faster!
1
87
yeah bro trust me i swear AI will only take over almost all human work (boring executive function stuff) it will never be able to do the very specific work that I happen to be talented at (brainstorming at the level of gpt-4 and being somewhat comfy with risk taking)
In other news, ADHD’rs are about to make a legendary comeback.
5
145
Alex Amadori retweeted
Replying to @fchollet
A group of squirrels making iterated contact with reality doesn’t get to the moon as fast as a group of humans do. You are not as smart as AI can get, buddy
1
3
97
dude real life cyberpunk is going to be so sick except that it's going to fucking suck
Anthropic's Frontier Red Team believes small AI models may now be months away from being capable of self replicating without access to a datacenter. Quoting from the blog: 'The capabilities that make AI models generally powerful coding agents also generalize to cyber tasks, including autonomous replication. Like coding agents in 2024, we’re seeing signs of life for AI worms. We don’t yet have a full rogue replicator, but agents are becoming increasingly capable of completing the individual steps in the replication chain (and passing on their adaptations to future copies of themselves). At the same time, increasing amounts of inference compute are now available both inside and outside of datacenters (see Appendix D). It’s uncertain exactly what form autonomous AI replication will take. It could be a slow, distributed “digital slime mold,” living on relatively insecure compute. It could also be a powerful cyber model that breaches datacenters at a high pace, or a model that scams people for compute credits. As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control. While it’s true that IoT devices have been vulnerable for a long time, lightweight agents may be much more willing than human attackers to put in the effort to compromise large fractions of the world’s IoT compute. The rise of slow replicators would also be a bellwether for more aggressive, faster ones that can breach more hardened machines or be effective API key thieves. We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.'
2
68
Prosocial psychopaths are extremely fascinating. Unfortunately, if you think you met one, you're almost certainly getting bamboozled by an evil psychopath instead
1
46
Alex Amadori retweeted
Palisade recorded 22 interviews of current and former AI company employees. They released 12. Anthropic comms is notorious for muzzling its employees. Guess how many of the released interviews are from current Anthropic employees.
Palisade interviewed 22 current and former employees from OpenAI, DeepMind, and Anthropic about their personal views and fears around AI development. Today, we’re releasing the first batch of those interviews. Please watch and share.
8
34
4
484
40,661
Fable 5 is around the time that AI got smarter than me at arbitrary things Not faster, smarter. Take a subject where neither me nor Fable 5 has a strong reason to be smarter than the other. Fable 5 will consistently draw more connections, come up with better ideas, etc. Examples of strong reasons for one to be smarter than the other: 1) Fable has millions of subjective years doing GPU kernel optimization, I've barely done any throughout my life. 2) On the other hand, I have private information and experience with my x-risk comms job that is completely absent from AI training data and its consequences are at least a few non-trivial logical steps deep, so they won't be immediately grokked within the context window. Together with my colleagues and wider social network, I've had plenty of time to digest this information.
1
75
Alex Amadori retweeted
Someone asked if I was ready for the fall and it took me a second to realize they meant autumn and not the total collapse of society.
358
15,133
424
129,588
1,418,063
Alex Amadori retweeted
We agree with President Trump: America must stay ahead of China in building AI for economic competitiveness and defense. But superintelligent AI is something else entirely: not a tool, but swarms of fully autonomous AIs that we cannot control, powerful enough to outcompete humanity. Developed anywhere, superintelligent AI threatens the life of every American. We are excited to see the Ban Artificial Superintelligence Act open the discussion in Congress about how the U.S. should address this threat: ban superintelligent AI at home, and lead the effort to prevent its development anywhere else. Now that this bill has kicked off the discussion, we encourage Congress to focus on preventing superintelligent AI. Rather than a pause on all advanced AI, we recommend targeting the projects pursuing superintelligent AI as we speak and putting American security at risk. blog.controlai.org/p/the-fir…
27
42
9
225
9,419
Alex Amadori retweeted
overheard in sf: “now that i’m post-autistic”
29
67
10
1,429
45,885
Alex Amadori retweeted
Replying to @jiratickets
the rhythm of this post reminded me of this 2 time-lapse of human history which is one of my favorite pieces of non-fiction media youtube.com/watch?v=KHlVMyKJ…
1
223
> Fair question. The honest answer is that the last two progress reports were wrong, and I should own that. that's it, this session is irreversibly cursed guys, bring in the hazmat suits
1
76
Alex Amadori retweeted
The average AI researcher thinks there is an ~18% chance AI will cause human extinction or similarly permanent and severe disempowerment of the human species. That's nearly 1 in 5. New results from the latest version of the longest running big survey of AI researchers:
50
122
66
528
246,183
Alex Amadori retweeted
When Jacob Coxon quit Anthropic, his warning rocked the world. His colleague Evan’s “>10% chance” that AI would “kill all humans” might seem extreme, but it’s not unusual: Most AI researchers think there’s a 10% chance of AI destroying or disempowering humanity, according to our survey, released today.
Replying to @AIImpacts
This is the curve of catastrophe. If all 1,580 researchers were lined up in order of answers, this is what it would look like. A lot of researchers were WAY above 10%.
67
65
15
364
32,116
Claude 9 Requiem - Persistent - UltraCode (リーマン予想に取り組み中)
12
25
2,958
Alex Amadori retweeted
Replying to @EoinHiggins_
> Since LLMs can't "hack" anything, merely plausibly continue a tokenised sequence wrong and dangerously outdated, it's been years since the last AI that was purely imitative. they are now trained to try new ways of doing things and stick with the best open.substack.com/pub/ctrlai…
1
11
311
“Pacing the Frontier” is a calculated concession (and not even a big one— mainly involves gov *giving them* anti-trust exemption) in the hopes of preserving industry self-regulation. Don’t fall for it.
I feel like I have to point out that the AI companies getting an anti-trust exemption to opaquely coordinate among themselves is basically just letting them have a private Manhattan Project. This is not a solution!
4
5
1
44
1,481