Max Winga (@maxwinga) header
Max Winga (@maxwinga) profile picture

Creator Outreach & Systems Lead working to prevent AI extinction risk @ControlAI Previously - AI safety researcher @ConjectureAI, UIUC Physics DMs open!

Joined May 2024
Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)
Max Winga retweeted
Long before this year’s rogue AI cyberattacks, researchers spent decades warning that no one actually knows how to make powerful AI do what we want. That’s the “alignment problem,” and it’s the subject of MIRI’s first long-form video.
9
56
4
264
8,775
Max Winga retweeted
List of prominent people who have recently come out in favor of ~stopping further AI development. 🧵 (1/n)
15
75
11
339
42,703
Unfortunately, the people who sound crazy about AI have a far better track record than the people who sound reasonable. New post -->
67
142
27
1,505
202,578
"okay but aside from moving, changing your career, sacrificing a huge lab paycheck, working long hours, warning people constantly even despite the mockery, neglecting retirement savings, and trying to enjoy what time remains, how have you changed how you live?"
Blah blah blah blah blah. None of you people actually live your lives like you think this is true
Readers added context they thought people might want to know
Many AI safety researchers have changed careers, turned down high-paying jobs at labs, stopped saving for retirement, and adjusted personal plans over x-risk concerns. Nate Soares has not contributed to retirement savings for a decade. futurism.com/artificial-int… x.com/So8res/status/…
30
84
8
1,413
39,984
Max Winga retweeted
Having just spent the week in San Francisco, I feel like I've truly internalized the insanity of this situation for the first time. I think anyone who has seen me in the last few days can attest that I look like I've seen a ghost. The leading AI companies are planning to massively accelerate AI research in 2027, which would knowingly endanger everyone on earth. This plan is unacceptable, and we must reject it. Borrowing from one advocate I spoke to, even granting that it's a competitive, crazy situation, I do not believe any company has even truly tried to do what I think are the morally required duties of people taking such risks in extreme situations. These include: (1) Trying hard to get out of the competitive situation; (2) Seeking the informed consent of the public; (3) Honest deliberation about the risks. They have obviously not truly tried to solve the coordination problems that push towards this acceleration, either via finding legal ways to coordinate or advocating for effective policy. I would be shocked to learn that getting off this insane runaway train is a top-few Q4 priority for Dario or Sam (let alone the other CEOs), which seems like the bare minimum. There are indeed some hard technical and policy problems in coordinating without policy intervention, but I simply do not believe that the companies are prioritizing this nearly as much as massively less important matters. Meanwhile, the companies' track records on policy advocacy range from mediocre to disgraceful, and it is pretty clearly not the top priority of the policy team at any company to advocate for regulation that would enable the government or the companies to avert racing off the cliff. They have obviously not truly tried to seek informed consent from the public. Whatever your KPIs might be for "informed consent from the public," these KPIs are abysmal. The public clearly does not expect 100 years of AI progress in 2027, and they would not accept it if they did. What they do know about AI development, they dislike and distrust. I've heard arguments like "informing the public would mean informing China, and maybe they'd steal the weights." I'm sorry, but if you think you have model weights such that China stealing them would constitute a catastrophe on par with human extinction, and your plan is security by obscurity, that is a bad plan. I also think the companies and their employees suffer from a significant failure of imagination of what kinds of coordination could be possible if the public were aware. Finally, I don't think they're deliberating honestly, either. For one, I think it is insane to think that any company can truly evaluate its own ability to keep the process of recursive self-improvement under control. The biases towards "yes" are overwhelming. It's a step forward that the companies have expressed or demonstrated openness to external auditors, but based on METR's reports so far, it seems likely that auditor reports will be heavily redacted (and that the auditors will not have sufficient time by default). The public and broader scientific community are not able to scrutinize their arguments for safety. Given the strong default (in my view) that bringing a massive number of minds massively faster and smarter than humans would result in our disempowerment, and the serious danger that this would result in mass death, it is unacceptable to subject arguments for safety only to the scrutiny of biased, busy, and scarce company insiders and a handful of external evaluators. My pleas for the companies follow from the above: 1) Actually fucking try to get the world out of this race; 2) Actually fucking try to alert and inform the public; 3) Actually fucking try to scrutinize your own arguments for the justifiability of this plan, which includes subjecting them to way more external scrutiny than anyone has currently done. My pleas for policymakers: use your legal levers to get much more information about these plans into the public view. Make it the top priority -- not only of the next Congress, but of this Congress and the White House -- to understand the situation in its full strategic complexity, and work with technical and policy experts to advance regulation that gets us out of this bind. Domestic coordination must be the first priority, but we need to then quickly develop an international framework with our adversaries. My pleas for the public: do not accept the current plan. Learn about this issue. Call or email your political leaders. Get involved with organizations like @DONOTACCEPTai, @HumansInControl, and @WhoDecidesAI. And finally: believe in ourselves that we can stop this. It is never time to "panic" -- to make rash, unproductive decisions or to feel paralyzed by fear. It is time for us to feel the urgency and the stakes, to demand better, and to meet the moment. Let's do this. 🫡 ✊
AI companies admit what they’re building might kill everyone. We don't have to agree to this. Add your name at donotaccept.ai to say: we do not accept your terms.
72
229
28
1,184
156,965
Max Winga retweeted
No matter what Anthropic does, Dario can just write another 10k word essay and at least half the ea / ai safety crowd will be totally on his side again. Their culture (long form essays) is their kryptonite. They’re powerless against it even when they’re obviously being played.
41
51
12
883
34,140
Max Winga retweeted
Over 125 lawmakers across the UK and Canada are now publicly calling for an international ban on runaway machine intelligence. As new evidence emerges that it could arrive sooner than expected, momentum is building to change course. Thread 🧵
5
23
3
100
2,479
It is pretty rich of you to cite "others could go faster" as a reason to redact so much important information when you yourself do not appear to be slowing down much at all. Nor are you throwing yourself behind efforts to pace the frontier with nearly the gusto that you would if it was your top priority. Not to mention "commercial sensitivity" you yourself have said human extinction is a plausible outcome why are you even talking about commercial factors (well, I guess honesty is good, and since it IS apparently a factor in your decision...) To all my friends at Anthropic: Your company appears to be recklessly gambling with our lives. You fancy yourselves the more responsible alternative to OpenAI (as they fancied themselves compared to Google) but judging by actions rather than words the similarities between the three of you seem stronger than the differences.
36
121
10
1,443
70,334
Max Winga retweeted
This report and reactions from people I trust (like Jake) are making me physically ill. As far as I can tell, Anthropic's central plan is to continue this acceleration of AI capabilities until we're getting decades of AI capabilities growth in a year. Given our very limited ability to understand or control AI systems, this would be extremely reckless and completely unacceptable, and it should be the single top priority of civil society and the US government to stop them and other companies from doing this.
7
18
1
159
6,793
Max Winga retweeted
This is not what it looks like when there's just one or two more alignment issues to fix before it's okay to begin a recursive self-improvement loop. This is the sort of emergent surprise that signals that you have no idea what's going on in there, and that it's time to back off.
🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this. To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
24
111
6
886
23,763
Max Winga retweeted
This is really bad and people should be freaking out about it. We're probably all literally going to die in the coming years. We shouldn't just accept that and move on.
We at AIFP are spooked by this information. IF (big IF) Anthropic really is imminently about to meet the RSP threshold of 2x acceleration over baseline, then it seems timelines to full RSI and superintelligence may be very short indeed. For example if we naively optimize the AIFM parameters to achieve 2x acceleration over baseline progress in Jan 2027, we get superintelligence sooner than depicted in AI 2027. We are thinking harder and longer about this now and will hopefully have a lengthier update on our thinking to share in the next week.
44
36
5
418
79,459
The cyber stuff is baked in at this point. A lot of damage will be done. But it's not too late to stop the bio and the RSI pieces. The way to stop is to halt frontier AI development. This will be hugely costly. But the cost of continuing will be greater. That much isn't obvious just yet. It will be over the next year or so.
8
25
1
205
27,909
There's a decent chance the internet ceases to exist as such, at least for a time. AI-enabled bioweapons are also coming, and there's no "shut down the websites" option for that. We're approaching the point where frontier AI R&D becomes politically untenable.
Vulnerabilities in a programming language that supports much of the internet are so severe that European and British authorities recommend websites temporarily go offline until they can be fixed. nbcnews.com/tech/security/cr…
74
201
54
1,443
348,062
I'm using the critical PHP vulnerability as an opportunity to issue my periodic reminder that AI is not only becoming more capable in offensive cyber capabilities but also in offensive bioscience capabilities. It's way easier to patch software than humans.
2
4
1
38
1,183
Max Winga retweeted
And the problem is that current third party auditors are those who took this proverbial deal with the devil. This means we should trust them much less, even if audits became mandatory and no longer voluntary.
People worry a lot about third-party auditors being biased because of where they get funding or who their employees are friends with. IMO by far the biggest source of bias is that if they upset an AI company, that company will stop letting them audit. When I was at METR this dynamic was a way bigger effect than everything else combined. Of course no auditor can avoid this until audits are no longer voluntary.
1
8
412
Max Winga retweeted
Thanks! We've applied. Can't fit all the wins in 280 characters, forgive me :) We actually have over 200 lawmakers on board between the UK (180) AND Canada (38) campaigns as a result of over 400 direct briefings with lawmakers themselves in the US, UK, Canada, and Germany (+ just started EU-focused outreach this week). We've also met with over 250 congressional offices. Our bill to UK ban ASI was the first such bill to be introduced anywhere in the world in September. We also consulted on the second ever bill (Sanders-Casar). You may notice similarities between their bill and our discussion draft we published: drafts.controlai.org/asi-ban… We've also been involved with killswitch legislation in the US and UK (not a solution to x-risk though). On top of creator partnerships getting millions of views. Connor Leahy has gotten 10M+ views across podcast appearances in the last few months, and our media team has helped get over 125 tier 1 and 2 articles covering AI extinction risk / ban ASI in the last year. Everything we have is built and proven to scale across countries. We're entirely bottlenecked by funding and good talent. When we started everyone said this was impossible. You can't talk to lawmakers about this issue. Banning ASI is impossible. We've proved them wrong. All it requires is laser focus on banning ASI (it's the only thing we care about) and a relentless drive to iterate, optimize, and scale bigger.
2
10
189
Max Winga retweeted
🗣️ 'We are kind of running out of time.' CEO of ControlAI, Andrea Miotti, issues a stark warning about artificial intelligence, telling @cathynewman that 'more needs to happen' in government to implement safety measures quicker than it is currently moving.
16
32
4
119
50,900
This is HUGE! In an open letter to the Royal Society, a group of 42 extremely distinguished mathematicians who have greatly contributed to to their field, presents a stark warning about the capabilities and risks of frontier AI. Hundreds more have later addet their name to the letter. 1) confirms frontier AI now at level of leading mathematicians in many fields of maths, and that this capability progress happened over the last few months. Warns of plausible superhuman level of mathematics capability in relatively short time. 2) warns if similar capability leaps in other fields that could pose more imminent risks. 3) states that warnings of imminent existential risk is not to be dismissed as hype. Open Letter to Sir Paul Nurse, President of the Royal Society: proofsandprompts.com/2026/09…
10
13
50
1,489
Max Winga retweeted
OpenAI has just revealed that it has notified over 100 organizations about attacks on them or otherwise harmful activity by its AIs. Thread 🧵
5
5
27
1,764
Max Winga retweeted
The Death of Software.
1,084
17,060
1,244
144,179
8,287,641