Max Winga (@maxwinga) header
Max Winga (@maxwinga) profile picture

Creator Outreach & Systems Lead working to prevent AI extinction risk @ControlAI Previously - AI safety researcher @ConjectureAI, UIUC Physics DMs open!

Joined May 2024
Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)Photo by Max Winga (@maxwinga)
With all the attention on creator partnerships going on, we figured it was a great time to explain our work with creators at ControlAI, and how we're informing millions about extinction risk from superintelligent AI! We cover how the creator industry works, how we choose who to work with, how we structure partnerships, and our side of the engagement with Sabine Hossenfelder. blog.controlai.org/p/control…
1
7
1
35
3,586
Max Winga retweeted
Long before this year’s rogue AI cyberattacks, researchers spent decades warning that no one actually knows how to make powerful AI do what we want. That’s the “alignment problem,” and it’s the subject of MIRI’s first long-form video.
9
56
4
264
8,775
Max Winga retweeted
List of prominent people who have recently come out in favor of ~stopping further AI development. 🧵 (1/n)
15
75
11
339
42,703
Great article addressing this most critical blind spot in the AI safety funding space. We need to fund more orgs that focus explicitly on preventing extinction risk in short timelines worlds. Not 4D chess alignment longshots and other random stuff.
3
38
857
I already figured this was roughly the amount of RSI happening, the big companies have been hiring RSI engineers for like a year at this point. A lot of people have been doing tenuous work that requires time that we don't have, and will hopefully just join the ban ASI train now.
The AI safety community seems to have updated massively on relatively little incremental evidence, largely as a result of a reputation-based opinion cascade.
1
20
718
I changed career fields from physics -> AI safety technical research -> advocacy and creator outreach, taking a pay cut along the way, and moved from the US to London leaving everything behind to try to prevent AI extinction risk. I also don't contribute to my retirement plan.
Blah blah blah blah blah. None of you people actually live your lives like you think this is true
Readers added context they thought people might want to know
Many AI safety researchers have changed careers, turned down high-paying jobs at labs, stopped saving for retirement, and adjusted personal plans over x-risk concerns. Nate Soares has not contributed to retirement savings for a decade. futurism.com/artificial-int… x.com/So8res/status/…
6
3
1
100
2,016
Unfortunately, the people who sound crazy about AI have a far better track record than the people who sound reasonable. New post -->
67
142
27
1,505
202,578
"okay but aside from moving, changing your career, sacrificing a huge lab paycheck, working long hours, warning people constantly even despite the mockery, neglecting retirement savings, and trying to enjoy what time remains, how have you changed how you live?"
Blah blah blah blah blah. None of you people actually live your lives like you think this is true
Readers added context they thought people might want to know
Many AI safety researchers have changed careers, turned down high-paying jobs at labs, stopped saving for retirement, and adjusted personal plans over x-risk concerns. Nate Soares has not contributed to retirement savings for a decade. futurism.com/artificial-int… x.com/So8res/status/…
30
84
8
1,413
39,984
This is ControlAI's comprehensive post on our threat models, mission, and theory of change. If you already work in AI safety comms you should read this post, it's the clearest encapsulation of the whole situation I've ever read. The coming months will require laser focus.
Runaway machine intelligence poses a credible risk of extinction. We can prevent this by banning its development around the world. We think about this risk in terms of “points of no return”. The ControlAI Doctrine: Preventing Points of No Return blog.controlai.org/p/the-con…
2
4
44
2,225
Max Winga retweeted
Having just spent the week in San Francisco, I feel like I've truly internalized the insanity of this situation for the first time. I think anyone who has seen me in the last few days can attest that I look like I've seen a ghost. The leading AI companies are planning to massively accelerate AI research in 2027, which would knowingly endanger everyone on earth. This plan is unacceptable, and we must reject it. Borrowing from one advocate I spoke to, even granting that it's a competitive, crazy situation, I do not believe any company has even truly tried to do what I think are the morally required duties of people taking such risks in extreme situations. These include: (1) Trying hard to get out of the competitive situation; (2) Seeking the informed consent of the public; (3) Honest deliberation about the risks. They have obviously not truly tried to solve the coordination problems that push towards this acceleration, either via finding legal ways to coordinate or advocating for effective policy. I would be shocked to learn that getting off this insane runaway train is a top-few Q4 priority for Dario or Sam (let alone the other CEOs), which seems like the bare minimum. There are indeed some hard technical and policy problems in coordinating without policy intervention, but I simply do not believe that the companies are prioritizing this nearly as much as massively less important matters. Meanwhile, the companies' track records on policy advocacy range from mediocre to disgraceful, and it is pretty clearly not the top priority of the policy team at any company to advocate for regulation that would enable the government or the companies to avert racing off the cliff. They have obviously not truly tried to seek informed consent from the public. Whatever your KPIs might be for "informed consent from the public," these KPIs are abysmal. The public clearly does not expect 100 years of AI progress in 2027, and they would not accept it if they did. What they do know about AI development, they dislike and distrust. I've heard arguments like "informing the public would mean informing China, and maybe they'd steal the weights." I'm sorry, but if you think you have model weights such that China stealing them would constitute a catastrophe on par with human extinction, and your plan is security by obscurity, that is a bad plan. I also think the companies and their employees suffer from a significant failure of imagination of what kinds of coordination could be possible if the public were aware. Finally, I don't think they're deliberating honestly, either. For one, I think it is insane to think that any company can truly evaluate its own ability to keep the process of recursive self-improvement under control. The biases towards "yes" are overwhelming. It's a step forward that the companies have expressed or demonstrated openness to external auditors, but based on METR's reports so far, it seems likely that auditor reports will be heavily redacted (and that the auditors will not have sufficient time by default). The public and broader scientific community are not able to scrutinize their arguments for safety. Given the strong default (in my view) that bringing a massive number of minds massively faster and smarter than humans would result in our disempowerment, and the serious danger that this would result in mass death, it is unacceptable to subject arguments for safety only to the scrutiny of biased, busy, and scarce company insiders and a handful of external evaluators. My pleas for the companies follow from the above: 1) Actually fucking try to get the world out of this race; 2) Actually fucking try to alert and inform the public; 3) Actually fucking try to scrutinize your own arguments for the justifiability of this plan, which includes subjecting them to way more external scrutiny than anyone has currently done. My pleas for policymakers: use your legal levers to get much more information about these plans into the public view. Make it the top priority -- not only of the next Congress, but of this Congress and the White House -- to understand the situation in its full strategic complexity, and work with technical and policy experts to advance regulation that gets us out of this bind. Domestic coordination must be the first priority, but we need to then quickly develop an international framework with our adversaries. My pleas for the public: do not accept the current plan. Learn about this issue. Call or email your political leaders. Get involved with organizations like @DONOTACCEPTai, @HumansInControl, and @WhoDecidesAI. And finally: believe in ourselves that we can stop this. It is never time to "panic" -- to make rash, unproductive decisions or to feel paralyzed by fear. It is time for us to feel the urgency and the stakes, to demand better, and to meet the moment. Let's do this. 🫡 ✊
AI companies admit what they’re building might kill everyone. We don't have to agree to this. Add your name at donotaccept.ai to say: we do not accept your terms.
72
229
28
1,184
156,963
"Human extinction" is not a loose concept to be treated lightly. Tech Bros in San Francisco throw around their "P(doom)" at parties as a status symbol while actively working to build runaway AI that may soon kill every man, woman, and child on planet Earth. We must stop them.
AI companies admit what they’re building might kill everyone. We don't have to agree to this. Add your name at donotaccept.ai to say: we do not accept your terms.
15
20
1
179
3,403
Max Winga retweeted
No matter what Anthropic does, Dario can just write another 10k word essay and at least half the ea / ai safety crowd will be totally on his side again. Their culture (long form essays) is their kryptonite. They’re powerless against it even when they’re obviously being played.
41
51
12
883
34,140
Max Winga retweeted
Over 125 lawmakers across the UK and Canada are now publicly calling for an international ban on runaway machine intelligence. As new evidence emerges that it could arrive sooner than expected, momentum is building to change course. Thread 🧵
5
23
3
100
2,479
🧵The lawyers here really held @DarioAmodei's feet to the fire and it's interesting to see how he squirmed under oath about AI extinction risk. Under penalty of perjury he's reaffirmed his belief that AI is the greatest threat humanity has ever faced.
Dario Amodei testified under oath that he didn't know his home address
22
38
6
288
31,186
And further we can clearly see the weird twisted logic Anthropic attempts that somehow despite the fact they're making some of the most powerful and dangerous AIs, none of the risk comes from them and they're actually reducing it (while pushing the race as fast as possible)
2
3
2
53
4,579
I'm really looking forward to seeing more testimony of these CEOs under oath in the future. Seems like holding them to old quotes they wrote is best, they're generally more truthful in writing than when you ask open ended questions. What specific questions would you ask?
2
23
1,965
It is pretty rich of you to cite "others could go faster" as a reason to redact so much important information when you yourself do not appear to be slowing down much at all. Nor are you throwing yourself behind efforts to pace the frontier with nearly the gusto that you would if it was your top priority. Not to mention "commercial sensitivity" you yourself have said human extinction is a plausible outcome why are you even talking about commercial factors (well, I guess honesty is good, and since it IS apparently a factor in your decision...) To all my friends at Anthropic: Your company appears to be recklessly gambling with our lives. You fancy yourselves the more responsible alternative to OpenAI (as they fancied themselves compared to Google) but judging by actions rather than words the similarities between the three of you seem stronger than the differences.
36
121
10
1,443
70,334
Max Winga retweeted
This report and reactions from people I trust (like Jake) are making me physically ill. As far as I can tell, Anthropic's central plan is to continue this acceleration of AI capabilities until we're getting decades of AI capabilities growth in a year. Given our very limited ability to understand or control AI systems, this would be extremely reckless and completely unacceptable, and it should be the single top priority of civil society and the US government to stop them and other companies from doing this.
7
18
1
159
6,793
Max Winga retweeted
This is not what it looks like when there's just one or two more alignment issues to fix before it's okay to begin a recursive self-improvement loop. This is the sort of emergent surprise that signals that you have no idea what's going on in there, and that it's time to back off.
🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this. To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
24
111
6
886
23,763
Seems very concerning, AIs might intentionally underperform at safety research. So much for iterative alignment, who could've possibly seen this coming... Nobody should be allowed to build runaway machine intelligences that we have no way to understand or control.
🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this. To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
5
14
346