Pinned Tweet
Where does the race to automate AI research end? This is a recording of a recent MATS research talk where I argue that the automation of AI research — which OpenAI and Anthropic say is imminent — could lead to an unrecoverable alignment failure. Three properties make it especially dangerous: oversight breaks down at scale, capabilities self-amplify, and capabilities will be sped up asymmetrically faster than alignment. The outcome could be a lethal, unrecoverable alignment failure.
🤖 Made with AI
BIG. Trump says he and Xi agreed to do joint guardrails on AI.
"It’s not artificial intelligence. It’s not artificial. It’s super. The problem is, it may be too super. So President Xi and I agree that we’re going to do some joint guardrails, the two of us... But whoever wins SI, wins. Remember that."
My position has been that both the US and China will want some kind of binding intl rules on AI, because neither wants rogue actors to have access to superhuman hacking or bioweapon capabilities. This will require technical and/or institutional ways to verify the rules are being followed, without relying on trust. But once you can do that, it you can set whatever rules you want (e.g. stop frontier development), so this is a big development.
I get into how a deal with China could work here: ai-frontiers.org/articles/th…
Which is adapted from my book Obsolete: obsoletebook.org/
This clip annoyed me especially, Dario totally ignores the very direct and very important questions "could AI kill all humans" and then claims that Coxen wasn't calling out Anthropic. But Coxen was very clearly calling out Anthropic in particular, about eg racing ahead to RSI.
This looped transformer thing may be a huge mistake by OpenAI
What assurance can OpenAI provide that 6.2 Astra's prowess on serial reasoning steps without chain of thought isn't because of leaning much harder into looping transformers relative to their peers?
It seems very confusing that 5.2 fable is a comparably capable model to 6.2 astra in many ways but can do so much less reasoning in a single output token.
I don't want this to become a self-fulfilling prophecy or lead other cos to also go harder into architectures that could damage monitor-ability when thats not even what OpenAI is doing, but this seems very odd and it also seems notable that none of the new third party evaluation orgs OpenAI is working with seem to have within their scope evaluating how architectural changes could impact monitor-ability.
1
30
795
"humanity might no longer be in control of our future within 3-12 months."
Quite devastating conclusions from the METR report
There is clearly lot's of this going on on X. Anytime a piece of news negative for the AI industry breaks a whole swarm of bots and some real people suddenly agree on a weird narrative and then relentlessly push it. Like when AI safety was suddenly a psyop, when export controls suddenly hurt the US, or all datacenter opposition was paid for by China, etc.
I just unraveled the biggest AI astroturfing campaign I’ve found yet.
I found hundreds of fake accounts posing as Americans. The operation traces back to a supposedly defunct Israeli influence company.
They posted thousands of times in the last year to support Nvidia, back chip exports and data centers, attack U.S. lawmakers, and dismiss concerns about AI.
When it broke that Jensen Huang, Nvidia's CEO, was romantically involved with an alleged spy (who runs a company accused of chip diversion and whom US officials believe was passing information to Beijing), dozens of these accounts rushed to Jensen’s defense.
These accounts have targeted people pushing for export controls that ensure advanced chips aren’t going to China. Chris McGuire, a former White House national security official, got more than 100 replies from the astroturf network attacking him and his position on chips. Lawmakers who criticized the chip sales were called “peabrains” and “pure evil,” and accused of spreading “anti-Trump propaganda.”
Throughout this yearlong pro-Nvidia campaign, the operators have been hiding their tracks. They quietly deleted nearly two-thirds of the pro-Nvidia posts that we found, something they seem to do for many of their campaigns.
These accounts have spent years running campaigns that appear to serve the interests of corporate and political clients — most recently supporting the Rapid Support Forces in Sudan’s civil war, a group the U.S. government has accused of genocide.
Many of them trace back to a supposedly defunct Israeli influence company, Koios. In 2025, Haaretz reported that Koios helped run a covert pro-Qatar campaign devised by aides to Benjamin Netanyahu. While Koios claims to have ceased operations in 2025, it looks to me that many of its associated people and accounts moved their work to a new company, IntellACT, that spun up just as the old one shuttered.
Nvidia told me it has no relationship with the Israeli team linked to these accounts, and that while it has recently become aware of the operation, it is also trying to find out who is behind it. So, who is funding a yearlong campaign to shape one of the most important policy fights in history? I’m not sure, but I think it deserves a formal investigation. And Congress has that power.
Read my full article:
nitter.gq/TheMidasProj/status/21…
Simon Lermen retweeted
Paul Christiano says there is meaningful risk of a catastrophe from AI in the next 6-12 months.
"I think there's meaningful risk even in the next six to 12 months."
"I think we have seen roughly the things we might expect to see shortly before something really bad were to happen."
One of the original AI alignment researchers, @paulfchristiano, at the Hinton Lectures:
"I think there is a high risk of an AI takeover within the next three years."
"If, say, you or me or Donald Trump wanted to stop [AIs], they would be as ineffective as if you or me was trying to stop the US military."
Wonderful video, there is so much to fight for.
AI companies admit what they’re building might kill everyone. We don't have to agree to this.
Add your name at donotaccept.ai to say: we do not accept your terms.
Simon Lermen retweeted
We are about to wipe out the jobs for our children and give them instead to computers so the tech oligarchs can become trillionaires.
This is f*cking wild. Someone asked Anthropic CEO Dario Amodei what a kid finishing high school should do right now. He didn't sugarcoat it:
"AI can do entry-level law work. It can do entry-level finance. It can do entry-level consulting.
AI will also eliminate jobs, and probably a large number of them"
In 42 seconds, he lists the first careers in the line of fire: junior law, junior finance, junior consulting
He's already put a number on it: half of entry-level white-collar jobs disrupted within 1 to 5 years
Every shift before this one, from farms to factories to the internet, gave people time to catch up. This one, he says, cuts deeper and moves faster
Chatting → Prompting → Agents → Harness → Loops → You Run the AI
His advice is simple: learn to use AI. But using it is now the bare minimum. What sets you apart is a harness that runs it for you
Bookmark this before your feed buries it
Every repo below costs nothing:
Ready-made Claude plugins for legal, finance, sales, HR and data
github.com/anthropics/knowle…
Analyst-grade agents: models, memos, research notes
github.com/anthropics/financ…
Skills for Excel, Word, PowerPoint and PDF
github.com/anthropics/skills
An agent that does financial research on its own
github.com/virattt/dexter
An agent that researches any topic in depth
github.com/assafelovic/gpt-r…
Ask your spreadsheets and databases questions in plain English
github.com/sinaptik-ai/panda…
Agents that work as a team, each with its own role
github.com/crewAIInc/crewAI
A harness with everything built in
github.com/langchain-ai/deep…
Anthropic's SDK for building your own agents
github.com/anthropics/claude…
A memory layer so your agents remember context
github.com/mem0ai/mem0
A loop that keeps the agent going until the job is done
github.com/snarktank/ralph
An agent that clicks through websites and fills forms for you
github.com/browser-use/brows…
Automate busywork with drag-and-drop workflows
github.com/n8n-io/n8n
18 free lessons on building agents
github.com/microsoft/ai-agen…
Total cost: $0
It won't be AI that takes your seat. It'll be someone who built a harness
95% will keep sending résumés for work AI already does, the 5% with a harness will be the ones managing it
Save it and forward it to anyone starting their first job
Then follow the guide below and build your harness while everyone else is still waiting ↓
We are now seeing a lot or promising signals from politicians, but unclear is we have enough time or they will go through with sufficiently harsh measures. IMO Anthropic accelerated timelines at least twice, once with Claude Code and now with their very aggressive RSI. So the final consequence of founding Anthropic might be that it shortened timelines just enough to prevent any reasonable policy action.
Conservative pundit @MattWalshBlog on banning superintelligence:
"That technology by definition is outside of human control."
"Should we tell them they can't do that? Yes, clearly we should."
Lot's of promising signals from politicians recently but we still have this guy in the white house.
As President, I am 100% in favor of Super Intelligence (SI), and will work with the many States in the U.S. that want to become RICH by having them. They are GOLD, and only foolish leadership will say “NO” to the riches they bring! President DJT
( TS: Nov 6 2026, 7:01 AM ET )
Look at this huge jump from model 4 (unreleased internal model). Seems like they already hit 2x speedup from previous models at AI capabilities research.
Replying to @METR_Evals
3. Interviews and surveys with staff are consistent with AI usage driving current acceleration, though this evidence is very hard to interpret. Staff self-report dramatic productivity gains, but say the most important AI R&D contributions aren’t yet fully automated.
On a meta level: Some people argue that even all AI *safety* staff at AI companies should quit - but you have to admit that then we wouldn't be getting investigations like this.
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
Simon Lermen retweeted
We at AIFP are spooked by this information. IF (big IF) Anthropic really is imminently about to meet the RSP threshold of 2x acceleration over baseline, then it seems timelines to full RSI and superintelligence may be very short indeed. For example if we naively optimize the AIFM parameters to achieve 2x acceleration over baseline progress in Jan 2027, we get superintelligence sooner than depicted in AI 2027. We are thinking harder and longer about this now and will hopefully have a lengthier update on our thinking to share in the next week.
Anthropic updated its S1 this morning. Notable additions re automated AI R&D and human oversight of it:
"[...] our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of 'dramatically accelerated progress attributable to AI.' We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data. We are also evaluating the implications of meeting the AI research and development capability threshold and will release further reports in the future as we set forth in our Responsible Scaling Policy."
"We are also considering additional measures, including updates to our Responsible Scaling Policy, to require human review and approval of certain decisions in our research and development processes as our models perform an increasing share of that work."
"We believe our investments in safety techniques will enable us to manage these risks and responsibly realize substantially more leverage from Claude than we do today in developing AI models. However, we do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible. Accordingly, we are considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development at Anthropic."
sec.gov/Archives/edgar/data/…

























