Matrice Jacobine🔸🏳️‍⚧️

Rereading Imas & Garicano's piece, I think it makes a significant mistake. They make a surprising claim: we haven't got any closer to the threshold for an intelligence explosion over the last two years. On their account, the ~2x faster progress METR found at Anthropic is mostly explained by R&D inputs growing about twice as fast. If true, that's a big deal: the speed-up would be mostly money, not AI accelerating research, and should fade once spending growth slows. It's also in tension with METR, who attributed most of the acceleration to AI. Let's back up. The key idea in their model: each doubling of cumulative R&D buys a roughly fixed number of AECI points. So if progress per year doubles, either (a) cumulative R&D started doubling twice as fast, or (b) each doubling started buying more points, meaning returns improved, bringing us closer to the threshold. They say (a). But I think that's because they're effectively using the wrong numbers for historical R&D spend. With Anthropic's actual R&D [1], cumulative R&D grew ~3.3x/yr before Oct 2025 and ~4.4x/yr after, while progress went from ~15 to ~29 AECI points/yr. So each doubling of cumulative R&D bought ~9 points before and ~14 after. Returns rose. That brings us closer to an intelligence explosion. Their fitted curve instead implies cumulative inputs grew ~2x/yr before and ~4x/yr after, so ~15 points per doubling in both periods. The difference comes from their fit implicitly assuming ~$2–3B of R&D before 2024, more than the historical numbers. A bigger starting stock makes 2024–25 spending look like a smaller proportional increase, so early input growth looks slower, and the apparent 2026 spending surge appears to explain the speed-up. But as I say, with the actual historical numbers this spending surge goes away. Spending still increases but it doesn't double more times than it did in the earlier period. (They also fit the two periods separately, letting the starting stock reset: the earlier fit implies ~$2B of pre-2024 R&D, the later one ~$0. In terms of their returns parameter φ: - their separate fits give ~0.55 in both periods. - With actual spending, it rises from ~0.45 to ~0.65. Possible I've missed something, and very happy to be corrected! But on the face of it one of their most surprising and important conclusions looks wrong. [1] S-1 figures: $0.4B in 2023, $2.3B in 2024, $6.3B in 2025, $19B in Jan–Sep 2026. I've added ~$0.2B for 2021–22, before the S-1 data starts.

Recently i've found myself getting more upset with discussions of AI risk on X Like, often it feels like the sceptics are just dunking on ppl who are worried And it's tough bc these issues are really complicated. I don't think there's any water tight case for an intelligence explosion happening soon, or it leading to AI takeover So there are loads of genuine uncertainties and ambiguities, and of course sceptics can and should raise them. But it's upsetting when they present them as slam-dunk proofs that the whole AI risk is flawed and silly. Like, do you really think that no one has thought of this before? Are you actually curious about whether your objection holds up? Remember how most issues are really complicated, and ppl are often wrong about stuff? And the bigger picture here is that hundreds of top experts inside and outside the industry think this will be a complete disaster. Like, that is obviously fucking scary and we should take it very seriously, not notice an apparent flaw in the argument and move on. To be clear, AI safety ppl can be annoying too. Historically, I've found them much more annoying to be honest. I've felt pressure not to say stuff that vibes as 'anti-safety', and noticed ppl engaging in bad faith with me when i do. I know none of this is specific to AI safety. Public discourse is dogshit on every topic. And probably X is waay better than average. But i still think it's a disgrace how low the standard is. For a topic this important everyone on both sides should try to do better

Agreed we're not *yet* above the threshold for an intelligence explosion. But, to be clear, those expecting an IE next year don't think we're above it either They think that we've moved *closer* to the threshold over the past year, and will cross it in the next 6 months I think they're plausibly right