Physics → AI safety VP of Research @d_model_ai
Berkeley, CA
Joined August 2010
- Tweets622
- Following141
- Followers319
- Likes13.1K
Adam Scherlis retweeted
Hello, giant surprise audience!
Here's a better one. No garbage bytes and.... oh hey what's that??
Adam Scherlis retweeted
A Weakened Scientific Case for P≠NP
In "The Scientific Case for P≠NP", Scott Aaronson makes the argument that P≠NP because, over and over, the asymptotic runtimes of unrelated algorithms seemingly *conspire* to keep NP-complete problems superpolynomial. It's as if an "invisible electric fence" prevents reductions from being fast enough or producing small enough problems to make the P and NP-complete regions touch.
As one example of many, he mentions the Set Cover problem, where a greedy algorithm produces collections that are at most log(n)+1 times larger than the optimal solution. If they were just (1-ε)log n times larger, then it would be the case that P=NP.
In the second half of the post, Scott argues that the same kind of conspiracy shows up when attempting to solve 3SAT in less than exponential time. Over and over, the runtimes of algorithms that can reduce 3SAT *coincidentally* turn out to run in exponential time.
At the time that post firmly convinced me to believe P≠NP. But, if the Exponential Time Hypothesis is indeed refuted, this argument is put in serious doubt. The seeming mathematical conspiracy that in the second half argued for "3SAT is Ω(c^n)", turns out to be coincidences instead!! Why wouldn't the "invisible electric fence" that separates P from NP turn out to be a bunch of coincidences as well?
Scott himself updated his subjective probability that P=NP from 3% to 5%, which is still low. I've asked him why in the comments to his blog (link in thread) because it's likely that I'm missing intuitions. But I'm pretty sure that the argumentum ad coincidentia parametrica in "The Scientific case for P≠NP" doesn't work anymore.
For this reason that I find the ETH result very shocking. The Platonic realm really doesn't look like what we thought. I feel awe at the power of what is now clearly a mathematical superintelligence.
Adam Scherlis retweeted
In physics, we started with high-level effective theories (Newtonian mechanics of objects in movement) and over time moved to smaller scales until reaching quantum field theory.
In NN interpretability, we start with the fundamental theory (the architecture and gradient descent equations) and attempt to construct higher-level effective theories of objects like "concepts".
Adam Scherlis retweeted
Source-level program logics are a performance optimization. The question is not if they're useful, but whether they'll be invented by humans or by AI. xitter.cf/diagram_chaser/status/…
Adam Scherlis retweeted
K230: 11,487 Wang tiles with no seed or boundary condition.
These tiles cover the entire plane if and only if the Turing machine K230 never halts. K230 halts if and only if ZFC is inconsistent (Glazer, proved in Rocq).
So the full tiling exists exactly when ZFC is consistent, and ZFC itself can neither prove nor refute that it exists, assuming ZFC is consistent and sound.
Adam Scherlis retweeted
increasingly concerned that ai is revealing the mathematical world as negatively curved. instead of everyone wandering about in idea-space and clustering around the same famous conjectures (local positive curvature), ai is pushing out the boundaries rapidly and revealing that the space of mathematical ideas is ~hyperbolic (negative curvature). though this means there's lots left to discover, it also means that small separations between people/agents will lead to wandering alone.
we may be emerging into an idea-space where there's so much to see and do but everyone is alone in a private heaven, unable to share their subtle joys with others...
new Cryptids!!
Over the past year, the Discovery team at @GoogleDeepMind, in a project led by student researcher @dsantosmarco, has made new discoveries in one of the most beautiful open problems at the edge of computation: Finding the 6th Busy Beaver (BB) number.
While most AI math systems solve human-generated problems, like theorems, this work tackles a different aspect of mathematical creativity: Discovering hidden problems.
For years, the BB Community has pursued one of theoretical computer science’s most captivating collaborative quests: finding the simplest programs that run the longest before halting. Because determining whether these programs ever stop is uncomputable, no universal algorithm can solve it, meaning new mathematical techniques need to be found and invented to make progress.
In this way, the BB challenge serves as an open ended domain for mathematical creativity and, in particular, a generator of hard math problems called Cryptids. These are tiny programs whose halting behavior is governed by and equivalent to deep, unsolved math problems in number theory, similar to the Collatz or Goldbach conjectures. Historically, the BB community had recognized ~20 Cryptids in BB(6), and the best problems were selected for the Busy Math Olympiad (BMO).
Building on their work, Marco built a pipeline powered by Gemini 4 Argon that:
1️⃣ Simulates raw Turing machine traces to compress recurring patterns.
2️⃣ Conjectures mathematical transition rules & runs synthetic experiments to test edge cases.
3️⃣ Proves the equivalence formally in Lean 4.
Using this pipeline, Marco and the team discovered 20 new potential Cryptids and verified 33 more machines previously studied in the BB community. All the proofs were open sourced and shared in the community discord channel and we hope that they will help the community to make progress towards resolving BB(6). We also contributed related halting/non-halting proofs directly to the community GitHub repository.
Finding these Cryptids shows hidden links between simple programs and number theory, and helps mathematicians map the limits of computation. I'm also excited about Gemini 4 Argon's ability to help researchers both solve hard problems and accelerate scientific discoveries, of which Busy Beaver is just one example.
Work led by our student researcher @dsantosmarco with paper contributions from @vivek_veeriah, @Dahoas1 , Andrew Bolt, @zuza777 , Satinder Singh, and myself with additional contributions from Hussain Masoom, @maxencefaldor , @MoritzFirsching and Petar Sirkovic.
Read more about it in our paper: arxiv.org/abs/2610.22603
Adam Scherlis retweeted
After reading for 15 mins: I think Nick is operating in a world where interpretability was just about enumerating the neurons and superposition was an obstacle to that plan, so now he's happy because maybe superposition isn't real. But:
1) there are v few empirical results. Unfortunately interp has a long history of significantly overclaiming how interpretable their findings are (SAEs, early circuits having poor causal scrubbing scores etc) so I'm not updating much at all on the evidence he's already shown. it's really got a long history of people getting overexcited. So looking forward to seeing the paper with all the empirical evidence!
2) I think it's not that big news for me even if true because superposition in something like the average case wasn't my main crux to thinking interpretability is tractable. some cruxes this doesn't really deal with at all:
- I think the correct 'circuit' is usually just an overwhelmingly large bag of random-seeming heuristics (a la Redwood Research’s interpretation of how an LLM predicts is vs was) and having it isn't very informative/doesn't help with much/doesn't lead to good predictions.
- More generally, It seems possible to enumerate all the 'low level' features in a neural network, and still have a hard time answering 'high level' questions like: "Is the model misaligned? / What would it do in this new situation? / If it is misaligned what should we do about it?" there's an abstraction/emergence gap to cross from the features to the questions of interest, and crossing that gap is likely much harder than enumerating features.
That said, if his claims hold up, I think it will be quite a big deal, and a useful precursor to tackling the thornier problems! There may also be some useful things to do without tackling the thorny problems: eg we'll be able to make use of masses of automated agents to scour the features and circuits to try and make sense of the mess, and maybe that’s good for something. although I suspect that even the most optimistic outcome here will not make alignment of superhuman AIs much easier (as some on twitter have claimed) because the neuron interpretability will not be adversarially robust, among other issues
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Nice to see coverage of this but the article (text+images) consistently conflates Chair44 with the undecorated "L", which misses the interesting part of the story. Also very sloppy and confusing wrt aperiodic (must be not periodic) vs nonperiodic (can be not periodic).
AI helped uncover a 3D einstein tile, a special object that can fill space without repeating patterns
spklr.io/6012Epfp6
For example the quotes from the interviewed mathematicians are contextualized and "explained" in pretty garbled ways.
I think Quanta did a good job of explaining the key points succinctly: quantamagazine.org/update/16…
Adam Scherlis retweeted
As a lifelong rhombicosidodecahedron enjoyer I cannot overstate my joy at this news 🥹 xitter.cf/ElliotGlazer/status/19…
After 2 years and 1000+ hours of self-motivated research, I have finally finished my proof that the Rhombicosidodecahedron is Nopert (does not fit through itself), settling one of the main open problems in a 333-year-old topic.
github.com/bence-hervay/nope…
