
Ass. prof. of Generative AI. PI of Generative Memory Lab (@DondersInst). Generative diffusion, language diffusion and statistical physics. AI realist.
Nijmegen, Nederland
Joined July 2011
- Tweets12.1K
- Following2.8K
- Followers7.1K
- Likes49.9K
Pinned Tweet
1/?) As promised to Sander Dieleman (@sedielem), we’re finally excited to share:
Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion
We show that continuous diffusion can achieve very strong language modeling performance when operating directly on bitstreams, outperforming masked and uniform diffusion baselines, and essentially matching autoregressive models under our evaluation settings.
Luca Ambrogioni retweeted
It's an amazing time to be alive.
Luca Ambrogioni retweeted
The first quote makes sense. The second is absurd. Bigger than the publication of Newton's Principia, Darwin's Origin of Species, Gödel's theorems and Einstein's field equations? We (collectively) are going insane and need to get a grip.
"This is at least ten times, perhaps 100 times as momentous as the proof of Fermat’s Last Theorem,” said Abhishek Saha, professor of mathematics at Queen Mary University of London. Joshua Gans of the University of Toronto went further: “This is probably the biggest day of scientific advancement in history"
spectator.com/article/the-fi…
Luca Ambrogioni retweeted
The largest math proof in human history is now computer-verified.
Today, Axiom announces a complete Lean auto-formalization of the Classification of Finite Simple Groups, the "periodic table of symmetry."
About 15,000 pages, built by over 100 mathematicians across generations.
Luca Ambrogioni retweeted
The mathematicians are demanding the immediate release of the theorems held hostage.
agmai.org/#request-immediate…
Luca Ambrogioni retweeted
🔥 Diffusion LMs promise what autoregressive models can’t: fixing their mistakes.
But new ones often change the noise and the sampler at once. Which matters?
We tested it on the hypersphere. Noise: 0.8 points. Sampler: 25.
We introduce Spherical Brownian Diffusion 🌐 👇🧵
Luca Ambrogioni retweeted
Here is a short and simplified proof of ETH is false.
My goal here is to obtain a simplified disproof of the basic statement of ETH, without optimizing log factors.
It is self-contained and easy-to-understand. I have added a worked example also.
Paper is on my homepage: shivakintali.github.io
PDF file: shivakintali.github.io/paper…
CC: @AlexKontorovich @__alpoge__ @jdlichtman
@stevenstrogatz @sahilsingla81 @MTHajiaghayi @AlgoSvensson @kenono691 @littmath @nasqret @PI010101 @wtgowers @MarcusduSautoy @DrEugeniaCheng @JSEllenberg @JohnDCook @JDHamkins @AxlerLinear @oveisgharan @ch_nira @rrwilliams @lreyzin @ChrisPeikert @fortnow @RadishHarmers @thomasfbloom @abhatt2 @mahdi_tcs_ @octonion @prz_chojecki @thegautamkamath @KWRegan
We are announcing three new discoveries in mathematics, computer science, and physics about harmonic analysis, complexity theory, and quantum field theory.
openai.com/index/building-sc…
Luca Ambrogioni retweeted
This Monday, Nov 2, Dmitry Baranchuk (@DmitryBaranchuk) will present his paper:
Representation-Space MMD for Diffusion Language Models
Project also co-led by Ilya Drobyshevskiy (@free001style) and Ilia Sudakov (@sudakov_il)!
Luca Ambrogioni retweeted
curious about this.
all of these theories in 4d are weakly interacting in the UV, which means that they are basically free particles at small scales. i never know how much of free quantum field theory really "exists" in the math sense, but in a sense i'm not shocked --
Replying to @danintheory
The physics collaboration will understand and explain a result that gives a mathematically rigorous existence proof of certain interacting quantum field theories containing gauge fields and fermions in four dimensions, including Quantum Chromodynamics (QCD) and a unified completion of the Standard Model of particle physics.
As a subproblem, this work also includes a proof of the existence of pure Yang–Mills theory, which is part of the Quantum Yang–Mills Theory Clay Mathematics Institute Millennium Prize problem. These proofs are all formally verified in Lean.
Our models produced a natural language proof for Yang–Mills existence, and then subsequently produced a proof for QCD After the formalization completed, we reached out to David Gross. David will lead a group including Michael Douglas, Arthur Jaffe, Igor Klebanov, and Nikita Nekrasov that will shepherd this work, and we will defer to them on the release of materials.
Gross’s work on asymptotic freedom in Yang-Mills theories helped establish QCD as the theory of the strong interactions and earned him a share of the 2004 Nobel Prize in Physics. Jaffe formulated the Quantum Yang–Mills Theory Millenium Prize problem with Witten and also envisioned the idea of Millennium Prize problems and co-founded the Clay Mathematics Institute. Douglas has already been a steward of this problem over the last 20 years; Klebanov helped pioneer the application of gauge–string duality to confinement, an important property of QCD, and now directs the Simons Collaboration on Confinement and QCD Strings; and Nekrasov has done groundbreaking work on exact methods for generalizations of Yang–Mills theory.
Luca Ambrogioni retweeted
Introducing Nestor, a new generation of scalable architectures that generalize MoE transformers and explicitly promote sparsity in both attentions and experts, hence learn structured deep representations more efficiently. Work with @TianzheC and @druv_pai: ma-lab-berkeley.github.io/Ne…
Luca Ambrogioni retweeted
The Exponential Time Hypothesis is false.
Quoting:
'Our models produced a natural language proof for Yang–Mills existence on September 9th, 2026, and then subsequently produced a proof for QCD on September 18th, 2026. Given the length and complexity of the work, we postponed release until the Lean formalization completed. We finished our formalization of Yang–Mills’ existence on October 16th, 2026, which led us to develop the collaboration described here.'
Luca Ambrogioni retweeted
OpenAI has announced to three new discoveries in mathematics, computer science, and physics:
- Local smoothing and the Kakeya conjecture
- The Exponential Time Hypothesis is false
- Existence of quantum chromodynamics and completions of the Standard Model
Luca Ambrogioni retweeted
Tomorrow @HarmonicMath launches something I've spent the last year working on.
Luca Ambrogioni retweeted
Update on Navier-Stokes: 🦜🦾
"On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding: there are several different groups of pde experts working on understanding it who say that they more or less understand now."
"Some on these ideas are in the literature (e.g. Leibovich-Stewartson 1983) but the way they are combined is definitely insane and definitely nowhere in the literature previously.
I also don't know of anyone suggesting that convex integration would play a role in the creation of a blow-up solution like this."
Since I'm now blocked by bluey from replying to this thread, I answer here by quote tweet
the openAI construction has more to do with convex integration than anything else.
In convex integration, the oscillations are typically built by hand and the iteration procedure creates wild weak solutions. The non-uniqueness construction of Albritton–Brué–Colombo used an unstable vortex to split solutions apart. The Córdoba–Martínez-Zoroa singularities amplify vorticity across scales by strain. In the openAI paper, a fluid instability is clever made so that the flow itself generates the oscillations: they grow by drawing energy from the background shear. The Reynolds stress then becomes the force that creates the collapsing smooth vortex. Using a physical instability to create the stress in this way has no precedent, and it is what produces a smooth force rather than a merely rough one.
Some on these ideas are in the literature (e.g. Leibovich-Stewartson 1983) but the way they are combined is definitely insane and definitely nowhere in the literature previously.
I also don't know of anyone suggesting that convex integration would play a role in the creation of a blow-up solution like this.
On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding: there are several different groups of pde experts working on understanding it who say that they more or less understand now. I expect digestions soon
td;lr: anyone saying that this OAI result is unsurprising is bullshitting
sorry!
Luca Ambrogioni retweeted
Can a neural network learn ~2^100 LDPC (Low-Density Parity-Check) codewords from just 10-20 examples?
👉 Grokking in LLMs and memorization-to-generalization transition in diffusion models are related phenomena — the model learns true process of data generation from a small number of training samples.
👉 It is hard to study this phenomenon analytically on natural datasets.
👉 LDPC codeword generation is a precise mathematical process in which the codewords are constructed as solutions to parity check equations.
👉 We train a Dense Associative Memory with a tiny fraction of codewords from the full vocabulary, without giving the network the generation algorithm itself. The model manages to learn the entire vocabulary on its own. It turns out, you only need log(n) samples to learn the entire space of 2^n codewords.
👉 The network only needs approximately P=10-20 training examples to correctly learn the full vocabulary containing ~2^100 codewords.
Our new paper with Andy Lucas: arxiv.org/abs/2610.27384.
Let us know what you think!
Luca Ambrogioni retweeted
Hot take: the community keeps asking whether diffusion LLMs can beat AR. That's the wrong question. Diffusion is a sampling method, and it can sit on top of an AR model to make it faster instead of replacing it.
Speculative decoding already speeds up AR, but it has two long-standing problems. Training a drafter takes a long time, and you have to build a complete pretraining corpus for it. Even then, token acceptance drops sharply on out-of-domain data.
Uno @ssahoo_ offers an elegant way out: it decouples sampling training from capability training. There's no separate drafter. The model itself, plus one LoRA, drafts several tokens in a single forward pass, and the LoRA learns to match the model's own next-token distribution, so there's no drafter corpus to build and no labels to collect.
We applied this to the Looped Transformer Nanbeige4.2-3B: 1 hour of LoRA training gives a lossless 2× speedup on agentic task. We haven't optimized anything yet, and we believe the acceleration can go much further.
The takeaway: diffusion doesn't need to beat AR. Any model that can score its next token can get a fast diffusion mode in an hour.
Page: fengsxy.github.io/LoopUno
Lora Adapters: huggingface.co/liddlefish/Lo…
Luca Ambrogioni retweeted
For the Navier-Stokes proof:
"On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding"
Since I'm now blocked by bluey from replying to this thread, I answer here by quote tweet
the openAI construction has more to do with convex integration than anything else.
In convex integration, the oscillations are typically built by hand and the iteration procedure creates wild weak solutions. The non-uniqueness construction of Albritton–Brué–Colombo used an unstable vortex to split solutions apart. The Córdoba–Martínez-Zoroa singularities amplify vorticity across scales by strain. In the openAI paper, a fluid instability is clever made so that the flow itself generates the oscillations: they grow by drawing energy from the background shear. The Reynolds stress then becomes the force that creates the collapsing smooth vortex. Using a physical instability to create the stress in this way has no precedent, and it is what produces a smooth force rather than a merely rough one.
Some on these ideas are in the literature (e.g. Leibovich-Stewartson 1983) but the way they are combined is definitely insane and definitely nowhere in the literature previously.
I also don't know of anyone suggesting that convex integration would play a role in the creation of a blow-up solution like this.
On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding: there are several different groups of pde experts working on understanding it who say that they more or less understand now. I expect digestions soon
td;lr: anyone saying that this OAI result is unsurprising is bullshitting
sorry!
Luca Ambrogioni retweeted
PhD applications at @GatsbyUCL are open!
Full funding available regardless of nationality.
Includes ✨new✨ Jump Trading studentships in AI with potential application to finance, eligible for employment opportunities with Jump Trading during PhD.
Deadline: 🚨November 9th🚨
Interested in a #PhD in machine learning & theoretical neuroscience?
Applications for 2027 entry to the Gatsby Unit PhD programme now open!
💰 Fully-funded 4-year programme
🤝 Close links to Sainsbury Wellcome Centre & the ELLIS Unit @ UCL (@uclcsml)
ℹ️ ucl.ac.uk/life-sciences/gats…





























