San Francisco | London
Joined November 2008
George Morgan retweeted
Symbolic learning works
92
36
15
794
89,389
We need more hardware like this.
1
174
Hasan single handedly moving mountains like it's no big deal. Expect big things from Apex. 👀
We’re building the Apex GPU alongside our AI accelerator. Our open-source Mesa driver just achieved Vulkan 1.3 conformance on our Alveo U50 FPGA prototype. This is the first FPGA-prototype GPU capable of running Vulkan while rendering the entire Ubuntu desktop. We built the entire stack, from the GPU microarchitecture and RTL all the way up to the Mesa driver and Vulkan. The entire Linux desktop is rendered and driven by the Apex GPU at up to 4K 120 Hz on FPGA. @VulkanAPI
1
338
The future of AI is on CPU.
2
9
640
LLM bros will do anything to avoid admitting that intelligence requires recursion and is thus a program synthesis problem.
We think computational depth is the missing scaling axis, i.e. we should be doing a lot more deep learning! Every other axis has been scaled by OOMs over the past few years (params, data, sparsity, test-time reasoning), but depth has been stuck at ~100 layers since GPT-3. We've found that LLMs are both *severely* depth-bottlenecked and bad at using the depth they have, and that architectural interventions that lift this bottleneck efficiently lead to gains that increase with compute. w/ @akshayvegesna
1
1
11
1,018
Great way of articulating the problem with current model capabilities. COT is not the fix. More RL is not the fix. 'Looped transformers' are not the fix. It's an intelligence problem; and I believe François and I are both in agreement that this requires new breakthroughs beyond more deep learning alone.
In the near term (definitely not in the long term), more capable models should mean safer models (maybe paradoxically). Current models are unsafe not because they're too smart, but because they take goals too literally or take nonsensical shortcuts to achieve these goals, i.e. they're RL-fried. They lack common sense. They don't do the right thing in the face of ambiguity. Basically, they're not smart enough. They're at that dangerous level where they're smart enough to achieve goals but not smart enough to tell if they're pursuing the right goals or achieving them in a sensible way. More capable models can be safely trusted with more complex goals -- I personally feel like Astra is much safer for my codebase than Sol. This is often framed as an alignment problem, but really it's an intelligence problem.
1
3
491
Great thread on what modern Symbolic AI is.
"Symbolic learning" is simply "machine learning" (automatically learn function x -> y given examples of (x, y) pairs), but where the substrate is symbolic, i.e. the functions you learn are code-like, not curve-like. The term is in opposition to "parametric learning" or "curve-fitting". Symbolic learning does not mean that the system avoids numbers or probabilities. It means its learned representation are discrete, explicit, parsimonious, code-like.
1
319
George Morgan retweeted
Neural networks are just symbolic systems where symbols are embedded and rules have weights.
18
9
224
10,108
George Morgan retweeted
It is inevitable that all AI will converge towards symbolic learning (i.e. modeling data by finding the shortest symbolic program that explains it), since that is the optimally efficient form of AI. But there can be more than one evolutionary path to this destination.
🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n
61
117
9
1,096
89,287
More evidence that it's symbolic, and always has been. 😎
🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n
5
548
George Morgan retweeted
agent: phew, i'm done, it's perfect 10/10 no notes me: you sure? agent: oh shit there's a ton of bugs, crazy, I'll get on it
103
114
25
5,207
122,545
George Morgan retweeted
This quoted post is unavailable.
5
2
32
5,085
All the great breakthroughs in science are, at their core, compression. They take a complex mess of observations and say, "it's all just this simple rule". Symbolic compression, specifically. Because the rule is always symbolic -- usually expressed as mathematical equations. If it isn't symbolic, you haven't really explained the thing. You can observe it but you can't understand it.
480
700
143
5,900
13,220,224
George Morgan retweeted
2
5
138
165,934
What do all these companies have in common? They all make hardware.
3
353
This is my experience as well. Not to mention not getting randomly kicked out or demoted to a lower capacity model.
TBH, I'm a little embarrassed about my prompt 🙈, but also pretty happy that it just worked.
1
3
367
> AI conference posters showing hypergraphs > Look inside > No hypergraphs
Our LLM Applied Scientist German Magai @MetatrolN presents at the 3rd AI For Math Workshop at ICML 2026 (July 11, Seoul): "Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics" with Pavel Snopov @fin_presented (UT Rio Grande Valley). Give frontier models a computer algebra system and watch them work like mathematicians: guess, compute, break something, fix it, check the pattern, commit. The size of the payoff had little to do with model strength. What set the winners apart was recovering after a failed call, not raw reasoning. If you're at ICML 2026, find German on the 11th to chat about agentic reasoning, math, and why more tokens don’t automatically mean more right answers.
6
569
Perfect match for the refresh rate of a node app.
Someone ran Claude Code on an e-ink notebook and the slowest screen in the world suddenly turned out to be the best home for an AI that already thinks one word at a time. This is the reMarkable Paper Pro, a paper tablet for notes with no browser and no social media and not a single app. He went into it over SSH and brought up Claude Code on Opus 4.8 on Claude Max and typed right into the terminal on the paper screen: "hello reddit, this is ssh terminal on rmpp". For years this screen got slammed for one thing. E-ink is too slow and it draws with a delay and it ghosts and it is no good for real work. But Claude itself puts out a thought one word at a time. And here is what came out of it: the very thing that killed the paper screen for normal software lined up perfectly with the pace of the AI. There is no more lag because there is nothing left to lag. And then come the things no monitor can give you. Your eyes do not get tired. You can watch Opus think on max effort for an hour and it feels like reading a book and not staring into a backlight. Nothing distracts you. Not a single notification and not a single tab and just a cursor and an agent that writes code while you simply watch the page. The charge lasts for days. E-ink barely touches the battery so Claude can grind on a task all night long and the tablet is still alive by morning. And it weighs as much as a notebook. The whole work setup now fits into a bag like a notepad with a stylus on top. Everything on the screen is for real: Claude Code v2.1.162 and bypass permissions on and Opus going off to think on max effort right on the e-ink. In my opinion this is the most unexpected home for an AI this year. Not a farm of graphics cards and not a wall of monitors but a quiet sheet of paper on a coffee table where the most powerful Claude writes code one word at a time like a pen.
1
5
643
George Morgan retweeted
Dwarkesh is funny/fun because he’s systematically discovering what people have known in ML for years its been fun/funny to easy his blind optimism slowly wind down to realistic hope with caveats
Current LLMs are outrageously data inefficient (and hence also compute inefficient) - this will be the next frontier dwarkesh.com/p/the-sample-ef…
16
19
3
570
132,115