Computer Scientist, SAP Expert, IT Manager, Consultant, Independent AI Researcher, AI Engineer 👉 AI = Deep Learning + Causal Inference + Symbol Manipulation
United States
Joined May 2022
- Tweets1.8K
- Following1.1K
- Followers270
- Likes132
Dirk Roeckmann retweeted
Reminder: all that gradient descent does is store memories in weight vectors by superposition.
arxiv.org/abs/2012.00152
Dirk Roeckmann retweeted
💯💯💯: “If AI is going to produce trustworthy and genuinely novel insights in science, medicine and beyond, we need systems whose conclusions arise from an auditable process of evidence, inference and belief revision.”
It is madness to believe otherwise.
Ten years ago, AlphaGo’s Move 37 shocked the world.
It wasn’t intuition alone that produced it. AlphaGo could search possible futures, test its instincts and reason about what would happen next.
In a new piece for @techreview, I argue that today’s most advanced AI systems are still missing something fundamental.
LLMs are remarkably capable, but generating longer chains of thought is not the same as genuine reasoning. They typically have no explicit, inspectable record of what they know, what remains uncertain, what evidence supports a conclusion or whether genuine progress has been made.
This is why I recently left @GoogleDeepMind. I believe we need a fresh approach to machine reasoning, drawing on some of the architectural lessons from AlphaGo.
If AI is going to produce trustworthy and genuinely novel insights in science, medicine and beyond, we need systems whose conclusions arise from an auditable process of evidence, inference and belief revision.
Dirk Roeckmann retweeted
In the Middle Ages only the priesthood could access the Word of God because only they could read Latin. Now only the AI priesthood can access the Mind of the Superintelligence because only they can interpret its weights. But the printing press is coming:
tensor-logic.org
Dirk Roeckmann retweeted
The narrative around why building AGI was a good idea has always been about invention -- "it will cure cancer", "it will solve fusion energy", etc.
So I would not "declare AGI" until we have AI that is, at last, capable of invention -- conceptual breakthroughs, novel insights, new real-world technology, etc.
Dirk Roeckmann retweeted
Simply retrieving a reasoning trace looks a lot like human reasoning, until it's time to navigate uncharted territory. If you memorized all reasoning traces of humans from 10,000 BC, you could automate their lives but you could not invent modern civilization.
Dirk Roeckmann retweeted
The reason symmetry is so important in physics is because symmetry is a highly effective compression operator. If a system is invariant under some symmetry, you only need to explain one axis of it. Scientific models represent the systematic exploitation of the universe's internal redundancies through symbolic logic.
Dirk Roeckmann retweeted
Current AI is a librarian of existing knowledge. Science requires an explorer of the unknown. You don't win a Nobel Prize by staying in the library.
Dirk Roeckmann retweeted
The time to learn how to think for yourself was before genAI, if you missed your chance, good luck
Dirk Roeckmann retweeted
The persisting importance of prompt engineering -- and now harness engineering -- is one of the best indicators of how far we are from AGI. A general system doesn't need a task-specific harness. And when provided with instructions, it is robust to phrasing variations.
Dirk Roeckmann retweeted
Sufficiently advanced agentic coding is essentially machine learning: the engineer sets up the optimization goal as well as some constraints on the search space (the spec and its tests), then an optimization process (coding agents) iterates until the goal is reached.
The result is a blackbox model (the generated codebase): an artifact that performs the task, that you deploy without ever inspecting its internal logic, just as we ignore individual weights in a neural network.
This implies that all classic issues encountered in ML will soon become problems for agentic coding: overfitting to the spec, Clever Hans shortcuts that don't generalize outside the tests, data leakage, concept drift, etc.
I would also ask: what will be the Keras of agentic coding? What will be the optimal set of high-level abstractions that allow humans to steer codebase 'training' with minimal cognitive overhead?
Dirk Roeckmann retweeted
Anthropic says Claude doesn't make you more efficient, but it does make you stupider.
arxiv.org/abs/2601.20245
Dirk Roeckmann retweeted
Claude is great at coding. Like really, really good, compared to using a regular LLM like Gemini.
There's only one issue with it. Because it never sees the full code of the app but uses grep search for relevant code snippets, it's myopic.
If grep returns a fragment of code similar to the bug description, it often doesn't look further and fixes an irrelevant part of the app or answers a question based on these fragments found by grep.
So, as the codebase grows, it becomes important for the user to know the codebase. Otherwise, Claude will reinvent the bicycle over and over again, creating duplicate implementations for the same functionalities in different places in the app.
This issue is probably fixable with additional finetuning, but right now this is how it works.
Dirk Roeckmann retweeted
I hate the idea of churning out research papers by the dozen. A research paper should be a work of art.
Dirk Roeckmann retweeted
The Logic and Learning group at the University of Helsinki is recruiting a postdoc to work on cracking the ARC-AGI-3 challenge via Inductive Logic Programming methods.
You'd work under the supervision of Prof. Andrew Cropper. Apply by February 5, 2026 -- link in next tweet!
Dirk Roeckmann retweeted
LLMs represent the "library" phase of AI. The next phase will be the "scientist" phase. A library contains answers, but a scientist knows how to find answers that don't exist yet.
Dirk Roeckmann retweeted
The goal of AI should not be to replace human thought and human agency, but to expand them. Not everything needs to be automated.
Dirk Roeckmann retweeted
Gradient descent is a powerful tool for optimization spaces that verify the manifold hypothesis, but the space of reasoning is discrete and combinatorial.
GD fails in cliff-like landscapes where a single discrete change (a logical step) alters the entire outcome.
Unless...???
Dirk Roeckmann retweeted
AI will evolve from being an automation machine to becoming an invention machine. This will require a fundamentally new paradigm, with symbolic search as its core, not curve-fitting
