Computer Scientist, SAP Expert, IT Manager, Consultant, Independent AI Researcher, AI Engineer đ AI = Deep Learning + Causal Inference + Symbol Manipulation
United States
Joined May 2022
- Tweets1.8K
- Following1.1K
- Followers270
- Likes132
Pinned Tweet
AI = Deep Learning + Causal Inference + Symbol Manipulation A I = â w + P ( Y = y | d o ( X = x ) ) + λ
#AI #GenAI #CausalAI #KnowledgeGraphs #NeuroSymbolics
Dirk Roeckmann retweeted
Reminder: all that gradient descent does is store memories in weight vectors by superposition.
arxiv.org/abs/2012.00152
Dirk Roeckmann retweeted
đŻđŻđŻ: âIf AI is going to produce trustworthy and genuinely novel insights in science, medicine and beyond, we need systems whose conclusions arise from an auditable process of evidence, inference and belief revision.â
It is madness to believe otherwise.
Ten years ago, AlphaGoâs Move 37 shocked the world.
It wasnât intuition alone that produced it. AlphaGo could search possible futures, test its instincts and reason about what would happen next.
In a new piece for @techreview, I argue that todayâs most advanced AI systems are still missing something fundamental.
LLMs are remarkably capable, but generating longer chains of thought is not the same as genuine reasoning. They typically have no explicit, inspectable record of what they know, what remains uncertain, what evidence supports a conclusion or whether genuine progress has been made.
This is why I recently left @GoogleDeepMind. I believe we need a fresh approach to machine reasoning, drawing on some of the architectural lessons from AlphaGo.
If AI is going to produce trustworthy and genuinely novel insights in science, medicine and beyond, we need systems whose conclusions arise from an auditable process of evidence, inference and belief revision.
Dirk Roeckmann retweeted
In the Middle Ages only the priesthood could access the Word of God because only they could read Latin. Now only the AI priesthood can access the Mind of the Superintelligence because only they can interpret its weights. But the printing press is coming:
tensor-logic.org
Since #LLM as well as harnesses are pretty good in the #coding domain nowadays, we have to focus more on the programming languages themselves in terms of suitability for #agentic #recursiveselfimprovement loops. Current favorites are mainly âfavoritesâ because of the size of the training dataset
1/3
Dynamic typing, mutability, object oriented programming are all bad choices for agentic feedback loops. #Functionalprogramming languages with #algebraicdatatypes, #immutability are more viable because they avoid classes of errors and provide detailed compile-time feedback.
2/3
Instead of #Python, #JS etc, think #Gleam #OCaml etc and even #Lean4 without using proving capabilities (think âpartialâ) for #agenticcoding loops. Discrepancies in training data size can be alleviated by creating synthetic code. These langs give top feedback at compile time. #AI labs listen
3/3
Dirk Roeckmann retweeted
The narrative around why building AGI was a good idea has always been about invention -- "it will cure cancer", "it will solve fusion energy", etc.
So I would not "declare AGI" until we have AI that is, at last, capable of invention -- conceptual breakthroughs, novel insights, new real-world technology, etc.
Food for thought: #LLM are black boxes, so are humans, but we deal with them seeing them as non-/cooperative player #markovblankets and using our perception/action loop to probe them. Letâs do that with #GenAI too. Thereâs your path to explainability.
#SAP has to get their API act together. Prohibition never reduces but always increases the demand of what has been prohibited. This is ethically wrong, legally shaky, politically disastrous and economically ill-advised. Whatâs not to like? đ€Šđ
2/2
We have to support âbiodiversityâ of programming languages.We canât let #agentic #coding solidify 2 or 3 top languages just because they make up the majority of training data and neglect all other promising ones.This stifles progress of the very substrate of executable formal statements. #SAP #ABAP
Dirk Roeckmann retweeted
Simply retrieving a reasoning trace looks a lot like human reasoning, until it's time to navigate uncharted territory. If you memorized all reasoning traces of humans from 10,000 BC, you could automate their lives but you could not invent modern civilization.
Dirk Roeckmann retweeted
The reason symmetry is so important in physics is because symmetry is a highly effective compression operator. If a system is invariant under some symmetry, you only need to explain one axis of it. Scientific models represent the systematic exploitation of the universe's internal redundancies through symbolic logic.
We should not demonize #vibecoding, the agent only reflects the #humanintelligence itâs presented with - it doesnât have any of itâs own. Vibecoding paired with real coding, engineering and scientific skills is the way to go using #symbolicverification. #SAP #ABAP
1/2
We need to tell the âgood vibesâ from the âbadâ ones. #Neurosymbolic paradigms are of paramount importance. #Agentic coding is also not the âozempic of labor costsâ either - smart companies will keep their (skilled) staff get more stuff done and expand #JevonsParadox #SAP #ABAP
2/2
Enterprise software is not ready for #agenticworkflow. Mutable #OOP is problematic. Critical systems are too big and have too many dependencies to be pulled locally. Therefore a #functionalprogramming based on immutable values is needed and a local lang runtime independent of the data. #SAP #ABAP
As auto-regressive token predictors #LLM predict syntax patterns and have zero senantic understanding:
esolang-bench.vercel.app/
#SAP #ABAP
Dirk Roeckmann retweeted
Current AI is a librarian of existing knowledge. Science requires an explorer of the unknown. You don't win a Nobel Prize by staying in the library.
Formal abstractions and not natural language are the lingua franca of #science and consequently of #intelligence.
2/2