I love to think. Programming for 21 years.
Joined May 2015
- Tweets37
- Following337
- Followers41
- Likes77
Excited to see @jetinfer joining @Architect_Fi's inference provider network!
We're focused on delivering fast, cost-efficient inference for open-weight models, and it's great to collaborate with teams pushing this ecosystem forward.
Thanks to Brett and the Architect team for having us on board! 🚀
Richard Nemeth retweeted
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
Richard Nemeth retweeted
Introducing PC-ALM, a local-learning alternative to backpropagation.
Our method trains 1000-layer neural nets using only local dynamics, and without backprop.
Blog: pub.sakana.ai/pc-alm/
Standard deep learning relies on backpropagation. The brain, however, cannot implement backpropagation, at least not exactly. How can a physical system, such as the brain, solve multilayer credit assignment without explicit use of backprop?
We look for inspiration in two related fields: distributed optimization and NeuroAI.
In NeuroAI, predictive coding asks each neuron activation to solve an energy-based inference problem instead of using a standard forward pass. That inference step can be implemented as energy-minimization dynamics on local prediction errors.
This perspective -- each layer as a dynamical system -- has proven promising, but performance of predictive coding hasn't scaled well with depth. Credit signals at far ends of the network struggle to diffuse into internal layers.
We turn to distributed optimization, generalizing predictive coding to use an augmented Lagrangian instead of energy. This motivation stems back to a classic 1988 paper by LeCun, showing that the Lagrange multipliers of a deep network can be identified with gradients of a supervised loss. The augmented Lagrangian then bridges LeCun's perspective to the standard predictive coding that is used in NeuroAI.
We find that this new perspective yields a natural PC-like alternative to backpropagation, resulting in a method we call PC-ALM. PC-ALM differs from PC in that it introduces dual neurons (Lagrange multipliers) as part of the layer-local dynamics, resulting in each layer acting as a PI feedback control system to minimize local prediction errors.
We find that PC-ALM is capable of propagating signals to seemingly arbitrary depth, especially in deep narrow networks where standard PC struggles to learn.
Ultimately, our motivation here is to understand how distributed physical systems, such as the brain, can compute credit signals using only local coupling and local dynamics.
PC-ALM may also inform deep learning in neuromorphic hardware, where dynamics are cheaper than on GPUs.
Paper: arxiv.org/abs/2605.31022
Code: github.com/SakanaAI/pc-alm
AGI may be the wrong goal.
Current LLMs already look like IAGI — Inefficient Artificial General Intelligence.
This benchmark is a good example:
Human: ~2.8k words of notes
GPT-6 Astra: ~22k
Fable 5.1: ~192k
The human gets away with far less external memory. We’re extremely good at compressing information into useful abstractions.
LLMs compensate with more memory, search and compute.
And still win.
Chess engines are the obvious analogy. Evaluation heuristics + searching huge numbers of positions were enough to become superhuman.
A* pathfinding works on the same principle. The heuristic doesn’t need a perfect representation of the world. It just needs to make search cheaper.
brute force → heuristics → efficient intelligence
Humans are far toward the efficient end.
AI may not need to get there. A worse representation can still win if enough compute sits behind it.
That’s basically Sutton’s Bitter Lesson: methods that can keep exploiting more compute tend to beat handcrafted human knowledge.
Not because they’re more elegant. Because compute scales.
AI may become far more capable than us long before it becomes remotely as efficient as us.
That’s IAGI.
Richard Nemeth retweeted
OPENAI COFOUNDER SAYS EMOTIONS MAY BE WHY WE CAN MAKE DECISIONS
ilya brings up a guy who lost emotional processing after brain damage. he could still talk normally and solve problems, but somehow became terrible at making basic decisions
"It would take him hours to decide on which socks to wear."
we spend so much time treating emotions like the annoying part of thinking. calm down, remove the feelings, be logical
then you remove them completely and even choosing socks becomes a serious problem
kinda changes how i look at AI advice too. it can give you 20 perfectly reasonable answers in seconds, but somebody still has to care which outcome actually matters
you can understand every option and still not know which one matters to you
~1,300 token/s — GPT-5.6 Sol
~3,500 token/s — Qwen 3.8 27B, bringing early-2026 frontier-level intelligence to insane inference speeds.
@cerebras cooking. 🔥
Richard Nemeth retweeted
we are releasing our own coding agent, I have been using it daily since a month now and my favorite place to work with open and close model
1. RLM:
* programmatic tool calling
* persistent ipython kernel across compaction for long term memory
* Subagent daemon and UI
2. Self improving continual harness, model have access to tool to programmatically edit their own harness and skills
Richard Nemeth retweeted
This is exactly right. Source code is on the verge of becoming like assembly.
The next step is getting rid of “source code” entirely and just making an efficient binary directly with AI.
Richard Nemeth retweeted
introducing BarunLM (35m) 🤗
i pretrained the world's best small model under 100m parameters.
a 35m parameter language model that outperforms models over 6x larger, including LiquidAI's lfm2.5-230m, while training on a single H200 gpu.
the gains come from a better architecture. we took inspiration from deepseek, kimi, and other frontier open source models.
BarunLM combines hybrid local and global attention, grouped query attention to reduce memory, partial rope, gated attention, bounded swiglu, and residual selectors to improve how information flows through the network.
after pretraining, we further improved the model with rlvr (reinforcement learning with verifiable rewards) using deterministic verifiers and rloo, allowing the model to reinforce correct reasoning without relying on human preference labels.
The future won’t be about money.
It will be about something much worse.
Mobile phones were an extreme luxury back in the day. Now almost everyone has one.
That is true for an incredible number of products.
Technology makes products cheaper and more abundant over time.
Automation temporarily killed jobs in the past, but people adapted and found new ones.
Now it is a bit different.
Robots will eventually be able to perform many jobs without human intervention, and maybe one day almost all of them.
The real question is when.
People were also extremely optimistic during the Industrial Revolution.
Machines were supposed to make everyone richer and life easier.
Instead, factory owners became incredibly powerful, workers were exploited and inequality became worse before society finally adapted.
Maybe this is a repeating pattern.
Technology creates abundance, but ownership becomes more concentrated.
This is what makes Alien: Earth so interesting.
In its future, Earth is no longer really governed by countries.
It is governed by five corporations:
Prodigy
Weyland-Yutani
Lynch
Dynamic
Threshold
They do not just own products.
They own territories, technology, synthetic bodies, intelligence, space infrastructure and access to new forms of life.
Prodigy is trying to transfer human consciousness into synthetic bodies and create something beyond both humans and machines.
Weyland-Yutani is searching space for biological resources and alien intelligence that could give it an advantage over every competitor.
The Alien itself is not even the most frightening part of that world.
The ownership structure is.
The future may not be controlled by whoever has the most money.
Money is only an accounting system.
Real power will belong to whoever owns the physical resources, energy, compute, robotics, intelligence and infrastructure that everyone else depends on.
The rich and corrupt who own the most will use those resources to acquire even more.
Eventually, corporations may become more powerful than countries.
They could control governments, populations, planets and perhaps one day entire parts of space.
Intelligence and ownership will become the next great monopolies.
Whoever enters that era with the most resources will have the greatest chance of owning everything that follows.
The gap between rich and poor may become wider than ever.
And because physical resources will remain scarce, governments may care more about protecting strategic infrastructure and its powerful owners than helping people who no longer have much economic value in the system.
