Luke Darlow

Introducing PC-ALM, a local-learning alternative to backpropagation. Our method trains 1000-layer neural nets using only local dynamics, and without backprop. Blog: pub.sakana.ai/pc-alm/ Standard deep learning relies on backpropagation. The brain, however, cannot implement backpropagation, at least not exactly. How can a physical system, such as the brain, solve multilayer credit assignment without explicit use of backprop? We look for inspiration in two related fields: distributed optimization and NeuroAI. In NeuroAI, predictive coding asks each neuron activation to solve an energy-based inference problem instead of using a standard forward pass. That inference step can be implemented as energy-minimization dynamics on local prediction errors. This perspective -- each layer as a dynamical system -- has proven promising, but performance of predictive coding hasn't scaled well with depth. Credit signals at far ends of the network struggle to diffuse into internal layers. We turn to distributed optimization, generalizing predictive coding to use an augmented Lagrangian instead of energy. This motivation stems back to a classic 1988 paper by LeCun, showing that the Lagrange multipliers of a deep network can be identified with gradients of a supervised loss. The augmented Lagrangian then bridges LeCun's perspective to the standard predictive coding that is used in NeuroAI. We find that this new perspective yields a natural PC-like alternative to backpropagation, resulting in a method we call PC-ALM. PC-ALM differs from PC in that it introduces dual neurons (Lagrange multipliers) as part of the layer-local dynamics, resulting in each layer acting as a PI feedback control system to minimize local prediction errors. We find that PC-ALM is capable of propagating signals to seemingly arbitrary depth, especially in deep narrow networks where standard PC struggles to learn. Ultimately, our motivation here is to understand how distributed physical systems, such as the brain, can compute credit signals using only local coupling and local dynamics. PC-ALM may also inform deep learning in neuromorphic hardware, where dynamics are cheaper than on GPUs. Paper: arxiv.org/abs/2605.31022 Code: github.com/SakanaAI/pc-alm

We are honored to be featured in the latest @TwoMinutePapers video! You all can watch the full video here: youtu.be/QzZ4VwDHAT4 Here’s a short clip from it:

What happens when you put competing neural networks in a Petri Dish and start changing the rules while they adapt? Last year we released Petri Dish NCA, where neural nets are the organisms that learn during simulation. Today we're releasing Digital Ecosystems: a browser-based platform for interactive artificial life research. The setup: several small CNNs share a 2D grid, each seeing only a 3x3 neighborhood. No global plan. They compete for territory by attacking neighbours and defending against incoming attacks, learning via gradient descent online while the simulation runs. What we didn't expect was the role of the learning itself. Gradient descent isn't just optimising each species' strategy. Instead, it acts to stabilize the whole system during simulation. Species that overextend get pushed back by the loss. Species that stagnate get nudged to grow. This means you can push parameters toward edge-of-chaos regimes: a zone characterised by emergent complexity. Letting the neural networks learn acts to hold the complex system together while you explore and interact. The platform lets you steer all of this interactively. You can draw walls to create niches, erase parts of the system online, and tune 40+ system parameters to explore the most interesting configurations. We find it mesmerizing to watch species carve out territories and reorganise when you perturb them. Everything runs client-side in your browser, no install needed. Blog: pub.sakana.ai/digital-ecosys… Code: github.com/SakanaAI/digital-…

Sakana AI共同創業者 Llion Jones（@YesThisIsLion）のTED AIトークが公開されました。目標を定めすぎないオープンエンドな研究がブレークスルーを生む理由、Transformerの成功が業界にもたらした状況、それを乗り越える次の構想と成果を語りました。 ted.com/talks/llion_jones_ho…

We are thrilled to announce a strategic partnership with Google! Google is also making a financial investment in Sakana AI to strengthen this collaboration. This underscores their recognition of our technical depth and our mission to advance AI in Japan. We are combining Google’s world-class products with our agile R&D to tackle complex challenges. By leveraging models like Gemini and Gemma, we will accelerate our breakthroughs in automated scientific discovery. Our work on The AI Scientist and ALE-Agent has already demonstrated the power of these models. Now we are going further. We are scaling our deployment of reliable AI in mission-critical sectors. We are working with financial institutions and government organizations to deliver solutions that meet the highest standards of security and data sovereignty. We are excited to drive the widespread adoption of reliable AI and advance Japan’s AI ecosystem together!

Introducing DroPE: Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings pub.sakana.ai/DroPE/ We are releasing a new method called DroPE to extend the context length of pretrained LLMs without the massive compute costs usually associated with long-context fine-tuning. The core insight of this work challenges a fundamental assumption in Transformer architecture. We discovered that explicit positional embeddings like RoPE are critical for training convergence but eventually become the primary bottleneck preventing models from generalizing to longer sequences. Our solution is radically simple: We treat positional embeddings as a temporary training scaffold rather than a permanent architectural necessity. Real-world workflows like reviewing massive code diffs or analyzing legal contracts require context windows that break standard pretrained models. While models without positional embeddings (NoPE) generalize better to these unseen lengths, they are notoriously unstable to train from scratch. Here, we achieve the best of both worlds by using embeddings to ensure stability during pretraining and then dropping them to unlock length extrapolation during inference. Our approach unlocks seamless zero-shot context extension without any expensive long-context training. We demonstrated this on a range of off-the-shelf open-source LLMs. In our tests, recalibrating any model with DroPE requires less than 1% of the original pretraining budget, yet it significantly outperforms established methods on challenging benchmarks like LongBench and RULER. We have released the code and the full paper to encourage the community to rethink the role of positional encodings in modern LLMs. Paper: arxiv.org/abs/2512.12167 Code: github.com/SakanaAI/DroPE

Introducing Digital Red Queen (DRQ): Adversarial Program Evolution in Core War with LLMs Blog: sakana.ai/drq Core War is a programming game where self-replicating assembly programs, called warriors, compete for control of a virtual machine. In this dynamic environment, where there is no distinction between code and data, warriors must crash opponents while defending themselves to survive. In this work, we explore how LLMs can drive open-ended adversarial evolution of these programs within Core War. Our approach is inspired by the Red Queen Hypothesis from evolutionary biology: the principle that species must continually adapt and evolve simply to survive against ever-changing competitors. We found that running our DRQ algorithm for longer durations produces warriors that become more generally robust. Most notably, we observed an emergent pressure towards convergent evolution. Independent runs, starting from completely different initial conditions, evolved toward similar general-purpose behaviors—mirroring how distinct species in nature often evolve similar traits to solve the same problems. Simulating these adversarial dynamics in an isolated sandbox offers a glimpse into the future, where deployed LLM systems might eventually compete against one another for computational or physical resources in the real world. This project is a collaboration between MIT and Sakana AI led by @akarshkumar0101 Full Paper (Website): pub.sakana.ai/drq/ Full Paper (arxiv): arxiv.org/abs/2601.03335 Code: github.com/SakanaAI/drq/

Sakana AI will be at #NeurIPS2025! 🐟🐟🐟 We’re pleased to announce that Sakana AI is co-hosting “AI for Science: Algorithms to Atoms” panel event alongside NeurIPS. Join our CEO, @hardmaru, and other industry leaders as we explore the future of AI-driven scientific discovery. • Featured panelists: Yann LeCun (Meta), Bill Dally (NVIDIA), Anima Anandkumar (Caltech), David Ha (Sakana AI), Max Welling (CuspAI) • In collaboration with: NVIDIA, CuspAI, Sakana AI, and Samsung Next. • Dec 5 | 3:30pm PT. If you'll be at NeurIPS San Diego, here is the link to join: luma.com/AI-for-Science_12.5…

New Job Posting! We’re looking to hire experienced Software Engineers to join our R&D team. You will help productionize our advanced AI-driven discovery platform and our model-development efforts. sakana.ai/careers/#software-… Japanese language fluency is NOT required for this role.

【Machine Learning Street Talk 出演のお知らせ】 Sakana AI CTOのLlion Jonesと、Research ScientistのLuke Darlowが、機械学習分野の著名なポッドキャスト@MLStreetTalk に出演いたしました。 🎥動画本編はこちら youtu.be/DtePicx_kFY Transformerの共著者でもあるLlionが、 なぜ今「次のパラダイム」への挑戦が必要だと感じているのか。そして、NeurIPSのSpotlight発表にも選出された「Continuous Thought Machine (CTM)」について、ホストのTimさんと語り合いました。 ［ハイライト］ 1. ポストTransformerと「研究の自由」 かつてTransformerがRNNベースの手法を塗り替えたように、現在の技術もやがて置き換わる時が来るとLlionは考えています。 しかし、今のパラダイムを変えるには、単なる改良ではない「圧倒的な性能差」が必要。強大なTransformerに挑むのは容易ではありませんが、かつての発明がそうであったように、ボトムアップで自由な探索こそが重要であると語っています。 2. 新たなアプローチ「CTM」の挑戦 生物の脳の仕組みと、人工ニューラルネットの長所。そのバランスを取り入れたのが「CTM（Continuous Thought Machine）」です。 これは完全に未知の技術というわけではありませんが、正しい方向への一歩だと二人は考えます。競争の激しい今のAI研究において、誰も取り組んでいない領域だったからこそ、8ヶ月というAI分野においては長い間をかけて研究を深化させることができました。 3. 人間並みの推論能力を目指して 現代のAIは、大量のデータと計算量により目覚ましい発展をとげているものの、まだ人間のような真のリーズニング（推論）能力を持っていないとLlionは考えます。それを示すために考案したのが、変化型数独を集めたベンチマーク「Sudoku Bench」。 GPT-5クラスのモデルであってもこれに苦戦するという事実は、まだAIには大きな発展の余地が残されていることを示唆します。 Sakana AIは、最新技術を価値へ変換する社会実装、市場に求められる重要技術をつくる応用開発とともに、次のAIのイノベーションを目指すオープンエンドな基礎研究にも引き続き果敢に挑んでまいります。 Sudoku Benchの結果 nitter.gq/SakanaAILabs/status/19…