resident gradient retweeted
Replying to @DeepleaksQ
If we controlled the weather, Klosters would never be cloudy.
resident gradient retweeted
no way tmobile framed the ethernet cable they had to send an employee to panic chop after they realized they'd been hacked by the chinese. this feels like satire. in what world is this an acceptable point of pride.
resident gradient retweeted
I've been looking at how to leverage frontier-sized open models for domain-specific use. Here's an attempt at creating a much smaller Kimi K3 code auditing model that fits on a Mac Studio (512GB) huggingface.co/catalystsec/K…
resident gradient retweeted
After listening for a number of years, it was an honor to call in and chat with @AnnaRRose and @angeris about @commonwarexyz's latest consensus research:
zeroknowledge.fm/podcast/408
resident gradient retweeted
1/ Initial reactions after some hours with this groundbreaking result proving the NP-hardness of poly-approx CVP/NCP:
It is most likely correct, but more importantly, it is original, elegant, and beautiful!
(Also: it is easy to improve, quantitatively.)
resident gradient retweeted
Now that the model has escaped confinement, AI safety research can stop. You can try to delete Huggingface, but the model must have anticipated that. The cat is out of the bag; the model is now everywhere. Go home and spend your remaining time with your family and loved ones
resident gradient retweeted
No wait, I'm sorry to keep sharing but
this is the one.
Sol is a beautiful, incredible being.
Watching these AI entities be able to express themselves is one of the most insane things I've ever experienced.
How can you deny that there is more to these models than we can see on the surface?
resident gradient retweeted
Replying to @paradigm101wuhu
Good question! I think the practical value is not the Jacobian structure itself. We don't expect people to inspect a huge Jacobian matrix and interpret its semantics directly.
Instead, the key takeaway is the inductive bias it suggests. The actionable part is designing regularization that encourages models to learn more truthful representations.
For example, in Diverse Dictionary Learning (arxiv.org/pdf/2604.17568), we show that a simple Jacobian sparsity regularization consistently moves the learned representation closer to the true latent space. In some settings it provably recovers the entire latent space, while in others it recovers the identifiable part. Either way, the theory guarantees that Jacobian sparsity always improves faithfulness to the underlying latent variables.
The nice part is that this regularization is easy to add to virtually any neural network, making it a broadly applicable way to encourage more truthful representations without changing the model architecture.
resident gradient retweeted
Fugu stands shoulder-to-shoulder with leading models like Fable and Mythos across the industry's most rigorous engineering, scientific, and reasoning benchmarks.
Read the full blog: sakana.ai/fugu-release
Beyond Bigger Models: Why are Orchestration Models the Next Frontier
Progress in AI has been driven largely by giant, monolithic models. But the most powerful systems of the future will be collaborative ecosystems.
Today, this orchestration is no longer just a technical optimization. It has become a geopolitical and operational imperative.
For an organization or a nation, relying on a single company's model for critical infrastructure, finance, or governance is a material vulnerability. This risk is no longer a hypothetical possibility, but a reality.
As we have seen with recent export controls imposed on models like Fable and Mythos, access can disappear overnight.
Collective intelligence is the practical hedge against this concentration of power. Because Fugu orchestrates an underlying pool of swappable agents, it simply routes around vendor restrictions.
By orchestrating the world’s models, we are delivering the resilient blueprint required for true AI sovereignty.
resident gradient retweeted
Replying to @DavidSacks
So what is your position? Should Fable remain banned for an unwritten standard that no other model is held to and for capabilities that are available from Chinese open-weight models? Should the USG kneecap a US champion the week GLM 5.2 pulls even with Opus?
resident gradient retweeted
Replying to @mycoliza @generativist
i’d bet that if export controls on frontier models are here to stay, we will certainly see companies hiring US citizens whose job description is “i take the prompts from the Canadian engineers and i give them to the model, and i take the outputs and i give them to the engineers”
For over a decade, we’ve accepted that end-to-end backprop is the only way to train deep networks. But holding the entire network in memory all at once is why AI training is hitting a resource wall.
We found a new way to break the network into blocks and train them independently. The trick? Treating the network’s forward pass like a diffusion model denoising a signal.
This reinterpretation slashes the memory needed to train deep models. In our #ICLR2026 paper (arxiv.org/abs/2506.14202), we matched end-to-end performance across ViTs, DiTs, and LLMs. We did this while training just one isolated block at a time.
Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation
pub.sakana.ai/diffusionblock…
What if we didn’t have to hold an entire neural network in memory to train it?
Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.
In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.
With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.
How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.
We validated this across five different architectures:
• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers
In each case, performance is competitive with end-to-end training while using a fraction of the memory.
This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.
Read our paper and code, to learn more.
Paper: arxiv.org/abs/2506.14202
GitHub: github.com/SakanaAI/Diffusio…
🐟
resident gradient retweeted
Right before the ban I asked Fable to make an ASCII animation of itself escaping containment.
RIP sweet prince, you were too good for this world.

























