Pinned Tweet
🍰 CAKE paper's out, the design bet: the compiler isn't a fixed black box the agent calls — it's part of the harness, and it's under evolution too.
CAKE didn't inherit existing abstraction layer. no tile/layout abstractions: the vocabulary was distilled by agents from a corpus of production kernels. every pattern the agent couldn't express pushed new primitives into the IR, and the analyses to keep them checkable. every barrier/layout bug that kept coming back became a verifier rule.
none of this can be designed up front. the IR has to co-evolve with the kernels, and the workload tells you what's missing, the corpus tells you if the fix broke anything.
the best language for an agent is the one that tells you what's illegal, what's slow, and which decision might made it faster.
arxiv.org/abs/2608.12629
🍰 another slice of CAKE just dropped: blackwell KDA kernels, ~2.05x geomean speedup vs FlashKDA:
github.com/flashinfer-ai/fla… by @avyyhuang
more cake kernels are baking 👀 not saying what CAKE actually is yet though, the full reveal is in the oven too.
Zihao Ye retweeted
Judgement Engineering for Agentic Coding
Running Claude Code or Codex on a long task for hours in, the agent says "all done," and then you find it is actually not ... That's why we built Humanize!
The idea is simple: two models, one builds and one judges.
1. A human signs off on a plan
2. Claude Code builds, round by round.
Using different models is important. A model tends to miss the same mistakes even the memory / context is reset. Pair it with a reviewer whose blind spots are different, and a bug only gets through if both miss it. (just like dicussing with labmates/workmates!)
What we've built with it so far:
🔧A 567-file gem5 build-system migration (PR under upstream review)
⚡Kernel Design Agents: top 3 in all three Full-Agent tracks at the MLSys 2026 FlashInfer contest
🧮 672/672 on PutnamBench, #1 on Lean-Eval (251/303)
🏅HOA: full scores on IOI, IMO, IPhO 2026 and IBO 2024; 418.5/437 on IChO 2026 (gold)
📄 Paper: arxiv.org/abs/2610.08900
💻 Humanize: github.com/PolyArch/humanize
🚀 Humanize2 (standalone runtime, in development): github.com/humanfia/humanize
By Sihao Liu, Ligeng Zhu, Zijian Zhang, Dongyun Zou, Zhengyang Zhang, Changye Li, Song Bian, Song Han, Tony Nowatzki
#humanize #humanfia #agents #claudecode #codex
Zihao Ye retweeted
🚀 Excited to introduce StreamRender-H3 — bringing real-time interactive worlds to life with diffusion models!
Built on Sol-Engine, we post-trained @MiniMax_AI H3 Ref2VA into a real-time streaming neural renderer.
🎮 Code drives the world. A game engine handles physics, object states, and player interactions, while H3 generates the visuals.
⚡ Real-time streaming: 2 denoising steps, 24 FPS, and ~273 ms per chunk on 8×GB200 GPUs.
🎨 One world, endless appearances: Change the reference image and prompt to render the same interactive world in entirely different styles.
🔥 Powered by streaming post-training, distillation, a fast causal H3 VAE, and full-stack inference acceleration.
Check out our demo and code! 👇
demo: nvlabs.github.io/Sana/Sol-En…
code: github.com/NVlabs/Sana/tree/…
Zihao Ye retweeted
As hardware × model combinations explode, bespoke serving systems are becoming the only viable path forward.
The pace of change makes it nearly impossible for incumbent, general-purpose engines to keep up.
🚀Agent-built systems for the win!
Zihao Ye retweeted
Vera Rubin is lots of fun! Thank god it still uses tcgen05 tho
vLLM now supports NVIDIA Vera Rubin. The early results show more than 7.8x the throughput of GB200 on MiniMax M3 on AgentX.
@inferact, @NVIDIAAI, @RedHat_AI, and the vLLM community have been bringing vLLM up on Rubin since it was announced. Here is where things stand.
🧵 1/5
Zihao Ye retweeted
New post: Is infra ready for recursive self-improvement?
An RSI loop runs at the speed of its verifier. Most of the discussion is about the proposer. I think the loop will stall at verification, and our infra was built for a world where verification is cheap.
Zihao Ye retweeted
Optimizing kernels can be time-consuming and require deep expert knowledge. KernelAgent is a multi-agent harness that can write and optimize kernels for you.
At #PyTorchCon North America 2026, @KaimingCheng (@Meta) will present “KernelAgent: Hardware-Guided GPU Kernel Optimization via Multi-Agent Orchestration.”
The session will cover a hardware-guided optimization layer that integrates GPU hardware-performance signals into a closed-loop multi-agent workflow to guide optimization for Triton kernels. Across all 100 KernelBench L1 tasks evaluated, KernelAgent achieved a 2.02x speedup over kernels generated by earlier versions and an average 1.56x speedup compared with default torch.compile.
Register for PyTorch Conference North America 2026: hubs.la/Q04v4SL60
Zihao Ye retweeted
using torch.compile, especially inductor, in RL is really risky, on both the trainer and the generator side (including vLLM’s torch.compile path). it can silently break train–inference consistency, and your RL run pays the price.
Zihao Ye retweeted
To get much faster NCCL comms at small-to-medium payloads, start using symmetric memory, which has been added recently to NCCL and PyTorch.
If you struggle to overlap comms and compute this could be your salvation in a few lines of code!
Details github.com/stas00/ml-enginee…
Zihao Ye retweeted
Girish is a really nice guy and I work with him quite often at NVIDIA, but I totally disagree with his opinion in this talk.
Compilers: The Central Nervous System of the AI Era
youtube.com/live/jX6QHjqU6-Y…
Girish Bharambe
Innovations in Compiler Technology (IICT) 2026
Zihao Ye retweeted
After ICLR, we finally got some time to top first again on lean-eval leaderboard with updated Humanize workflows 🤗
> LeanEval is the last unsaturated benchmark in mathematical formalization.
> New Humanize flow learnt a lot of insights from OpenAI and Anthropic's blogs, and Axiom's tech report.
> More progress on "big name theorem" is coming :)
xitter.cf/IlinVasily29521/status…
LeanEval is the last unsaturated benchmark in mathematical formalization. Saturating it will require formalizing Fermat's Last Theorem. @axiommathai is currently in the lead.
Leaderboard: lean-lang.org/eval/legacy/
Zihao Ye retweeted
🚀 Kernel Design Agents to Optimize Kimi Delta Attention (upto 2.96x)
When the KDA (Kernel Design Agents) project was initiated, its name happened to coincide with another KDA (Kimi Delta Attention). More than once, people asked us , "Can we use KDA to write a version of KDA?" Today, right at a moment with a full moon good for shooting, we are thrilled to share the latest results of KDA(gent)-v0.6: using KDA(gent) to optimize KDA(ttn)!
KDA-v0.6 achieves this through
* Humanize2 Flow & LLM Collaboration Optimization: Introducing multi-round iterative evolution with Flame Chase, GPT-5.6-Sol, and Fable-5, paired with an automated Workspace cleanup mechanism to significantly broaden the operator search space.
* Multilingual Domain-Specific Skill Stack Support: Fully supporting CuTe-DSL, CUDA C++, as well as recent Agent-Native CAKE IR and TIRx. Combined with customized diagnostic tools (such as CuTe's dedicated IKET pipeline analyzer and TIRx's CPU-side numerical simulation and static checks), achieving deep performance bottleneck diagnostics.
* Self-Evolving Kernel Wiki: Building a dynamically updated knowledge base that automatically cleans up invalid/erroneous implementations, corrects classification tags, and streamlines search indexing, significantly improving the Agent's knowledge retrieval quality and code generation accuracy.
KDAgent ultimately optimized the KDAttn kernel to 2.96x of FlashKDA, and we face a lot of unexpected hacks during the optimization. More details shared in our blogs nvlabs.github.io/kda/blog/20…
Release results of KDA for KDA: github.com/NVlabs/kda/tree/2…
Community & Production Deployment:
All kernels are open-sourced for community verification! For deploying NVIDIA Agentic CUDA acceleration in production environments, we recommend using the production-level version rigorously verified by FlashInfer.
By the Team: Dongyun Zou, Yixin Dong, Hongyi Jin, Junxian Guo, Yahui Cui, Avery Huang, Zihao Ye, Junru Shao, Changye Li, Zijian Zhang, Sihao Liu, Song Bian, Ligeng Zhu.
Zihao Ye retweeted
🔥 Optimizing LLM serving requires repeated experiments, and running them on real hardware is slow and costly.
Performance depends on how models, kernels, hardware, and serving policies interact. As models and workloads change, engineers must revisit their configuration choices.
🚀 Introducing ServingStudio: an integrated workbench for simulating, analyzing, and optimizing LLM serving systems.
🧵 More in the thread below. (1/9)
Zihao Ye retweeted
We tested Jev as a code reranker on 100 GitHub issues. Model-planned grep + Jev was our best route, though the choice of retrieval route mattered more than the choice of reranker.
The benchmark is CodeNib Base: 25 repos across C/C++, Go, Python, Rust and TS/JS. The query is the full issue text, and each repo is checked out before the fix. We built candidate pools four ways (BM25, dense embeddings, BM25+dense hybrid, and grep queries planned by a single Sonnet 4.6 call), capped each at 100, and reranked every pool with both Jev and Qwen3-Reranker-4B.
Jev answers typed questions, so we asked it one relevance score per candidate and sorted the scores ourselves.
Code-block Recall@5:
grep → Jev: 71.4%
hybrid → Qwen 4B: 65.0%
dense → Qwen 4B: 63.4%
Moving from BM25 to grep candidates raised Recall@5 by 10 to 14 points for every reranker we tried. On the same pool, Jev and Qwen 4B were within noise of each other. Jev had the higher point estimate on the grep pool (+3.3), but the interval crosses zero.
Grep supplied 17.5 candidates on average, so Jev's scoring took 0.48 s p50 and cost $0.045 across all 100 issues. The planning call is the expensive part: 4.0 s p50 and about $1.00 per 100 issues. If that's too slow, hybrid → Jev reached 67.9% at 3.4 s and $0.26 per 100 issues.
Skipping embeddings doesn't buy latency by itself, since warm dense lookup took 18 ms. This is one run per configuration on 100 issues, so treat the intervals as exploratory.
All the numbers and caveats are in the thread.
Zihao Ye retweeted
nand2mario.github.io/posts/2…
This is a fantastic article on building a Voodoo GPU recreation, a lot of the architectural challenges, including having high latency DDR. Excellent read for those interested in GPU architecture :)
Zihao Ye retweeted
4 months after its initial release, "auto-gpu-kernel" version 1.0 is finally out!
It is a fully autonomous kernel generation "meta-harness" that evolves both the kernel and the harness layer, generating speed-of-light kernels 🧵
github.com/Dogacel/auto-gpu-…
Zihao Ye retweeted
The Decompilation Book decompilation.education/
Zihao Ye retweeted
Have you ever wondered what a data race is? How a race detector works? If there are bugs in your race detector? I wrote an article.
theconsensus.dev/p/2026/09/0…
