Rethinking AI efficiency fundamentally
Berkeley, CA
Joined October 2025
- Tweets17
- Following33
- Followers20
- Likes19
Rethinking AI Lab retweeted
🎯 AI discovers SOTA sparse attention strategies for LLM inference decoding
[ADRS Blog #12] We explore using AI to accelerate LLM inference by reducing memory traffic and latency. Using Cursor and the @skylight_org framework, we evolved a naive baseline into a sophisticated algorithm that matches SOTA sparse attention strategies for LLM decoding!⚡️
✍️ Read the Blog: adrs-ucb.notion.site/skyligh…
🌐 SkyLight Website: sky-light.eecs.berkeley.edu/…
📖 ADRS Blog Series: ucbskyadrs.github.io/
📄 ADRS Paper: arxiv.org/abs/2510.06189
👩💻 Code: github.com/UCB-ADRS/ADRS
Rethinking AI Lab retweeted
We’ve been assuming that extreme attention sparsity must destroy model quality.
That assumption is now false.
New SkyLight Release: vAttention ⚡️
We added vAttention to the Tier1A leaderboard. The data is striking:
👑 #1 Practical Method: Surpasses the previous lead (PQCache), closing the gap to dense models within 1% at up to 10x sparsity..
👑 Saturates Benchmarks: Dominates the sparsity–quality frontier (w/ oracle top-k).
💡 Introducing Verified-X: A new paradigm for reliable inference-time sparsity.
In this week’s blog, we share an in-depth look at the algorithm and how to run it via SkyLight 👇
Rethinking AI Lab retweeted
Selecting the top-k highest scores is the intuitive standard for sparse attention. However, our latest SkyLight Tier 1A benchmarks show a non-trivial gap between Oracle Top-k and dense baselines at high sparsity.
PQCache currently tops the leaderboard as the strongest practical method, closing the gap between approximate-top-k and oracle-top-k for upto 10x sparsity levels.
Closing the gap between dense and practical sparse attention methods will clearly require new ideas. In this week’s blog, PQCache authors share an in-depth look at the algorithm, and how to run it via SkyLight 👇
Rethinking AI Lab retweeted
There has been significant recent work in sparse attention promising to unlock long context and improved efficiency -- but does it work?
We recently launched a benchmarking effort to evaluate sparse attention methods and help practitioners understand the capabilities and limitations of SoTA methods.
We are also working to make these methods more accessible in open platforms like vLLM.
Sparse attention offers a viable path to solving the computational and memory issues of long-context LLM serving. However, we observe a surprising gap between the rapid pace of academic research and the reality of open-source adoption.
We identify the core barrier as fragmentation where new methods are published rapidly, but with missing comparative grounding.
Today we’re excited to share SkyLight, where we are building a unified framework designed to bridge this gap. SkyLight standardizes the implementation and evaluation of sparse attention to enable rigorous, controlled benchmarking.
Our first major effort is to understand the frontier of inference time sparse-decoding in LLMs.
1/N 🧵
Rethinking AI Lab retweeted
🎯 AI agents generate production-ready GPU kernels that outperform compiled models
[ADRS Blog #6] This week, we feature work from @datadoghq on BitsEvolve: an ADRS framework for systems optimization. Using evolutionary search, we show how AI can automate the generation of custom, high-performance GPU kernels!
✍️ Read the blog: adrs-ucb.notion.site/datadog
🚀 Previous BitsEvolve Post: datadoghq.com/blog/engineeri…
📄 ADRS Paper: arxiv.org/abs/2510.06189
👩💻 Code: github.com/UCB-ADRS/ADRS
💬Join the community: join.slack.com/t/adrs-global…
Rethinking AI Lab retweeted
🚀 AI optimizes tensor kernels to run 17x faster than human expert designs!
[ADRS Blog] Programming hardware accelerators is notoriously hard. We describe Autocomp, the first LLM-driven optimizer for tensor accelerators, which outperforms hand-tuned expert kernels on AWS Trainium by up to 17x!
✍️ Read the blog: adrs-ucb.notion.site/autocom…
📖 ADRS Blog Series: ucbskyadrs.github.io
📃 Autocomp Paper: arxiv.org/pdf/2505.18574
👩💻 Code: github.com/ucb-bar/autocomp
Rethinking AI Lab retweeted
Building agentic systems is fun. debugging them is hell. 📉
Most failures aren’t due to weak models but due to brittle workflows.
MAST pinpoints why agents fail in an automated way, and helps you get 53% higher accuracy in your agentic system without changing the LLM. 🧵👇
Rethinking AI Lab retweeted
🚀 End the GPU Cost Crisis Today!!!
Headache with LLMs lock a whole GPU but leave capacity idle? Frustrated by your cluster's low utilization?
We launch kvcached, the first library for elastic GPU sharing across LLMs.
🔗 github.com/ovg-project/kvcac…
🧵👇 Why it matters:
Rethinking AI Lab retweeted
Also, our blog is available on SIGOPS!
Barbarians at The Gate: How AI is Upending Systems Research by @audreyccheng, @LynnLiu41887950, @melissapan, @istoica05, and the @ai4research_ucb team,
sigops.org/2025/barbarians-a…
This's first article of the The Next Horizon of System Intelligence blog series.
