gitgud

We climbed +110 ranks across the Open Kernel Challenge leaderboards since yesterday! I also open sourced some of our best tricks for an easy shortcut to get on the leaderboard :) 1) Prefill Layer: 297,235.47 µs down to 266,195.82 µs 1.303× to 1.455× #276 to #202 (+74 ranks) 2) Attention Decode: 2,090.71 µs down to 1,966.22 µs 1.801× to 1.915× #254 to #218 (+36 ranks) 3) Whole Decoder Layer: 12,674.38 µs down to 12,628.57 µs 1.785× to 1.791× #143 to #172 (−29 ranks) 4) Routed MoE: 6,615.94 µs down to 6,606.29 µs 2.033× to 2.036× #145 to #159 (−14 ranks) 5) Dense BF16 Projections: 3,324.00 µs down to 3,313.53 µs 1.453× to 1.458× #134 to #169 (−35 ranks) We net gained + 32 ranks due to many more contributions, and we're now #172 overall (from #140). Top 50 on the whole decode layer, which qualifies for early access to the final serving leaderboard, now requires about 13% faster kernels than ours. Interesting observations: 1) We found human experts coordinating agents progresses over 2x faster than AI coordinators like Astra alone, because even frontier models make mistakes experts can resolve. Human + AI coordinate works if you can teach the coordinator your best practices and step in to fix mistakes. The most common mistake is they keep forgetting you can combine all the kernel optimizations into the full decode layer, and it's funny to periodically remind them for a huge breakthrough. 2) I updated the Auto-research agent harness with the muse challenge and CUDA best-practice prompts to make it easy to get on the leaderboard: github.com/rohanarun/Auto-re… You don't need a GPU to participate: off-board checks score on their GPUs (20 per day per leaderboard). 3) Many top teams are making progress without GPUs by using their free 30 checks per day on the official leaderboard, and some are even grinding it out with single agents. If you need free GPUs to run experiments, I opened up the queue system I used to get my GPUs to 100% utilizations: app.getsupers.com/experiment… You can run 5 experiments for free on our queue, and also monetize your idle GPU compute by adding your node to the marketplace. This introduces a new primitive for agents called "experiments" which are measured in H200 seconds and more efficient for auto-researchers. Using Experiments is ~90.4% to 92.5% cheaper (over 10× cost savings) than renting an on-demand H200 instance because developers only pay for the exact seconds the kernel executes. Most auto-research cycles waste a lot of GPU time generating the code to test, so this should save money if you don't run a massive fleet. You can use this to monetize “weird” GPU hours or unused minutes, or get cheap/free GPU computer for running experiments. If anyone wants to monetize their idle compute, DM me because this market is exploding and everyone needs to run tiny experiments. Share it with a friend to get $5 in experiments(about 100 experiments) for free for both of you. Test it out and let me know how it goes :) 4) We were able to hillclimb with a fleet of Gemini 3.8 flash agents constantly running experiments and a human coordinator guiding them away from common mistakes. We now run automated A/B tests across our H200 queue to push the Pareto frontier of auto-research, comparing randomized perturbations, intentional breakthroughs, and different prompting strategies. I will report progress and best practices as we learn them. 🫡

We're launching a $100,000 challenge to see who can make Muse Spark run fastest: openkernelchallenge.dev/