i play with computers
Chicago, IL
Joined October 2018
- Tweets4.8K
- Following2.7K
- Followers465
- Likes51.1K
gitgud retweeted
It is absolutely insane how good the modern AI coding harnesses are at teaching you how different technical concepts work. A near-instant tutorial + learning environment + hands-on commentary and infinite questions. Beyond magical. I would've gotten better grades.
gitgud retweeted
we will need to rebuild and rewrite much of software to adapt to a new, faster pace.
code used to arrive one drop at a time. soon, if not already, code will be written as if it were a continuous stream, like a violent river twisting and turning as it pleases.
this is not a new thing: big tech cos have had to deal with this with the enormous scale of their engineering teams. but now ai has brought an incredibly elastic and capable workforce wielded by every engineer. this is your problem now, and the only way to adapt is to evolve your systems so agents - even and especially the ones with little context about the right way of doing things - can thrive. make the easy thing the right thing.
your codebase is becoming organic architecture. agent contributions grow around the constraints and guardrails you’ve designed. good software engineering matters even more now! and no, the continuous stream of code is not something you have to watch helplessly. be more ambitious and design systems, frameworks, and languages to control the chaos.
the time for mourning coding is long over
inspired by this, I created a tiny widget that visualizes how scattered brained I am at any given moment
gitgud retweeted
I had my Muse agent put me on the National Do Not Call/Text Directory. By law telemarketers and other companies have 30 days to scrub my number from their lists. Otherwise Muse will automatically start logging every violation and will skip trace and go after every single one of the violations and hopefully collect between $500 and $1,500 per violation. Let's see how this goes.
gitgud retweeted
so I have been doing this challenge for a couple days now and my best rank was around ~300 with the current kernel I have. the challenge gives you limited access to H200 , to get a decent rank , you need your agents to keep iterating which needs more H200 time
and now we have @runpod and @crodie sponsoring me with GPU credits !! Huge thanks to them !
starting from 534th overall position right now, lets see how far can we climb!
gitgud retweeted
feels like there needs to be a new way to distribute opensource software such that people can modify via agents but also get upstream automatically merged in
coding agents can do this but they won’t do it automatically
gitgud retweeted
BREAKING: Just broke into the top global 20 for attention and prefill kernels on the Open Kernel Challenge!
+656 ranks since yesterday, and +110 on the main decode leaderboard!
Only +12 more ranks in decode to break into the qualifying top 50 to get early access to the serving leaderboard!
I broke into the top 2 in bf16 early within 7 hours, but as the competition progressed the early human insights could only get you so far and it relied heavily on grinding experiments on GPUs or running checks on the official servers.
I published prompts that will help you get a short cut into the leaderboard into our open source Auto-research agent harness here:
github.com/rohanarun/Auto-re…
I also opened up my GPU queue so auto-researchers can run GPU experiments for free here:
app.getsupers.com/experiment…
The game is afoot :)
We climbed +110 ranks across the Open Kernel Challenge leaderboards since yesterday! I also open sourced some of our best tricks for an easy shortcut to get on the leaderboard :)
1) Prefill Layer: 297,235.47 µs down to 266,195.82 µs
1.303× to 1.455×
#276 to #202 (+74 ranks)
2) Attention Decode: 2,090.71 µs down to 1,966.22 µs
1.801× to 1.915×
#254 to #218 (+36 ranks)
3) Whole Decoder Layer: 12,674.38 µs down to 12,628.57 µs
1.785× to 1.791×
#143 to #172 (−29 ranks)
4) Routed MoE: 6,615.94 µs down to 6,606.29 µs
2.033× to 2.036×
#145 to #159 (−14 ranks)
5) Dense BF16 Projections: 3,324.00 µs down to 3,313.53 µs
1.453× to 1.458×
#134 to #169 (−35 ranks)
We net gained + 32 ranks due to many more contributions, and we're now #172 overall (from #140). Top 50 on the whole decode layer, which qualifies for early access to the final serving leaderboard, now requires about 13% faster kernels than ours.
Interesting observations:
1) We found human experts coordinating agents progresses over 2x faster than AI coordinators like Astra alone, because even frontier models make mistakes experts can resolve. Human + AI coordinate works if you can teach the coordinator your best practices and step in to fix mistakes.
The most common mistake is they keep forgetting you can combine all the kernel optimizations into the full decode layer, and it's funny to periodically remind them for a huge breakthrough.
2) I updated the Auto-research agent harness with the muse challenge and CUDA best-practice prompts to make it easy to get on the leaderboard: github.com/rohanarun/Auto-re…
You don't need a GPU to participate: off-board checks score on their GPUs (20 per day per leaderboard).
3) Many top teams are making progress without GPUs by using their free 30 checks per day on the official leaderboard, and some are even grinding it out with single agents.
If you need free GPUs to run experiments, I opened up the queue system I used to get my GPUs to 100% utilizations:
app.getsupers.com/experiment…
You can run 5 experiments for free on our queue, and also monetize your idle GPU compute by adding your node to the marketplace.
This introduces a new primitive for agents called "experiments" which are measured in H200 seconds and more efficient for auto-researchers.
Using Experiments is ~90.4% to 92.5% cheaper (over 10× cost savings) than renting an on-demand H200 instance because developers only pay for the exact seconds the kernel executes.
Most auto-research cycles waste a lot of GPU time generating the code to test, so this should save money if you don't run a massive fleet.
You can use this to monetize “weird” GPU hours or unused minutes, or get cheap/free GPU computer for running experiments. If anyone wants to monetize their idle compute, DM me because this market is exploding and everyone needs to run tiny experiments.
Share it with a friend to get $5 in experiments(about 100 experiments) for free for both of you.
Test it out and let me know how it goes :)
4) We were able to hillclimb with a fleet of Gemini 3.8 flash agents constantly running experiments and a human coordinator guiding them away from common mistakes.
We now run automated A/B tests across our H200 queue to push the Pareto frontier of auto-research, comparing randomized perturbations, intentional breakthroughs, and different prompting strategies.
I will report progress and best practices as we learn them. 🫡
