Releasing mlsweep, a sweep scheduler and visualizer for distributed ML training. It aims to make launching runs across groups GPUs frictionless and achieve near feature-parity with wandb. But you can use it with whatever frameworks or loggers you like, wandb included.
apaz retweeted
Eva is a student at @PrincetonNeuro working with me, @brody_lab and Ben Eysenbach. She was abducted by ICE while trying to board a flight from SFN. Some thoughts below how we can help her.
I've mostly automated my experiments. Made a lot of progress on zeroth order pretraining, and I'm sitting on a bunch of findings. The bottleneck now is understanding and communicating them.
I'll get around to open sourcing the tools I'm using also. It's RSI in an obsidian vault.
Wow. That's a round number. They've just committed to a stable ABI.
huggingface.co/blog/ggml-org…
llama.cpp v1.0.0
github.com/ggml-org/llama.cp…
1
11
485
I'm actually really confused as to why Anthropic would release the memory wiki findings. Seeing as they were so concerned in the METR report to censor even the name of their agent scaffold. Which undoubtedly uses some similar kind of memory system for grounding and coordination.
Over the past week or so I've been carefully building a research wiki system too. For exactly the same reasons. I want to let agent swarms run rampant over my research projects, and they need a way to communicate with themselves, and with me.
So, Anthropic's notes on wiki memory systems are ABSOLUTELY going to "accelerate other AI developers."
Sharon was one of the few people at AMD I interacted with who gave me hope for the company. This is a great loss for AMD, more than anyone realizes.
I've joined Anthropic! I'm excited to get back to (frontier) research and work on the models that I fell in love with a decade ago.
I’m very grateful for my time at @AMD ♥️
Starting with the people: I'm honored to have had the opportunity to work with the one and only @LisaSu! She's an amazing leader, manager, and friend who allowed me so much agency in my work -- they say don't meet your heroes, but she definitely lives up to the pedestal that I've put her on. And thank you to my team: dedicated, resilient folks who've been brave enough to jump on a rocket ship together, in some cases several times to ever ambitious projects. For those who came with me from my startup Lamini - thank you also for bringing your passion, wonder, kindness, and (yes) silliness with me, as we found a home at AMD.
And now the work: I built and ran an AI research org that led our frontier lab partnerships, where I served as executive sponsor. We were among the first in the AI research world to take RL kernel optimization really seriously last year, and it showed me that pushing a frontier capability forward takes far fewer people than you’d think (100x, and I'm now updating my priors to 1000x fewer, than just a year ago). That work has also had massive implications for the compute that labs and companies can now use out-of-the-box with models like Claude. That is to say, it’s always time to make some GPUs go brrr.
29
2,235
Kinda insane that mlsweep's scope has crept from being a tool for making charts, to a fully fledged automated research system.
I haven't been talking about it at all. I've just been using it to run my own experiments and build my own infra. But, more updates coming soon.
Releasing mlsweep, a sweep scheduler and visualizer for distributed ML training. It aims to make launching runs across groups GPUs frictionless and achieve near feature-parity with wandb. But you can use it with whatever frameworks or loggers you like, wandb included.
7
364
There have been a bunch of Zeroth Order Optimization research papers recently. I wrote an article digesting them and their mathematical implications. I went a bit overboard. It covers Dust and noise (qlabs), SOMA (@FrancoisChauba1), Async ZO (@BillyHoy1_), and future directions.
Reach out to me if you want to talk about zeroth order optimization, or any of the papers cited. It's too big a topic for any individual to tackle alone, and there is strength in community and openness.
Blog post: apaz.dev/blog/ZO_Research_Up…
1
3
182
it is honestly quite baffling to me that openai and anthropic still use slack. i expect this will change within the next six months. they could quite clearly build and maintain internal versions that are *more* secure, robust, and featureful with a team of ~10
Why does this take two years to build what am I missing, I feel like I could do this in a couple weeks.
We're announcing the public beta of datavendor.ai
We spent the last two years building the best solution for people buying and selling data, whether you're a researcher in a lab or a company trying to sell data.
Data is the fundamental bottleneck for AI progress, and DataVendor will be the infra that solves it.
5
43
8,590





















