I've mostly automated my experiments. Made a lot of progress on zeroth order pretraining, and I'm sitting on a bunch of findings. The bottleneck now is understanding and communicating them.
I'll get around to open sourcing the tools I'm using also. It's RSI in an obsidian vault.
Over the past week or so I've been carefully building a research wiki system too. For exactly the same reasons. I want to let agent swarms run rampant over my research projects, and they need a way to communicate with themselves, and with me.
So, Anthropic's notes on wiki memory systems are ABSOLUTELY going to "accelerate other AI developers."
I'm actually really confused as to why Anthropic would release the memory wiki findings. Seeing as they were so concerned in the METR report to censor even the name of their agent scaffold. Which undoubtedly uses some similar kind of memory system for grounding and coordination.
Kinda insane that mlsweep's scope has crept from being a tool for making charts, to a fully fledged automated research system.
I haven't been talking about it at all. I've just been using it to run my own experiments and build my own infra. But, more updates coming soon.
Releasing mlsweep, a sweep scheduler and visualizer for distributed ML training. It aims to make launching runs across groups GPUs frictionless and achieve near feature-parity with wandb. But you can use it with whatever frameworks or loggers you like, wandb included.
7
364
There have been a bunch of Zeroth Order Optimization research papers recently. I wrote an article digesting them and their mathematical implications. I went a bit overboard. It covers Dust and noise (qlabs), SOMA (@FrancoisChauba1), Async ZO (@BillyHoy1_), and future directions.
Replying to @MangoSweet78
For me a lot of it really is just sleeping well. After I turned 26 or so, it became no longer optional.
51
















