Blu3322

I’ve been playing around with EvoMap’s AutoResearch, trying to see if I could get it to handle some research work for me. was looking through the repo while it was running and noticed karpathy in the contributors section. I actually stopped for a second because I wasn’t expecting to see his name there. So, I started looking more closely at what AutoResearch is doing. there’s a paper behind it, “Insight In, Hallucination Out” (arXiv 2608.17906), from EvoMap’s Infinite Evolution Lab. it’s also sitting at the top of Hugging Face’s trending papers right now. I’ve seen a lot of projects call themselves an “AI scientist,” but most seem to do the same thing, one AI comes up with an idea, runs the test, then checks its own work. AutoResearch takes a different approach. It gets three different AI models to come up with ideas separately, and then they check each other’s work. there’s also a blind review at the end, so the reviewer doesn’t see the earlier reviews. what I found interesting is that this isn’t just something they say they do. the setup checks for it. you assign models to different jobs in the config, like screener, judge, ideator, planner, code reviewer and critic. if two names actually point to the same model, the check fails before you start spending money. there’s even a “freshness refresher” that updates the plan with newer models, datasets and baselines before the experiment starts. so you’re not testing your idea against something that’s already two years old. the part I liked most is that it keeps everything. the plan, code, test results, what went wrong, the numbers, and the reviews all stay in the project folder. there’s also an append-only decisions log and a work queue, so you can follow what happened instead of only seeing the final answer. so, if an experiment fails, you still have the full record. you can stop it, come back later, and continue from where you left off. it also runs a cheap pilot before the real experiment. based on that result, it can decide whether to scale up, change the plan, or stop and keep the failed result. that last part stood out to me because most AI agents are built to keep going until they can show you a win. they even report a Django task from SWE-bench Lite that went from 2/7 on the new feature tests, to 4/7, and eventually 7/7 while keeping all 203 regression tests passing. you can run it through the Claude Code CLI, start with a small test, and see what happens before letting it go further. worth saying though, the alpha entrypoint starts Claude Code with permissions skipped. run it in a throwaway box, not the machine holding your keys. It’s already around 2.2k stars on GitHub. I started using it because I wanted to see what it could actually do. then I saw Karpathy’s name and decided to share the whole thing.