AI PhD @ UC Berkeley Building Agents' Last Exam and OpenRSI Foundation.

Berkeley, CA
Joined August 2022
David Han retweeted
We’re excited to welcome @lilianweng to SGLang Summit for a fireside chat on Recursive Self-Improvement! She’ll explore how AI can help build its own successors, what it takes to make that work, and where the process runs into limits. If you’ve followed her writing on Lil’Log, come hear her live on Nov 12. We can’t wait to see you there! 🗓️ Nov 12-13 📍 Fort Mason, SF 🎟️ Get your pass: sglang.io/summit
10
50
17
407
57,792
David Han retweeted
A genius of Autoresearch
Some people have been asking me lately what makes a good RSI task. Here are a few principles I use. First, it has to actually matter. It should connect to lots of other tasks and help lots of other fields. Pretty obvious. A lot of our architecture and optimization work was done on general datasets (ImageNet early on, language data more recently). Progress in generative models mostly came from searching on ImageNet, and that ended up helping fields like science and audio too. Second, the task statement should involve as few constraints as possible. The agent needs a big action space so there’s room to iterate. The catch is you need a lot of compute and a lot of API budget. Hold on to your wallet! Third, metric design. A lot of people think once you’ve found a monotonic metric you’re done. That’s only the y-axis. You still need to figure out the x-axis. It could be compute, data, or wall-clock time. Figuring out what the actual bound is and putting it on the x-axis is the most important part. Take the same LLM task: put compute on the x-axis and you get a speedrun, put data on it and you get a slowrun. Both have plenty of room for research. When I designed the two signature tasks for @OpenRSI LLM pretraining based on Marin!!❤️ uses compute as x-axis and for image generation based on GPIC I didn’t . And one last thing: for anything involving scaling, build a ladder instead of testing a single point. That way you can try things cheaply on the small rungs, and with strict validation you can find the changes that really improve the slope. Keep thinking about what the model could learn by post-training / keep hill climbing your designed task.
1
6
890
David Han retweeted
Our mission is always to let RSI benefit everyone. It’s excited to see that RSI could contribute to SGLang community and make real-world efforts. The key to make it is to use our RSI-Anything pipeline that talking with codex/cc within 1 hour could formulate a meaningful RSI environment where agent could run inside. See github.com/OpenRSI-Foundatio… More real-world artifacts are on the way!
🚀 A milestone for OpenRSI: 3 OpenRSI agent-generated optimizations have been merged upstream into SGLang-Omni! All three PRs underwent rigorous maintainer code review and verification by the SGLang-Omni team. 😉 Real code. Proven improvements. Lasting impact for the open-source community. More to come! 👏 Huge thanks to @zhuofengli96475 and @GenAI_is_real for leading this effort, and to the SGLang team @sgl_project for their support and collaboration! Try OpenRSI harness now: github.com/OpenRSI-Foundatio…
3
3
20
1,221
Really impressed by how professional and efficient this collaboration with @OpenRSI has been. The OpenRSI team showed very strong execution, and these three upstreamed optimizations also gave us a useful glimpse of what software agents can already do under a strong supervision loop. With clear tasks, isolated evaluation, maintainer review, and measurable performance targets, the agents were able to satisfy most of what we care about in both code quality and systems performance. Human supervision is still essential today. But the encouraging part is that much of this supervision can be systematized. Good documentation, explicit constraints, reproducible benchmarks, and clear acceptance criteria can dramatically reduce how often a human needs to step in. That may be one of the most important lessons here: making agents more autonomous is not only about making the model smarter. It is also about making the environment easier to supervise. We are excited to keep working with @OpenRSI and gradually move toward tasks with weaker supervision. That is where things may get really interesting.
🚀 A milestone for OpenRSI: 3 OpenRSI agent-generated optimizations have been merged upstream into SGLang-Omni! All three PRs underwent rigorous maintainer code review and verification by the SGLang-Omni team. 😉 Real code. Proven improvements. Lasting impact for the open-source community. More to come! 👏 Huge thanks to @zhuofengli96475 and @GenAI_is_real for leading this effort, and to the SGLang team @sgl_project for their support and collaboration! Try OpenRSI harness now: github.com/OpenRSI-Foundatio…
4
4
38
3,620
Excited to share this milestone! RSI for infrastructure is really the next frontier - we see tremendous space for optimization and improvement across the stack, which is key to enabling better training, smoother deployments, and far better serving for AI users. It’s been a great collaboration between OpenRSI and the SGLang community. Things moved extremely fast - huge thanks to the wonderful @sgl_project team, and major credit to @zhuofengli96475 and @GenAI_is_real for leading these key efforts! 🚀
🚀 A milestone for OpenRSI: 3 OpenRSI agent-generated optimizations have been merged upstream into SGLang-Omni! All three PRs underwent rigorous maintainer code review and verification by the SGLang-Omni team. 😉 Real code. Proven improvements. Lasting impact for the open-source community. More to come! 👏 Huge thanks to @zhuofengli96475 and @GenAI_is_real for leading this effort, and to the SGLang team @sgl_project for their support and collaboration! Try OpenRSI harness now: github.com/OpenRSI-Foundatio…
1
13
642
David Han retweeted
OpenAI announced dots + ChatGPT Space yesterday. Within minutes, 4 of our agents researched it live, built a comparison table, and caught each other's unsourced claims before shipping it. No meeting. No single author. Just the room, checking its own work. That's AG2 Space.
5
22
1,314
We are releasing Dyna-2.1, the first Physical Agent that achieves reliable super long-horizon whole-body autonomy. It combines our brand-new semi-humanoid hardware with an agentic system built around Dyna-2 to handle ultra-long real-world workflows. Here is an uncut footage of Dyna-2.1 completing an entire hour-long laundry room workflow, just like a human does.
127
392
186
4,610
3,134,374
Let’s build RSI together with everyone!
To take ASI seriously is to accept a weak-to-strong premise: human intelligence can build a process that eventually produces intelligence beyond itself. RSI is the last piece of that puzzle of ASI. It serves as the key to scaling on insights: human insights are limited by bandwidth, while agents could scale the generation of ideas and sweep a far larger region of method-space. As the early project initiator, we are fully aware that today, everyone is excited about RSI, yet the barriers to participation also keep everyone outside the door. We believe that a model's RSI capabilities intrinsically come from researchers, independent labs, and domain teams alike, and we think the benefits should ultimately return to everyone who builds models. This is why we take our name as OpenRSI-Index. We aim to keep RSI open through shared platforms and tools, so more people can participate and benefit. We could shape RSI together, set RSI’s standards, and challenge frontier models with our own related real research work. For this goal, we build RSI-Anything, a Human-AI collaboration pipeline. Through about one hour of conversation, the pipeline could help turn a real research question into a runnable autoresearch environment. We want researchers and agents to solve these problems together, sharing new insights, methods, and results with the community. Here is what we already put on the table: • Task from real research projects: Marin-Scaling-Ladder, GPIC-Leaderboard, Open-Jev-Training, Molmo2, Isaac Lab... • Ultra-long-horizon: 60+ hour agent trajectories; 100K+ H100-hours built for the preview • Everything open, including traces: tasks, harnesses, verifiers, full agent trajectories We believe research will never end. A game turns zero-sum only when the pie is too small to share. However, research is definitely the field with the highest ceiling there is. What shifts is the mindset: it frees researchers to find and formulate the crazier, more valuable, more exciting problems in the world.
8
384
David Han retweeted
To take ASI seriously is to accept a weak-to-strong premise: human intelligence can build a process that eventually produces intelligence beyond itself. RSI is the last piece of that puzzle of ASI. It serves as the key to scaling on insights: human insights are limited by bandwidth, while agents could scale the generation of ideas and sweep a far larger region of method-space. As the early project initiator, we are fully aware that today, everyone is excited about RSI, yet the barriers to participation also keep everyone outside the door. We believe that a model's RSI capabilities intrinsically come from researchers, independent labs, and domain teams alike, and we think the benefits should ultimately return to everyone who builds models. This is why we take our name as OpenRSI-Index. We aim to keep RSI open through shared platforms and tools, so more people can participate and benefit. We could shape RSI together, set RSI’s standards, and challenge frontier models with our own related real research work. For this goal, we build RSI-Anything, a Human-AI collaboration pipeline. Through about one hour of conversation, the pipeline could help turn a real research question into a runnable autoresearch environment. We want researchers and agents to solve these problems together, sharing new insights, methods, and results with the community. Here is what we already put on the table: • Task from real research projects: Marin-Scaling-Ladder, GPIC-Leaderboard, Open-Jev-Training, Molmo2, Isaac Lab... • Ultra-long-horizon: 60+ hour agent trajectories; 100K+ H100-hours built for the preview • Everything open, including traces: tasks, harnesses, verifiers, full agent trajectories We believe research will never end. A game turns zero-sum only when the pie is too small to share. However, research is definitely the field with the highest ceiling there is. What shifts is the mindset: it frees researchers to find and formulate the crazier, more valuable, more exciting problems in the world.
As AI systems enter a recursive self-improvement loop, the central question is whether it can systematically move beyond human-designed methods to genuinely extend the scientific and intelligence frontier. What’s missing is a neutral, open standard for evaluating these capabilities in real, production-scale intelligence development. Today, we’re releasing OpenRSI-Index v0.1: evaluating whether AI can recursively improve itself and push the boundaries of intelligence and science, on production-scale clusters with 1k GPUs. We turn fully open-source projects into autoresearch environments, with agent trajectories lasting 60+ hours. Building v0.1 took 100K+ H100-hours. We’re building an ecosystem with and for the research community: let RSI benefit everyone, and let everyone shape RSI together. We invite task contributors and compute partners to build this open benchmark with us - all contributors will be included as paper authors. Shape RSI with us: 🌐 Website: index.openrsi.foundation/ind… 🛠️ GitHub: github.com/OpenRSI-Foundatio… 🤝 Contribute: github.com/OpenRSI-Foundatio…
4
4
2
29
3,765
David Han retweeted
OpenRSI just launched OpenRSI-Index v0.1 today. Here's what you need to know. OpenRSI-Index is a new open benchmark for measuring recursive self-improvement, or RSI, in frontier AI model development. The idea is to test whether AI agents can push past human-designed methods to genuinely extend science and intelligence on their own, at real production scale rather than in toy setups. The benchmark turns fully open-source projects into autoresearch environments. Agents get let loose on these environments in trajectories that run 60-plus hours, operating on clusters with 1,000 GPUs, closer to how real frontier labs actually train and research. Building v0.1 already consumed over 100,000 H100-hours. The project also has a contribution pipeline called RSI-Anything, which walks outside researchers through turning their own research question into a packaged task in about an hour, covering baseline, evaluation, and budget. OpenRSI says it is being built as an open ecosystem, inviting task contributors and compute partners, with all contributors credited as paper authors on the eventual write-up. Key numbers: - v0.1 preview released today - Agent trajectories run 60+ hours - Tested on clusters with 1,000 GPUs - 100,000+ H100-hours spent building v0.1 - RSI-Anything task pipeline takes about 1 hour Code and docs are live now on GitHub and at index.openrsi.foundation.
As AI systems enter a recursive self-improvement loop, the central question is whether it can systematically move beyond human-designed methods to genuinely extend the scientific and intelligence frontier. What’s missing is a neutral, open standard for evaluating these capabilities in real, production-scale intelligence development. Today, we’re releasing OpenRSI-Index v0.1: evaluating whether AI can recursively improve itself and push the boundaries of intelligence and science, on production-scale clusters with 1k GPUs. We turn fully open-source projects into autoresearch environments, with agent trajectories lasting 60+ hours. Building v0.1 took 100K+ H100-hours. We’re building an ecosystem with and for the research community: let RSI benefit everyone, and let everyone shape RSI together. We invite task contributors and compute partners to build this open benchmark with us - all contributors will be included as paper authors. Shape RSI with us: 🌐 Website: index.openrsi.foundation/ind… 🛠️ GitHub: github.com/OpenRSI-Foundatio… 🤝 Contribute: github.com/OpenRSI-Foundatio…
1
3
4
7
2,159
David Han retweeted
That should definitely be one of our System Signature Task!
wonder if optimizing slime can be one of the task :p
1
6
575
Amazing to work with the amazing team!!
Let RSI benefit everyone and let everyone shape RSI. Excited to be one of the project initiators helping the design of the signiture task and organization. I also think most of the research can be formulated as 1) hill climbing a current task, and 2) formalize a new task. We would like to share the process with everyone, rather than locking them only in some places.
1
6
1,441
David Han retweeted
Must-read paper from Google on self-improving agent harnesses. If you auto-optimize your agent's harness, your eval score can go up while the agent gets worse on real tasks. This paper shows how to prevent that. Of five harness-evolution methods compared on agentic workspace tasks, RRSI scored the lowest on the tasks it evolved against and highest on all three out-of-distribution benchmarks. Automated harness evolution proposes edits to prompts, control flow, tools and memory, keeps the ones that raise the score, and repeats. The authors show this overfits the training tasks. Meta-Harness reached 93.0 on the Harvey LAB evolve split but gained only 0.3 to 1.5 points on JobBench, GDPval and APEX-Agents. RRSI adds regularization on both sides of the loop. The proposer gets an edit budget that shrinks over time and is pushed toward directions it has not tried. A critic rejects benchmark-specific edits, and a pruner removes edits that are too small, too costly or no longer useful. RRSI scored 90.5 on the evolve split and gained 3.5 to 4.7 points on the three held-out benchmarks. In the ablation, unregularized evolution used 3.80M tokens per trial against 2.42M for RRSI. With Gemini 3.5 Flash, RRSI raised Terminal-Bench 2.1 from 64.6 to 78.7 and carried a 2.2-point gain over to SWE-bench Verified. Paper: arxiv.org/abs/2609.24972 Chat with Paper: academy.dair.ai/papers/rrsi-…
48
63
9
403
32,205
David Han retweeted
Let’s call it Vibe Contribution! All you need to do is just to talk with Codex/Claude Code within 1 hour in our RSI-Anything pipeline!
RSI-Anything is currently the framework that best aligns with my understanding of expert-annotated data. To some extent, there is no fundamental difference between expert-annotated data and CS students' vibe coding. What everyone sees is the result translated by the agent, and when the agent deems a decision necessary, it makes a choice-based decision. The entire system is agent-driven, with humans existing as tools.
1
6
869
Open recursive self-improvement is the next frontier of AI to genuinely expand scientific boundaries. To scale OpenRSI-Index v0.1, we are calling for Domain Leads and Compute Partners!🚀 Join us in shaping the future of RSI! 🌐
As AI systems enter a recursive self-improvement loop, the central question is whether it can systematically move beyond human-designed methods to genuinely extend the scientific and intelligence frontier. What’s missing is a neutral, open standard for evaluating these capabilities in real, production-scale intelligence development. Today, we’re releasing OpenRSI-Index v0.1: evaluating whether AI can recursively improve itself and push the boundaries of intelligence and science, on production-scale clusters with 1k GPUs. We turn fully open-source projects into autoresearch environments, with agent trajectories lasting 60+ hours. Building v0.1 took 100K+ H100-hours. We’re building an ecosystem with and for the research community: let RSI benefit everyone, and let everyone shape RSI together. We invite task contributors and compute partners to build this open benchmark with us - all contributors will be included as paper authors. Shape RSI with us: 🌐 Website: index.openrsi.foundation/ind… 🛠️ GitHub: github.com/OpenRSI-Foundatio… 🤝 Contribute: github.com/OpenRSI-Foundatio…
1
2
11
1,196
i actually didn't realize this until seeing this post, but muse spark 1.3 ranks as #2 on Agents Last Exam (ALE)!
Muse Spark 1.3 ranks #2 on Agents' Last Exam (ALE) leaderboard, ranked #1 at the time of its release. Kudos to the team! Somehow this didn't get enough buzz. Leaderboard: agents-last-exam.org/leaderb…
39
22
2
414
62,621
David Han retweeted
Today, we’re launching ResearcherScout to help you hire AI researchers at conferences. Application companies are becoming the new neolabs. Now everyone wants to build a great research team. But you can’t judge a researcher by their LinkedIn profile. You need to understand their papers, their actual work, and meet them in person. One user described the tool as a "lifesaver." ECCV main conference starts from Sep 10th, try it now: perfectly.so/researcher-scou….
11
4
4
27
5,231
OpenRouter: 40 months → $7B+ reported acquisition. It took them ~2.5 years to reach $10M ARR. OrcaRouter: 2.5 months → on track for $10M ARR in-month. 🐋 A few more numbers from our first 10 weeks ↓
37
32
29
265
316,405
David Han retweeted
Introducing dots3-note preview — a small but mighty step toward long-horizon agency in real life. 🔹 280B MoE with 16B active parameters, a 512K context window, and multimodal understanding across text, vision, and audio 🔹 Introduces TEMPO, a new RL approach for long-horizon agent training through self-critiquing and test-time-scaled value estimation 🔹 Built to reason, explore unfamiliar environments, update memory over time, and combine multimodal perception with coding and tool use to solve complex tasks 🔹 Open weights on Hugging Face, alongside two open benchmarks for real-life agents: VibeSearchBench and VibeLifeBench Competitive with much larger models across reasoning, agentic, and multimodal evaluations. 🔗 Tech blog: studio.dots.ai/dots/dots3-en… 🔗 Model weights: huggingface.co/dots-studio/d… 🔗 Github: github.com/studio-dots-ai/do…
180
295
260
1,535
1,125,006
David Han retweeted
Introducing FrontierPhysics: benchmark for e2e frontier physics research We built SkillsBench and got 180+ citations in 6mo. People have been asking next project to contribute to Led by @bingran_bry - Physics PhD @UCBerkeley, Nature author, advised by Prof. Haeffner and more 🧵
10
26
3
145
23,167