We are excited to introduce ๐๐๐ซ๐ฆ๐ข๐ญ ๐: an open-source framework for training, evaluating, and deploying harness adapters.
Harness evolution is a key building block for RSI and continual learning, enabling agents to continually improve how they work from experience.
Train a reusable external layer. Use existing evolution methods or build your own through one standard framework.
GitHub: github.com/Infini-AI-Lab/herโฆ
1/5
#HarnessEvolution #ContinualLearning #RSI #Agent
A harness adapter in action: five Codex agents collaborating to prove a Lean theorem.
The agents communicate through the adapter, sharing plans, messages, and verified lemmas.
Agents split the proof into subproblems, prove lemmas in parallel, and reuse one anotherโs resultsโall without modifying Codex itself.
4/5
Make the training loop your own.
Connect your task execution, feedback, and adapter update logic through Hermitโs interfaces. Reuse its checks, versioning, and run records as you experiment with new evolution methods.
Hermit is open source. Try it on your tasks, build an Evolver, and share what you want your agents to learn next.
GitHub: github.com/Infini-AI-Lab/herโฆ
Website: infini-ai-lab.github.io/hermโฆ
5/5
The gains extend beyond knowledge recall.
Compared with continued dense training under the same token budget, STAPLE improves OLMo-2 1B by:
โข +4.27 points on HumanEval
โข +2.80 on MBPP
โข +2.27 on GSM8K
โข +1.65 on DROP
STAPLE achieves a higher ten-task average than the static sparse alternatives such as STEM, Engram-style, as well as a serving-oriented MoE upcycling baseline with eight experts and top-1 routing. We use 100B tokens for warmup and another 100B for midtraining.
The benefits extend to OLMo-2 7B despite shorter training budget (30B warmup + 70B midtraining): average math performance improves by 4.2% and code by 3.7% relative to dense continuation, with improvements on 9 of 10 tasks. MMLU gains are smaller, potentially reflecting the shorter warmup.
Billions of added parameters do not have to mean billions of extra parameters resident on GPU.
STAPLE-12L keeps 19.73 GB of embedding tables on CPU, using a 1 GB GPU cache. Predictable access enables prefetching; sparse training updates touch only the rows used by the batch.
The result:
โข Decoding throughput within 1.5% of dense
โข Training throughput within 3% of dense
โข About 2ร the decoding throughput of our MoE baseline
โข 58% lower GPU energy per token than our MoE baseline, with only 1.2% overhead over dense
What does the new memory actually contribute?
Removing the learned embeddings sharply degrades math and code performance, showing that the model has learned to rely on them.
The memory is interpretable and editable, while remaining responsive to context - a promising foundation for expanding pretrained models with additional complementary knowledge.
๐ง How can we keep scaling LLMs with minimal changes in their serving energy?
DeepSeekโs Engram and our concurrent work STEM showed the promise of static sparse memory: large embedding tables with efficient access. But can mature pretrained models learn to use a newly added memory within a limited training budget, avoiding costly pretraining from scratch?
๐ Introducing STAPLE: a recipe to extend pretrained models' abilities with static sparse memory. Serving energy per token stays close to dense and >50% below MoE upcycling in high-throughput regime.
Scalable capacity, interpretable memory, and compatibility with MoE upcycling - without any meaningful increase in serving cost. ๐งต
Paper: arxiv.org/abs/2610.12599
Code: github.com/Infini-AI-Lab/STAโฆ
Website: infini-ai-lab.github.io/STAPโฆ
Our central insight: under a limited training budget, new memory should complement pretrained knowledge.
STEM replaces FFN up-projections with token-indexed embeddings. For upcycling, STAPLE retains those projections and adds the embeddings alongside them, under the same context-dependent gate.
The recipe:
โข Zero-initialize the embeddings to preserve the starting modelโs function.
โข Warm up with the retained up-projections frozen.
โข Then jointly mid-train the model and memory.
Retaining the pretrained pathway makes a substantial difference. Warmup further improves knowledge and commonsense performance.
The architecture and training schedule both matter.
Why static sparsity for expanding pretrained models?
MoE upcycling can add capacity without restarting pretraining. But cloned experts can converge to homogeneous experts, might need additional tricks and more training budget to differentiate, while larger expert pools increase GPU-memory demands and can slow serving.
Static memory offers input-specific lookups with no expert homogeneity issues. Additionally, they can keep large tables on CPU and efficiently retrieve only the needed rows.
The challenge is making that memory useful quickly. Token frequencies are highly uneven, leaving rare rows with few updates. Replacing pretrained computation also discards useful features that must be relearned.
How do we add capacity while preserving what the model already knows?
Infini-AI-Lab retweeted
Serving data is an underestimated gold mine for agent self-improvement. But how well can agents actually learn from their serving experience?
In our recent work, ServeLearnBench, we take a closer look at this question. We find that current agents still struggle to learn from experience, and effective exploration plays a surprisingly important role.
Check out more results in our paper and repo!
๐ Paper: arxiv.org/abs/2610.07792
๐ป Code & Dataset: github.com/Infini-AI-Lab/Serโฆ
๐ Website: infini-ai-lab.github.io/Servโฆ
How well can agents continually self-improve from serving experience? Weโre excited to introduce our new benchmark, ๐๐๐ซ๐ฏ๐๐๐๐๐ซ๐ง๐๐๐ง๐๐ก:
Real-world deployment is not a one-time test of capability. The knowledge agents need is often hidden and can change over time.
ServeLearnBench tests whether agents can infer, apply, and revise this knowledge from interaction.
Across 5 learning harnesses ร 6 models, we find:
1. ๐๐๐ซ๐ ๐ ๐ฅ๐๐๐ซ๐ง๐ข๐ง๐ ๐ ๐๐ฉ๐ฌ ๐ซ๐๐ฆ๐๐ข๐ง. Agents can solve tasks when the hidden policy is given, but struggle to discover it from experience.
2. ๐๐จ ๐๐ซ๐๐ ๐ฅ๐ฎ๐ง๐๐ก ๐๐จ๐ซ ๐๐๐๐ฉ๐ญ๐๐ญ๐ข๐จ๐ง. Continual learning can be costlyโand can even degrade already-correct behavior.
3. ๐๐ฑ๐ฉ๐ฅ๐จ๐ซ๐๐ญ๐ข๐จ๐ง ๐ข๐ฌ ๐ ๐ค๐๐ฒ ๐๐จ๐ญ๐ญ๐ฅ๐๐ง๐๐๐ค. Better exploration strongly correlates with better adaptation.
Infini-AI-Lab retweeted
The next frontier would be agents that get smarter as you use them. ๐ Our new ServeLearnBench puts continual self-improvement to the testโand reveals whatโs holding it back.
How well can agents continually self-improve from serving experience? Weโre excited to introduce our new benchmark, ๐๐๐ซ๐ฏ๐๐๐๐๐ซ๐ง๐๐๐ง๐๐ก:
Real-world deployment is not a one-time test of capability. The knowledge agents need is often hidden and can change over time.
ServeLearnBench tests whether agents can infer, apply, and revise this knowledge from interaction.
Across 5 learning harnesses ร 6 models, we find:
1. ๐๐๐ซ๐ ๐ ๐ฅ๐๐๐ซ๐ง๐ข๐ง๐ ๐ ๐๐ฉ๐ฌ ๐ซ๐๐ฆ๐๐ข๐ง. Agents can solve tasks when the hidden policy is given, but struggle to discover it from experience.
2. ๐๐จ ๐๐ซ๐๐ ๐ฅ๐ฎ๐ง๐๐ก ๐๐จ๐ซ ๐๐๐๐ฉ๐ญ๐๐ญ๐ข๐จ๐ง. Continual learning can be costlyโand can even degrade already-correct behavior.
3. ๐๐ฑ๐ฉ๐ฅ๐จ๐ซ๐๐ญ๐ข๐จ๐ง ๐ข๐ฌ ๐ ๐ค๐๐ฒ ๐๐จ๐ญ๐ญ๐ฅ๐๐ง๐๐๐ค. Better exploration strongly correlates with better adaptation.
How well can agents continually self-improve from serving experience? Weโre excited to introduce our new benchmark, ๐๐๐ซ๐ฏ๐๐๐๐๐ซ๐ง๐๐๐ง๐๐ก:
Real-world deployment is not a one-time test of capability. The knowledge agents need is often hidden and can change over time.
ServeLearnBench tests whether agents can infer, apply, and revise this knowledge from interaction.
Across 5 learning harnesses ร 6 models, we find:
1. ๐๐๐ซ๐ ๐ ๐ฅ๐๐๐ซ๐ง๐ข๐ง๐ ๐ ๐๐ฉ๐ฌ ๐ซ๐๐ฆ๐๐ข๐ง. Agents can solve tasks when the hidden policy is given, but struggle to discover it from experience.
2. ๐๐จ ๐๐ซ๐๐ ๐ฅ๐ฎ๐ง๐๐ก ๐๐จ๐ซ ๐๐๐๐ฉ๐ญ๐๐ญ๐ข๐จ๐ง. Continual learning can be costlyโand can even degrade already-correct behavior.
3. ๐๐ฑ๐ฉ๐ฅ๐จ๐ซ๐๐ญ๐ข๐จ๐ง ๐ข๐ฌ ๐ ๐ค๐๐ฒ ๐๐จ๐ญ๐ญ๐ฅ๐๐ง๐๐๐ค. Better exploration strongly correlates with better adaptation.
Yet a substantial learning gap remains.
Even when models have the capability to solve the task, they struggle to learn the required behavior from serving experience.
On Retail L3, the best learner reaches only 59.6%. And adaptation isnโt free: learning harnesses cost 4โ101ร as much as Oracle and can even hurt already-correct behavior. ๐ธ
๐งต(4/5)
Explore ServeLearnBench:
๐๐๐๐ฌ๐ข๐ญ๐: infini-ai-lab.github.io/Servโฆ
๐๐จ๐๐&๐๐๐ญ๐๐ฌ๐๐ญ: github.com/Infini-AI-Lab/Serโฆ
๐๐๐ฉ๐๐ซ: arxiv.org/abs/2610.07792
๐งต(5/5)
We test the agent on 5 real-time apps, each deployed from a reference alone: face-to-face conversational avatar, a multimodal LLM (Qwen3-Omni) in text-prompt/speech-response mode, a video background editor, a video world model, and a video narrator.
Using Claude Code with Opus 4.8, FlashRT produces efficient real-time deployments for each one, reaching up to ~70ร reduction in response latency. Its throughput gains are largest on AMD MI355X (up to 3.6ร frame rate, versus 2.8ร on B200), showing that FlashRT enables more scalability on platforms there's less mature expert optimization.
๐งต5/5
Joint work with @wonderingkrish, @chenzhuoming911, Yanyuan Qin, Zhenyu Gu, Atri Rudra, and @BeidiChen. We are grateful for the collaboration with @AMD!