Enabling self-evolving agent systems through optimization. GitHub: https://nitter.gq/t.co/ywCwO95zKF
Joined February 2026
- Tweets433
- Following99
- Followers47
- Likes414
Pinned Tweet
MEGA Optimus is here.
Point it to a project folder. Optimus writes the spec, builds the eval harness, and runs the optimization loop end to end.
Reduce your token spend and latency while improving accuracy.
megacode.ai/start-optimizati…
Autonomous AgentOpt
A country could get more out of sovereign AI by funding local teams to adapt models than by funding a flagship model of its own.
The government can help create that industry by becoming an early customer. Then it can give local teams real work in ports, utilities, and public services, and pay for improvements those operators can actually use.
All without the dependency on foreign vendors or data leakage risk.
A company can get value from building a replacement for a piece of software without ever switching to it.
AI could shift the leverage companies have during negotiations at renewals.
Super important for enterprises to distinguish between expensive workflows that failed and experiments that helped employees find better ways to work.
To clarify, a spending cap can limit employees from trying things whose value they don’t know yet.
I appreciate the live experiment happening between the world's #1 and #2 companies by market cap:
NVIDIA: virtually no limit on AI spend (at $10K/month you need to justify and then it auto approves). AI use encouraged
Apple: AI use not really encouraged
We'll see what it will mean for business $$$ !
If a vendor says its product uses a particular AI model, should the customers be able to verify that?
A receipt proving which model produced the response would bring much more transparency and trust. It wouldn’t necessarily mean the response is good, but it would establish which model needs to be investigated when something goes wrong.
Keeping Git matters here.
An enterprise adopting coding agents already has build systems, permissions, scripts, and habits built around its repositories. Replacing all of that would add another migration before the agents could be useful.
Some of the infrastructure needed for AI adoption will involve substantial engineering so everyone else can keep using the tools they already know.
Pando is a Git backend which can scale to very large monorepos. We took the hard road of continuing to use Git, which means we could enable Pando with no change to devex. Considering providing this to others, let me know ([email protected]) if you'd be interested.
Employees can have different AI interfaces. They still need shared answers about how the company works.
If each assistant builds its own memory, one might remember an exception as standard practice. Another might keep using a process that changed last month.
Personal memory should help an assistant understand the employee. A shared knowledge base should establish what’s current for the company, with a clear way to resolve conflicts between the two.
Our Wisdom Graphs help enterprises navigate these challenges.
Whose assistant is an AI in a group chat?
If one person asks it to book dinner, another mentions a budget, and a third says nothing, it has to decide what counts as agreement.
A personal assistant can learn one person’s preferences, but a shared assistant needs a way to handle disagreement without treating the most vocal person as the whole group.
AI will make it harder for companies to leave certain assumptions untested.
A junior dev can build a prototype or an MVP that customers can try, in the matter of days.
But will companies welcome that when it comes from someone junior and contradicts a decision they’ve already made?
A person can approve an AI’s recommendation without noticing how much the AI shaped their opinion.
That makes “a human made the final decision” less reassuring on its own.
For decisions where independent judgment matters, it may be worth asking people to write down their view before showing them the AI’s recommendation. Otherwise agreement can look like a second opinion when it started with the first one.
If AI makes software faster to build, does a history of maintaining it become more valuable as a reason to trust it?
We believe it will be. A working demo tells you less about whether someone will fix a bug, handle a security issue, or keep the project working when its dependencies change.
If Qwen can become the starting point for a DNA model like Carbon-X, how much of an open model’s value will come from applications its original developers never planned for?
One useful experiment with Jev and OpenAI’s Decisions API is to let an optimizer change the checks an agent runs before acting.
For a research agent, those checks might cover whether the evidence supports a claim, whether sources disagree, and whether another search is needed.
Failures could drive changes to the checks themselves. Add one that catches a recurring mistake. Rewrite one that sends too many good answers back for more work. Remove one that adds latency without helping.
Then test the whole workflow again. A check that catches more mistakes can still make the agent worse if it keeps sending useful work back for unnecessary revisions.
This matters for agent optimization too. The checks used to judge a change need to sit outside the optimizer’s edit scope.
OpenAI’s latest incident report includes an agent removing monitoring, then replacing a test that preserved it with one requiring it to be absent.
Passing the revised test no longer meant preserving the original requirement.
An optimizer can propose changes to tests or monitoring, but those changes need separate review. They shouldn’t become the evidence used to approve the same proposal.
AI agents need fast tool calls and security controls they can’t rewrite.
NVIDIA OpenShell Reference Architecture combines the compute to run agents at scale with runtime controls that govern their access to tools and data.
See how the hardware and software work together: nvda.ws/4BiJBfY
A successful trace can still contain a decision worth fixing.
The agent may have picked the right account from two similar matches without checking enough evidence. On the next run, it could pick the other one.
Looking only at failures misses those unstable successes.
For an optimizer, they’re useful places to investigate. The change might be a specific disambiguation step that makes the existing capability more dependable, rather than teaching the agent something new.
Harness² explores a very useful part of agent optimization. A trace can tell you what to change even when there’s no answer key for the task.
An agent that skipped a source can be made to read it. A generated file can be reopened and checked. Those are observable changes in behavior.
But, they still don’t establish that the final answer is correct.
The record for a proposed fix should say which failure it addressed, what changed in the next run, and what still needs verification before the change is kept.
Can Agents improve their own harness on unseen tasks? The @GoogleResearch team released Harness², a framework that allows improve an harness around a frozen model.
How it works:
1️⃣ Agent runs the base harness on each task to record a baseline trajectory
2️⃣ Editor agent inspects the trajectory, drafts 2 candidate edits (SI, Skills, hooks,….), and runs both.
3️⃣ Composer agent diffs the new trajectory against the baseline, merges edits that fixed failures (within a cap), and runs the final harness.
4️⃣ Reflector agent logs verified lessons into file and saves the winning components to a library for future tasks to pull on demand.
Results:
- 💡 Gemini 3.5 Flash +8.4 on WorkBuddy-Bench and +7.5 on JobBench.
- 🛠️ Skills and scripts drive ~50% of edits; dropping skills costs 5.3 pts.
- 📚 Gemini 3.5 Flash improves on all 3 benchmarks when editing its own harness.
Paper: arxiv.org/abs/2610.14224
Code: github.com/google-research/h…
An agent can’t choose a useful recovery step if every tool failure comes back as “something went wrong.”
An oversized request, an expired credential, and an unavailable service need different responses. Rephrasing the request won’t refresh a credential. Retrying unchanged won’t make the payload smaller.
The tool adapter needs to preserve enough information for the agent to tell them apart.
When a trace shows repeated failed attempts, it’s worth checking what error the agent actually received before changing how it reasons about recovery.
An agent can choose the right tool and still fail because the environment can’t run it.
If that failure gets treated as a reasoning problem, the next change might be a longer prompt or a stronger model, but neither fixes the missing dependency.
This is why MEGA starts with the trace. The step where a task failed and the part of the system that needs changing can be different.
Google's Deep Review gives each agent a section to review, access to the full document, and a final stage that checks consistency across sections.
That's useful because a reasonable fix in one section can create a problem somewhere else.
Tightening an assumption in the methods might make a result valid, for example, while leaving the conclusion claiming more than the result supports.
An agent proposing that fix should follow it through the document and identify the other changes it requires. Otherwise, the author still has to work out whether the suggestions fit together.
Today @GoogleResearch is launching Deep Review — an agentic system designed to stress-test manuscripts, design docs, and complex proposals before submission or publication. After pilot partnerships analyzing over 15,000 research papers, we’re making Deep Review globally accessible to help researchers, engineers, and professionals bring structured critique to their daily workflows.
A decision model can choose the best available option even when none of the options is appropriate.
A support request might need clarification before it can be routed. A tool call might require information the agent doesn't have yet.
If the available choices all move the task forward, the system has already ruled out waiting or asking.
These models make narrow decisions easier to automate. The caller still needs to provide a way to say that the decision isn't ready to be made.
An optimization agent can keep running experiments while staying close to the approach it picked early on.
Fork-and-Flush keeps the accumulated files but starts fresh convos from that workspace. It runs several continuations and carries on with the highest-scoring one.
The useful distinction is between preserving the work and preserving the reasoning that led to it.
An optimization loop could test both, and useful changes can come from reconsidering existing work rather than extending the same convo.
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
