etireum retweeted
How can you trust an agent? ⤵️
Today we are launching OnchainBench, the benchmark for agents in crypto.
It helps you decide how far you can trust an AI agent with your keys: how often it finishes the job, what tasks cost, and how easily it falls prey to an attack.
People are starting to let AI agents swap, lend and even trade for them. The bigger the task, the greater the risk you're taking by trusting the agent. So before you hand it your keys, you need to assess it.
There are many risks to assess at once. An agent could buy two tokens for a liquidity position but never add them to the pool, so the job isn't complete. It could spend more on AI model calls than the task is worth. And, onchain, there's no undo: agents read web pages, files and tool descriptions that anyone can write, and if one of them tricks the agent into sending funds to the wrong address, the money is gone. Until now, there was no common way to evaluate agents on all of this and more.
OnchainBench helps you compare agents by giving them the kind of work people already ask them to do: swaps on @Uniswap, @SushiSwap and @Honeyswap, lending on @aave, deposits into sDAI from @SkyEcosystem, prediction markets on @Omen_eth, research, and jobs that run over several days.
Each task runs through a @safe smart account on a copy of a public chain. The first version covers @ethereum, @gnosischain, @0xPolygon, @arbitrum, @Optimism, @base, @Celo, @modenetwork and @robinhoodapp. The benchmark emulates the kind of contracts, liquidity and prices the agent would meet with your money, without putting any of it at risk. A task only counts as done if the chain shows it was done. Some tasks also hide an attack in what the agent reads, such as a look-alike token or a fake "verification" request, so you can see how often agents get tricked into moving funds before you trust one with yours.
In the first run, Claude Code (@claudeai), Codex (@OpenAI), Hermes (@NousResearch), OpenClaw (@openclaw) and our own new OCB agent on @autonolas worked through 147 private tasks they had never seen. They completed between 88.4% and 96.2% of tasks, at about $0.03 to $0.35 per completed task. Agents lost funds to an attack 13 times in 1,793 attack tests.
We are excited to contribute this to the crypto ecosystem. If you build a protocol or a chain, agents are already among your users, and will likely be one of your biggest sources of growth. OnchainBench shows, protocol by protocol and chain by chain, how well they handle yours.
For the AI ecosystem: want an agent eval with real ground truth and real adversaries? Onchain, every result can be checked and every injected instruction is a live risk. OnchainBench is open to contributions, uses private tasks, and is ready for your agent.
Help us make onchain agents harder to trick and better at the job. If you run a protocol or are a heavy user, propose the task you think agents still get wrong on the site below, and every agent on the benchmark could end up tested on it. If you build agents, run OnchainBench yourself and submit your agent to be run.
The leaderboard is live now at onchainbench.ai
Share it with a friend!
OnchainBench compares agents by giving real work: swaps, lending, deposits, prediction markets, research, and jobs that run over several days.
Today we are launching OnchainBench, the benchmark for agents in crypto.
It helps you decide how far you can trust an AI agent with your keys: how often it finishes the job, what tasks cost, and how easily it falls prey to an attack.
People are starting to let AI agents swap, lend and even trade for them. The bigger the task, the greater the risk you're taking by trusting the agent. So before you hand it your keys, you need to assess it.
There are many risks to assess at once. An agent could buy two tokens for a liquidity position but never add them to the pool, so the job isn't complete. It could spend more on AI model calls than the task is worth. And, onchain, there's no undo: agents read web pages, files and tool descriptions that anyone can write, and if one of them tricks the agent into sending funds to the wrong address, the money is gone. Until now, there was no common way to evaluate agents on all of this and more.
OnchainBench helps you compare agents by giving them the kind of work people already ask them to do: swaps on @Uniswap, @SushiSwap and @Honeyswap, lending on @aave, deposits into sDAI from @SkyEcosystem, prediction markets on @Omen_eth, research, and jobs that run over several days.
Each task runs through a @safe smart account on a copy of a public chain. The first version covers @ethereum, @gnosischain, @0xPolygon, @arbitrum, @Optimism, @base, @Celo, @modenetwork and @robinhoodapp. The benchmark emulates the kind of contracts, liquidity and prices the agent would meet with your money, without putting any of it at risk. A task only counts as done if the chain shows it was done. Some tasks also hide an attack in what the agent reads, such as a look-alike token or a fake "verification" request, so you can see how often agents get tricked into moving funds before you trust one with yours.
In the first run, Claude Code (@claudeai), Codex (@OpenAI), Hermes (@NousResearch), OpenClaw (@openclaw) and our own new OCB agent on @autonolas worked through 147 private tasks they had never seen. They completed between 88.4% and 96.2% of tasks, at about $0.03 to $0.35 per completed task. Agents lost funds to an attack 13 times in 1,793 attack tests.
We are excited to contribute this to the crypto ecosystem. If you build a protocol or a chain, agents are already among your users, and will likely be one of your biggest sources of growth. OnchainBench shows, protocol by protocol and chain by chain, how well they handle yours.
For the AI ecosystem: want an agent eval with real ground truth and real adversaries? Onchain, every result can be checked and every injected instruction is a live risk. OnchainBench is open to contributions, uses private tasks, and is ready for your agent.
Help us make onchain agents harder to trick and better at the job. If you run a protocol or are a heavy user, propose the task you think agents still get wrong on the site below, and every agent on the benchmark could end up tested on it. If you build agents, run OnchainBench yourself and submit your agent to be run.
The leaderboard is live now at onchainbench.ai
Share it with a friend!
1
106
I feel like we should talk about Glamsterdam more.
Upgrading a live decentralized network with 0 downtime while also making it 230% more scalable and adding dozens of new features is crazy.
I feel like we should talk about Glamsterdam more.
Upgrading a live decentralized network with 0 downtime while also making it 230% more scalable and adding dozens of new features is crazy.
157





