Gideon retweeted
Less than a third of tasks completed by the best models. AI agents still have quite a way to go.
We're launching Bank Arena: 52 long-running banking tasks across several banking apps.
We gave frontier computer-use agents a bank to run, from recovering transfers to settling merchant payouts. The median task takes experts ~75 minutes to complete, and the highest-scoring models completed <31% of tasks.
Here's what we found ↓
Gideon retweeted
🔥🔥🔥
We're launching Bank Arena: 52 long-running banking tasks across several banking apps.
We gave frontier computer-use agents a bank to run, from recovering transfers to settling merchant payouts. The median task takes experts ~75 minutes to complete, and the highest-scoring models completed <31% of tasks.
Here's what we found ↓
Gideon retweeted
Nice 🔥🔥
We're launching Bank Arena: 52 long-running banking tasks across several banking apps.
We gave frontier computer-use agents a bank to run, from recovering transfers to settling merchant payouts. The median task takes experts ~75 minutes to complete, and the highest-scoring models completed <31% of tasks.
Here's what we found ↓
Gideon retweeted
This was awesome to help work on. It’s obvious to us that making agentic financial operations more reliable is huge. Feel free to reach out, would love feedback on the benchmark!
We're launching Bank Arena: 52 long-running banking tasks across several banking apps.
We gave frontier computer-use agents a bank to run, from recovering transfers to settling merchant payouts. The median task takes experts ~75 minutes to complete, and the highest-scoring models completed <31% of tasks.
Here's what we found ↓
Gideon retweeted
Leaderboard, task breakdowns, and recorded trajectories are live at bankarena.ai/.
If you're training computer-use agents or building finance models, we'd love you to evaluate on Bank Arena - dm or [email protected]!
Gideon retweeted
In 33 of 52 tasks, new information arrives after they've acted, forcing them to revisit earlier decisions.
In one task: a customer's €85k transfer shows as canceled, so they send another. Then the first settles, the duplicate comes back, and a fraud signal means the first needs to be recalled. Some easy ways agents get this wrong: credit twice, recall the wrong transfer, or close the case early.
Gideon retweeted
We built a banking environment, simulating real-world banking applications with exact UI and workflow fidelity: a real ledger, policy and evidence vault, work tracker, ticketing tool, and ops desk.
Agents get screenshots and URLs, then click and type across payments, policies, work tracking, and customer tickets.
Gideon retweeted
We're launching Bank Arena: 52 long-running banking tasks across several banking apps.
We gave frontier computer-use agents a bank to run, from recovering transfers to settling merchant payouts. The median task takes experts ~75 minutes to complete, and the highest-scoring models completed <31% of tasks.
Here's what we found ↓
Gideon retweeted
super stoked to announce that HumanLayer is joining forces with factory
incredibly grateful to all our customers, investors, friends and family who cheered us on this crazy journey in the past 3 years
read our in-depth take, why we're so excited, what happens to customers and humanlayer OSS, and more 👇
Welcome to the team @dexhorthy and @0xblacklight! Let's build.
Read more: factory.com/news/humanlayer
Gideon retweeted
Finally, you can get real healthcare directly from your Claude, ChatGPT, GrokBot, Muse, or Instinct.
Just tell them to use FutureClinic. Agents welcome.
Healthcare will never be the same again.
Backed by @ycombinator
Congrats @raphaelschaad - both a clear and prescient thinker, and helped us a bunch. Excited for the future.
We’re excited to welcome @vivianmshen and @raphaelschaad as YC’s newest General Partners!
Vivian and Raphael first came to YC as founders.
Vivian co-founded Juni Learning (W18), which teaches kids coding and math, and later founded Acely, an AI tutor for college prep. Raphael founded Cron (W20), the calendar app that became Notion Calendar.
They built products used by millions of people, then returned to YC as Visiting Partners to help other founders. Now they're doing it full-time.
ycombinator.com/blog/welcome…
Gideon retweeted
Today, we're finally launching FrameOS!!!
Let's start with the why.
Recording your iPhone leaves that red dot indicator in your entire video and this has been a massive problem for me.
To add, I'm good on camera, so adding a camera feed + microphone involves juggling multiple editors....just to create a simple screen recording while I talk.
Onto the what: FrameOS
FrameOS is a simple tool for recording your screen. Connect your device, add a camera feed, and click record. It has all the editing features your used to: crop, resize, trim, and there's a lot more to come!
What you can do with FrameOS?
You can share feedback of your app with your product team, report a bug, or create a polished product demos of your investors - all without the red dot indicator.
Link in the comment below ⬇️
We're hiring a Founding GTM Lead @RulebaseHQ in New York. Come join our extremely focused team.
Our AI agents run customer operations for some of the fastest-growing financial services companies and public companies.
