The Revenue Agent for the Global 2000
San Francisco, CA
Joined January 2024
- Tweets626
- Following92
- Followers2.8K
- Likes845
Rox retweeted
I know X doesn’t care about Gartner, but... we won
Only Visionary in the 2026 Magic Quadrant for Revenue Action Orchestration. Year one of selling.
rox.com/reports/gartner-magi…
Rox is the ONLY Visionary in the 2026 Gartner® Magic Quadrant™ for Revenue Action Orchestration. 📈
We're the youngest company included in this year’s report.
Thank you to our customers and partners for helping shape what we’re building, and to the Rox team for making it happen.
rox.com/reports/gartner-magi…
We found similar results to @Harvey using graphs for our revenue agent:
8.9% → 99.9% accuracy on CRM questions at 1/20th the cost.
Graphs are the gold standard for agentic work moving forward.
We built context graphs to help agents work through long-horizon legal tasks.
Starting with Legal Agent Bench tasks in 11 categories like M&A diligence and contract negotiation, we had agents build context graphs of the underlying documents (without access to the eval tasks or grading rubrics).
Context graphs dramatically improved agent performance on LAB tasks:
- Firm Knowledge tasks: 65% → 80%
- Precedent lists tasks: 67% → 80%
- Matter counting tasks: 53% → 83%
They also boost agent efficiency: we found a 48% cost reduction for agents given access to a context graph to assist their work.
Full video below - more to come soon.
Our full write-up from August: rox.com/articles/knowledge-g…
Rox retweeted
How to use Devin to automate evals from production traffic.
Nobody at Rox writes evals anymore.
Devin now QAs our agent every week, on real traffic.
It reads real conversations with our agent, rebuilds each failure in a full copy of Rox, and turns it into an eval. We just review the PR.
The eval suite already covers 72% of real usage happening in the product, vs. 43–57% for the evals we used to write by hand.
And it’s growing week over week with Devin.
Built with @cognition and @braintrust:
Rox retweeted
Living Evals: an eval suite that grows from production from @rox_ai ⚡
Nobody at Rox writes evals anymore.
Devin now QAs our agent every week, on real traffic.
It reads real conversations with our agent, rebuilds each failure in a full copy of Rox, and turns it into an eval. We just review the PR.
The eval suite already covers 72% of real usage happening in the product, vs. 43–57% for the evals we used to write by hand.
And it’s growing week over week with Devin.
Built with @cognition and @braintrust:
2/ Fewer evals, more coverage
Instead of one eval per conversation, Devin classifies each one into a behavior using four labels: goal (what the user wants), output (what comes back), mutation (what changes), and target (named by the user, or inferred by the agent).
Devin proposed this scheme itself after its first rule split 1,337 conversations into 1,214 groups. Today, hundreds of conversations a week map onto 132 behaviors.
3/ What it changed for us
→ Bugs show up as failing evals before anyone reports them
→ Fixes only ship once their eval flips from red to green → The full suite runs 3x every night, at ~4.5¢ per eval
Full write-up by @gopalkgoel1, @XuBriGuy, and Akshar: rox.com/articles/living-eval…
1/ From failure to eval
Our agent had outgrown our hand-written evals: they kept passing while bug reports kept coming.
Devin pulls conversations from Braintrust and replays each request word-for-word in a preview environment: a full copy of Rox, no mocks, seeded with synthetic accounts, calls, and emails.
If our agent doesn’t repeat the production mistake, Devin adjusts the data until it does.
Nobody at Rox writes evals anymore.
Devin now QAs our agent every week, on real traffic.
It reads real conversations with our agent, rebuilds each failure in a full copy of Rox, and turns it into an eval. We just review the PR.
The eval suite already covers 72% of real usage happening in the product, vs. 43–57% for the evals we used to write by hand.
And it’s growing week over week with Devin.
Built with @cognition and @braintrust:
2/ Fewer evals, more coverage
Instead of one eval per conversation, Devin classifies each one into a behavior using four labels: goal (what the user wants), output (what comes back), mutation (what changes), and target (named by the user, or inferred by the agent).
Devin proposed this scheme itself after its first rule split 1,337 conversations into 1,214 groups. Today, hundreds of conversations a week map onto 132 behaviors.
3/ What it changed for us
→ Bugs show up as failing evals before anyone reports them
→ Fixes only ship once their eval flips from red to green → The full suite runs 3x every night, at ~4.5¢ per eval
Full write-up: rox.com/articles/living-eval…
Rox retweeted
Coding agents are outgrowing git because it was built for humans, not agents.
Revenue agents are outgrowing traditional CRMs which are built for humans to do data entry (junction objects, API limits etc) and not agents. Agents work differently. They scan every account, join emails, calls, and product usage, and keep discovering relationships nobody modeled.
New on our Engineering blog: Creating in-house source control.
In May, we merged around 2,500 code changes per day. In September, we merged more than 30,000 per day. As a result, we became bottlenecked by developer infrastructure built for human (rather than agent) scale.
Here, we share how we built a new version control system—and fully migrated our monorepo from git—in five months: anthropic.com/engineering/re…
2/ The surprise:
We tested on 33,933 real emails.
Using only how sellers edit the AI’s drafts, our model’s top 20% of emails got 1.64× the average reply rate.
When we also trained on real opens and replies, it rose to 1.75×.
We ran the same test using GPT 5.6-luna and Claude Sonnet 4.6 as judges. They only got 1.33× and 1.23×, which is barely better than random.
3/ Today, our agent drafts many emails, scores each, and sends the best.
Now, we can let it hill-climb toward better outbound without waiting out a weeks-long sales cycle.
This is the first paper in a series of research on building a self-improving revenue agent.
--
Thank you to our partners at @hiddenweights; @jstnlvndski, @ihabilyas, @andrew_ilyas, Mina Farid, Hedi Driss, and the rest of the team.
Our inference providers: @togethercompute, @modal, @baseten
Sovereign AI providers who inspire our own work: @cognition, @harvey, @tryramp, @appliedcompute
And our additional research partners like @SnorkelAI helping us push the frontier
Full write up by @gopalkgoel1, Calvin Yost-Wolff, and @marcus_kuhne:
rox.com/articles/the-outboun…
Coding agents got good by writing code, getting feedback, and improving thousands of times.
Agents doing commercial work never got that loop, until today.
With @hiddenweights, we trained an Outbound Reward Model on real cold emails + the edits sellers make to AI drafts before they hit send.
The reward model:
- Is 5x faster and up to 30x cheaper than an LLM judge
- Picks the seller-edited draft 80% of the time
- Gets 1.75x the avg reply rate on top-scored emails
🧵 on how we did it:
1/ Full paper: rox.com/articles/the-outboun…
Agents improve by hill-climbing a reward. In revenue, the outcome you care about is sparse (~1.5% reply rate), delayed (weeks), and confounded by timing and data quality.
Our insight: use the dense signal already in production. Every time a seller edits an AI draft before sending, that’s an expert labeling what was wrong, instantly, for free.
We trained a 3B model to score edited drafts above the originals. To widen coverage, @hiddenweights generated synthetic edits aimed at its blind spots, weighted by metagradient descent toward real replies.
