Rishi Mehta (@rishicomplex) header
Rishi Mehta (@rishicomplex) profile picture

Rishi Mehta

@rishicomplex

Solve i̶n̶t̶e̶l̶l̶i̶g̶e̶n̶c̶e̶ ̶ coding, use it to solve everything else | Research @AnthropicAI | Past: RL @GoogleDeepmind: AlphaProof co-lead, Gemini.

 San Francisco, CA
rishimehta.xyz
Joined July 2009
  • Tweets339
  • Following368
  • Followers3.6K
banger
Image from Rishi Mehta (@rishicomplex)
5
174
Opus 5.5 tops RoastBench
Image from Rishi Mehta (@rishicomplex)
Image from Rishi Mehta (@rishicomplex)
Rishi Mehta (@rishicomplex) profile pictureRishi Mehta
@rishicomplexJun 1
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
Image from Rishi Mehta (@rishicomplex)
2
1
12
1,517
vibes
6
590
Couple bangers from them
Image from Rishi Mehta (@rishicomplex)
Image from Rishi Mehta (@rishicomplex)
1
1
326
RoastBench results for Fable 5.1 and GPT-6. They're tied on the main set, but fable does a lot better on the second set. Neither felt like a major step up though.
Image from Rishi Mehta (@rishicomplex)
Image from Rishi Mehta (@rishicomplex)
Rishi Mehta (@rishicomplex) profile pictureRishi Mehta
@rishicomplexJun 1
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
Image from Rishi Mehta (@rishicomplex)
1
1
6
2,068
if you've been holding out on fable due to price or zdr you really should try fable 5.1
Image from Rishi Mehta (@rishicomplex)
Claude (@claudeai) profile pictureClaude
@claudeaiSep 1
We’re introducing Claude Fable 5.1 and Claude Mythos 5.1. They're the world’s most advanced models for coding and knowledge work.
2
8
101
10,911
also lol'd at this from kimi
Image from Rishi Mehta (@rishicomplex)
3
252
this one made me laugh
Image from Rishi Mehta (@rishicomplex)
1
2
334
Added Opus 5, GPT 5.6 Sol and Kimi K3 to RoastBench. Opus 5 narrowly edges out Fable at the top
Image from Rishi Mehta (@rishicomplex)
Rishi Mehta (@rishicomplex) profile pictureRishi Mehta
@rishicomplexJun 1
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
Image from Rishi Mehta (@rishicomplex)
1
2
865
The one joke that made me laugh
Image from Rishi Mehta (@rishicomplex)
1
3
579
Fable 5 beats Opus 4.8 on RoastBench (but still well behind humans)
Image from Rishi Mehta (@rishicomplex)
Rishi Mehta (@rishicomplex) profile pictureRishi Mehta
@rishicomplexJun 1
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
Image from Rishi Mehta (@rishicomplex)
2
1
8
2,129
For the first time I don't feel the need to review its code line-by-line. It works autonomously over long horizons, on underspecified prompts, figuring things out as it goes.
Image from Rishi Mehta (@rishicomplex)
1
1
693
The models can kind of figure out the beginnings of a setup but their punchlines just fall flat. It's like they don't yet have a good model for what causes a human to laugh.
Image from Rishi Mehta (@rishicomplex)
Image from Rishi Mehta (@rishicomplex)
1
3
496
Load more