Locke Cai (@couplefire12) header
Locke Cai (@couplefire12) profile picture

Locke Cai

@couplefire12

MTS @thinkymachines | prev @xai, @MIT_CSAIL, @cursor_ai, @togethercompute

Joined August 2023
  • Tweets23
  • Following164
  • Followers517
6Media
Photo by Locke Cai (@couplefire12)Photo by Locke Cai (@couplefire12)Photo by Locke Cai (@couplefire12)Photo by Locke Cai (@couplefire12)Photo by Locke Cai (@couplefire12)Photo by Locke Cai (@couplefire12)
RL for reasoning often rely on verifiers — great for math, but tricky for creative writing or open-ended research. Meet RARO: a new paradigm that teaches LLMs to reason via adversarial games instead of verification. No verifiers. No environments. Just demonstrations. 🧵👇
Image from Locke Cai (@couplefire12)
24
77
30
604
181,270
Locke Cai retweeted
Nim Ravid (@Nim_Ravid1) profile picture
Nim Ravid
@Nim_Ravid1Jul 16
Introducing Aidan, your newest AI employee Aidan is the first computer-using AI built for realtime conversation We’ve raised $45M from @sequoia and @8VC to bring Aidan to the world @NotionHQ and @DecagonAI already use Aidan to run customer interactions. This is how @sablehq works:
298
143
138
1,535
744,313
pretty cool paper on learning to reason from demonstrations arxiv.org/abs/2511.21667 tl;dr instead of SFT'ing on expert demonstrations, try doing inverse RL to learn a critic. This lets you do RL on domains without 'natural' verifiers
Escaping the Verifier: Learning to Reason via Demonstrations

Escaping the Verifier: Learning to Reason via Demonstrations

Training Large Language Models (LLMs) to reason often relies on Reinforcement Learning (RL) with task-specific verifiers. However, many real-world reasoning-intensive tasks lack verifiers, despite...

arxiv.org
7
24
2
244
25,316
“heard you guys had trouble verifying the unverifiable” - GANfather
Image from Yacine Mahdid (@yacinelearning)
Locke Cai (@couplefire12) profile pictureLocke Cai
@couplefire1211 Dec 2025
RL for reasoning often rely on verifiers — great for math, but tricky for creative writing or open-ended research. Meet RARO: a new paradigm that teaches LLMs to reason via adversarial games instead of verification. No verifiers. No environments. Just demonstrations. 🧵👇
Image from Locke Cai (@couplefire12)
11
9
2
362
42,453
RL on LLMs with no verifiers & environments... But how? This paper trains LLMs to reason without using verifiers or human preferences, all by turning expert demos into an adversarial game models can then learn self-correcting reasoning, even rivaling verifier-based RL methods!
Image from alphaXiv (@askalphaxiv)
11
39
3
223
28,764
This is super cool, basically GANs for LLM post-training policy tries to mimic expert answers, critic tries to identify expert answer vs policy answer I'm curious to try this out on some medical tasks... I had similar ideas about 2 years ago which I was discussing in EleutherAI but I was trying to apply it to RLHF and didn't pursue any further... skill issue on my part I guess lol
Locke Cai (@couplefire12) profile pictureLocke Cai
@couplefire1211 Dec 2025
RL for reasoning often rely on verifiers — great for math, but tricky for creative writing or open-ended research. Meet RARO: a new paradigm that teaches LLMs to reason via adversarial games instead of verification. No verifiers. No environments. Just demonstrations. 🧵👇
Image from Locke Cai (@couplefire12)
11
16
269
26,439
Locke Cai retweeted
christina (@luoluo_ai) profile picture
christina
@luoluo_ai11 Dec 2025
someone made self-play work! RARO is cool to tackle the unstable adversarial training setup with shared weight update for both generator and discriminator with regularized entropy to “distill” the implicit expert policy by minimizing the reverse KL. Long live GAN lol
Locke Cai (@couplefire12) profile pictureLocke Cai
@couplefire1211 Dec 2025
RL for reasoning often rely on verifiers — great for math, but tricky for creative writing or open-ended research. Meet RARO: a new paradigm that teaches LLMs to reason via adversarial games instead of verification. No verifiers. No environments. Just demonstrations. 🧵👇
Image from Locke Cai (@couplefire12)
12
21
253
42,020
Woke up to some amazing feedback, thanks everyone!! @provilkov and I are working hard to release a plug-and-play RARO repo soon — what domains do you want to see supported? If you have specific model/dataset requests, let us know in the announcement thread! 👇
Locke Cai (@couplefire12) profile pictureLocke Cai
@couplefire1211 Dec 2025
RL for reasoning often rely on verifiers — great for math, but tricky for creative writing or open-ended research. Meet RARO: a new paradigm that teaches LLMs to reason via adversarial games instead of verification. No verifiers. No environments. Just demonstrations. 🧵👇
Image from Locke Cai (@couplefire12)
1
1
18
1,286
🙌 HUGE shoutout to @provilkov & @m_ryabinin for the incredible mentorship! Grateful to my friends at @togethercompute & @MIT for insightful discussions that shaped RARO! 📄 Paper: arxiv.org/abs/2511.21667
Escaping the Verifier: Learning to Reason via Demonstrations

Escaping the Verifier: Learning to Reason via Demonstrations

Training Large Language Models (LLMs) to reason often relies on Reinforcement Learning (RL) with task-specific verifiers. However, many real-world reasoning-intensive tasks lack verifiers, despite...

arxiv.org
2
4
55
3,635
Update: we are working hard to release plug-and-play code for everyone soon, if you have any specific model/dataset requests, let us know here! Thanks everyone for their support!
1
17
1,965
Load more