Ben

Training Group Relative Policy Optimization, GRPO, models on the hardest problems delivers the biggest gains when annotation budgets are tight. 30 to 40 point jumps on reasoning tasks, plus 20% out of distribution lift. Labeling is expensive, so the paper asks which 10% of data is worth buying. They estimate difficulty by sampling several answers from the base model and computing a pass rate. That ranks prompts into easy, middle, hard, then GRPO trains on only that slice. GRPO learns by sampling several rollouts per prompt and rewarding outputs that beat the group average. If every rollout gets the same reward, the learning signal vanishes and progress stops. Easy prompts become solved fast, so variance collapses, while hard prompts keep a mix of wins and fails, so gradients stay informative. Across GSM8K and Tracking Shuffled Objects, hardest 10% beats easy, middle, and random on multiple models. On AIME-2025, only the hard-trained model tops the base, about 20% relative improvement. The practical rule, buy problems the base model rarely solves but sometimes gets right. This sustains learning under a fixed budget and improves robustness on harder tasks.