Incredibly neat rule of thumb from the How to Scale Your Model book! jax-ml.github.io/scaling-boo…
Transformer matmuls are communication-bound for per-device batch sizes (in terms of tokens) < the computational intensity of your accelerator. (240 for TPUv5e in photo, ~300 for H100s)
Results: Models trained on cardinal data prioritize the right tradeoffs. For example, relative to a model trained on binary feedback, our models prioritize improvements in content over style. We also see downstream improvements on benchmarks like Arena-Hard.
Alignment algorithms already choose to make tradeoffs during training. DPO improves loss on most prompts, but consistently some worse -- choosing which tradeoffs to make blindly.
1
2
52
🚨 We prove an impossibility result: Algorithms using only ordinal preferences ("response A > response B") are insufficient for alignment.
The problem? They can't resolve tradeoffs across prompts.
The solution? Cardinal feedback. This lets us recover preferences over models, not just responses.
New paper: arxiv.org/abs/2508.08486
Dataset: huggingface.co/datasets/card…
w/ coauthor @whitfill_parker 🧵👇




