EAGLE-4 is here. From EAGLE-1 to EAGLE-3, we’ve kept making inference faster. EAGLE-4 is our fastest yet. 961.6 tokens/s on @Alibaba_Qwen’s Qwen3.8-27B with TokenSpeed (BF16, 4× @NVIDIA B300, TP4, one request, T=0). Github: github.com/eagle-hq/EAGLE-4

Then we put it on a phone. Motorola Razr 70 Ultra CPU: HumanEval: 82.6 vs 22.8 tokens/s, 3.62× vanilla. Long-form generation: 74.7 vs 23.3, 3.21×. 4-bit Gemma-4-E2B-it, single request, T=0, cool device. Unmodified, non-rooted phone.

EAGLE-4 on @AMD Instinct MI350X. 262.89 vs 75.27 tokens/s for a single request: 3.49× vanilla. At 32 concurrent requests: 2,360.83 vs 1,390.52 aggregate tokens/s, 1.70× vanilla. AMD team’s day-0 results. 1 GPU, TP1; 117-turn SPEED-Bench; max 4,096 new tokens.

On @NVIDIA’s Nemotron 3.5 Super VL: Single request: 647 vs 236 tokens/s, 2.74× vanilla. At 512 concurrent requests: 33,480 vs 19,823 aggregate tokens/s, 1.69× vanilla. SPEED-Bench qualitative, 4× B300, TP4, T=0.

The same trained Qwen3.8-27B drafter, now at 512 concurrent requests: EAGLE-4: 28,340 aggregate tokens/s DFlash 2: 16,641 DSpark: 14,229 Vanilla: 21,203 1.34× vanilla, 1.70× DFlash 2 and 1.99× DSpark. Paper SPEED-Bench comparison, T=0, SGLang.

This is a great step forward for open-source models. The cost advantage makes it possible to scale inference and verification in ways that would be expensive with closed models. The tradeoff is latency. Verification can require dozens of inference passes per task. While most can run in parallel, some must run sequentially, making end-to-end latency multiple times longer than a single generation. With EAGLE, we can accelerate those inference passes and bring the end-to-end time faster than a single standard generation, while keeping the benefits of self-verification.

AI hardware has advanced rapidly over the past few years, delivering unprecedented compute for training and inference. As AI systems continue to scale, achieving higher performance increasingly depends on both more capable hardware and more efficient software. Speculative decoding is a prime example. Rather than requiring new hardware, it unlocks more performance from existing accelerators by fundamentally changing how inference is executed. A lightweight draft model predicts multiple future tokens, which are then verified in parallel by the target model. By reducing sequential decoding steps and improving hardware utilization, speculative decoding enables substantially higher throughput from the same underlying compute. This reflects a broader shift in AI infrastructure. Software is redefining AI performance by unlocking more from increasingly powerful hardware. As accelerators continue to evolve, advances in inference algorithms will be just as critical as advances in the hardware itself, together defining the next generation of AI systems.