Tommy Xiao

Qwen3.8-27B has reportedly been compressed from 55.6GB to just 5.9GB, while retaining 98.2% of its aggregate benchmark performance. That’s an 89.4% reduction in model size. The compressed version is more than nine times smaller than the original. The result comes from Bonsai 2, a compression method developed by PrismML. The team reports that it improves on its previous generation, which retained approximately 95% of the original model’s benchmark performance. Here are the reported figures: Original model: 55.6GB Bonsai 2 version: 5.9GB Size reduction: 89.4% Aggregate performance retained: 98.2% Previous Bonsai generation: approximately 95% retention Evaluation coverage: 20 benchmarks The interesting question is how much of a 27B model’s capability can survive when its representation becomes this small. 1. The technology behind the compression Bonsai 2 uses ternary quantization. Traditional model weights can represent many different numerical values. Storing those values across billions of parameters requires considerable space. Ternary quantization takes a more aggressive approach by mapping weights to just three possible states: -1 0 +1 Scaling factors help preserve information about the magnitude of the original weights. This allows the compressed model to use far fewer bits to represent its parameters. The challenge is that model weights aren’t just numbers taking up storage. Their relationships encode the patterns and capabilities that allow a model to follow instructions, reason, write code and solve problems. Reduce the precision too aggressively and useful information can disappear. The goal is therefore not simply to make the model smaller. It’s to preserve as much of its learned behaviour as possible while reducing the storage required. 2. What improved with Bonsai 2? PrismML reports that its previous Bonsai generation retained around 95% of Qwen3.8-27B’s aggregate benchmark performance. Bonsai 2 reportedly raises that to 98.2% across 20 benchmarks. The team also highlights improvements in: Coding agents Multimodal reasoning Long-horizon tool use These capabilities deserve attention because they involve more than getting a single answer right. A coding agent, for example, may need to inspect an existing repository, understand dependencies, plan changes, edit several files, run tests, interpret errors and correct its implementation. Weaknesses can accumulate across those steps. A model that performs well on isolated coding questions may still struggle to complete a longer development task. That’s why performance on agent workflows is worth examining separately from aggregate benchmark retention. 3. What this could change for local AI A 27B model typically requires substantial memory, which can make local deployment difficult on consumer hardware. A 5.9GB compressed representation could lower the barrier to distributing and experimenting with a model of this size. Potential applications include: Running larger models on consumer GPUs with limited VRAM. Testing 27B-class models on memory-constrained laptops. Deploying models on edge devices. Hosting more models within the same server memory budget. Building local AI applications with less dependence on cloud APIs. For developers experimenting with coding agents, this could make it easier to test more capable models without needing expensive hardware from the start. 4. One important distinction The 5.9GB figure refers to the compressed model file, not the total memory needed for inference. Running the model still requires memory for the KV cache, activations, inference buffers and other runtime overhead. Actual requirements will also depend on context length, concurrency, the inference engine and hardware support for the compressed format. The 98.2% figure needs similar care. It’s an aggregate across 20 benchmarks, not a guarantee that every individual task retains exactly 98.2% of the original model’s capability.