A 27-billion-parameter model reduced to 5.9 GB

PrismML releases Ternary Bonsai 2 27B, a ternary version of Qwen3.8 27B announced to be nine times more compact, with 98.2% of its aggregate performance.

Fitting a 27-billion-parameter model into less than 6 GB without reducing it to a severely degraded version. That is the promise of Ternary Bonsai 2 27B, a Qwen3.8 27B variant designed by PrismML to run locally on a laptop or a single graphics card.

The model retains the parameter count and general architecture of its foundation. PrismML has not created a smaller variant in the conventional sense. Instead, the company changes how the weights are represented and processed to reduce their memory footprint.

The full FP16 version would require approximately 54 GB for its weights alone. Bonsai 2’s most compact format occupies 5.95 GB, a reduction of nearly ninefold. A second version, slightly larger but generally faster at processing prompts, reaches 7.21 GB.

This compression relies on ternary weights. Instead of storing each value with 16 bits, the model restricts most of its weights to three possibilities: −1, 0, or +1. A shared FP16 scaling factor is applied to groups of 128 weights to restore an appropriate magnitude for each part of the network.

A value with three states theoretically contains approximately 1.585 bits of information. After accounting for the scaling factors and the small number of tensors maintained at higher precision, PrismML calculates an average of 1.72 bits per weight. The most compact GGUF format rises slightly to 1.75 bits because of its storage method.

The conversion does not simply round every weight to the nearest ternary value. The matrices are first transformed block by block into a different mathematical basis using a Hadamard rotation. The runtime applies the corresponding transformation to the activations during inference.

This step is intended to distribute the values more evenly before reducing them. The transformed weights are stored directly in compact form and are not fully reconstructed in FP16 when the model generates a response. The memory savings can therefore be maintained during use, provided that the system has access to the specialized compute kernels developed by PrismML.

Bonsai 2 is derived from Qwen3.8 27B, with no announced changes to its architecture. The model card published on Hugging Face lists 27.36 billion parameters: 24.35 billion in the language backbone, 2.54 billion in the input and output layers, and approximately 460 million in the vision module.

The model uses 64 blocks and a hybrid attention system. Approximately three-quarters of the layers rely on a form of linear attention, while the remaining quarter uses full attention. This design helps make its advertised 262,000-token context window more practical.

It accepts text and images but generates only text. The module responsible for understanding images is not included in the 5.95 GB promoted for the language model. In the GGUF distribution, it requires an additional 630 MB in Q8. The BF16 reference version reaches 930 MB.

The 5.9 GB figure must therefore be interpreted precisely. It refers to the language core using the `PTQ10` format, which packs the three states close to their theoretical storage cost. A complete multimodal installation requires more space, in addition to the runtime software, temporary data, and memory used by the context.

PrismML also offers a `PQ20` format. Each ternary weight occupies a two-bit slot, simplifying some computations but increasing the language file to 7.21 GB. This is the version downloaded by default through the demo repository, along with the vision module. The project’s basic installation therefore retrieves approximately 7.8 GB of weights before optionally adding Open WebUI and the code interpreter.

On Apple devices, the MLX version occupies 8.6 GB on disk. The language model accounts for 7.67 GB, while the vision module adds 920 MB. The difference comes from the MLX container, which stores a scale and bias for each group. The logical representation remains ternary, but its practical storage cost reaches 2.25 bits per weight.

The claim that Bonsai 2 is “nine times smaller” does not therefore apply uniformly to every distribution. The reduction approaches ninefold for the `PTQ10` language core compared with the 54 GB FP16 version. It falls to approximately 7.5 times for `PQ20`, then decreases further once the vision system and software components are included.

File size also does not represent all the memory required during a session. Intermediate representations, model state, and the cache associated with the context must be added to the weights. The longer a conversation, code repository, or document becomes, the greater this additional memory requirement.

The cache documentation estimates that a 100,000-token context requires approximately 6.3 GiB with the default FP16 cache. An experimental four-bit mode can reduce this amount to approximately 1.8 GiB, at the cost of slightly slower generation and a small quality loss if suitable calibration is not applied.

A maximum window of 262,000 tokens does not therefore mean it will be available on every configuration with just enough memory to load the 5.9 GB of weights. The device must also accommodate the cache, runtime environment, and other active applications. The maximum length describes a model capability, not a practical guarantee for every machine.

PrismML’s central claim concerns retained quality. The company says Bonsai 2 preserves 98.2% of the aggregate performance of Qwen3.8 27B at full precision. That percentage does not mean that 98.2% of its answers will be identical or that every capability loses exactly 1.8%.

The figure represents the ratio between two averages calculated across a collection of tests. In the detailed technical model card, the FP16 version earns an average score of 86.32 across 14 evaluations, compared with 84.78 for Bonsai 2. Dividing the second value by the first produces the advertised 98.2%.

The tests cover knowledge, reasoning, math, coding, instruction following, tool use, and image understanding. PrismML ran them on H100 GPUs using EvalScope and vLLM, with the