DiffusionGemma: Google is testing text diffusion to accelerate local inference

Google DeepMind releases DiffusionGemma, a 26B MoE model using text diffusion to generate 256-token blocks in parallel, boosting local inference speed 4x.

Google DeepMind releases DiffusionGemma, an experimental open-source model under an Apache 2.0 license that breaks away from the token-by-token generation of autoregressive LLMs. Stemming from the Gemma 4 family and Gemini Diffusion research, this 26-billion-parameter Mixture of Experts (3.8 billion active during inference) generates 256-token blocks in parallel, based on the principle of image generators that progressively refine an initial noise. The claimed result: up to four times faster inference, i.e., over 1,000 tokens per second on an H100 and over 700 on an RTX 5090, all within 18 GB of VRAM once quantized.

Bidirectional attention, where each token sees all others, enables use cases that sequential models struggle with: in-line editing, code infilling, non-linear structures like amino acid sequences. The model also self-corrects by re-evaluating the entire block with each pass. Google outlines the limitations: output quality remains inferior to standard Gemma 4, and the speed advantage diminishes in high-throughput cloud serving, where batching already saturates the hardware. The benefit therefore targets local inference and low concurrency scenarios.

The weights are available on Hugging Face, with vLLM, MLX, and Transformers support, llama.cpp coming soon, and optimized NVFP4 kernels with NVIDIA.