The model is fast. The catch is concurrency.

DiffusionGemma turns Gemma 4 26B A4B into a discrete-diffusion model: it refines 256-token canvases in parallel, averaging 19.74 tokens per forward and 1,479 output tokens per second on a single H100 in batch-1 measurements. 1

That makes single-user latency the target, not every serving regime. The paper trades repeated memory transfers for more compute inside each denoising step; it reports that autoregressive decoding starts to regain a throughput advantage at around 32 concurrent requests, while the diffusion mode gives up some quality against its AR initialization. 1

Start with the public Apache 2.0 checkpoint, google/diffusiongemma-26B-A4B-it, through the model card's Transformers example. 2 Replay a fixed slice of your own workload at batch 1 and your expected peak, then log task success, output length, p50/p95 latency, throughput, and stutter rate. Route only after that shadow test; keep the AR path as the rollback for long or complex tasks.