Quick Answer
Google says a quantized DiffusionGemma configuration can fit within an 18GB VRAM target, but that is a model-fit statement—not a guarantee that every 18GB system will run the same prompt, context, multimodal input, runtime, or throughput.
Plan local deployment in this order:
- Pick the exact official checkpoint and license record.
- Choose a runtime that explicitly supports DiffusionGemma’s diffusion decoding.
- Select a quantized format supported by that runtime and GPU.
- reserve memory for context, caches, inputs, runtime kernels, and the operating system;
- start with Google’s recommended sampling configuration;
- benchmark a representative task against standard Gemma 4;
- accept the model only if quality, latency, memory, power, and review effort pass together.
DiffusionGemma is experimental. Google says it can be substantially faster on dedicated GPUs at low-to-medium concurrency, but it also says overall output quality is lower than standard Gemma 4. Use the DiffusionGemma model guide for the entity record and the diffusion vs autoregressive guide for the architecture decision.
Source check: Google’s model page, June 10 launch announcement, and developer overview were checked on July 30, 2026. Runtime support and checkpoint formats can change; confirm the current official model and runtime documentation before deployment.
Understand the Model Before Sizing Hardware
DiffusionGemma is a 26B-total sparse Mixture of Experts model built on Gemma 4 and Gemini Diffusion research. Google says it activates about 3.8B parameters during inference; its developer documentation rounds this to 4B active.
The model accepts text, image, and video inputs and generates text. Audio input is not supported in Google’s current developer overview.
Its decoder iteratively denoises 256-token blocks. That is materially different from a standard autoregressive model that emits the next token from left to right. A runtime must support the diffusion sampling path; loading a file format alone does not prove correct or optimized inference.
Interpret the 18GB VRAM Target Correctly
Google’s launch materials say a quantized configuration can fit within 18GB of VRAM on high-end consumer hardware. Record the qualifiers:
- quantized, not necessarily the original weight precision;
- model fit, not a promise about every context or workload;
- dedicated GPU, where the published local speed argument is strongest;
- runtime-dependent, because kernels, caches, allocation, and format support differ;
- experimental, with quality and compatibility still requiring testing.
Memory pressure can rise with:
- longer text prompts or outputs;
- image or video inputs;
- larger cache and context settings;
- higher-precision weights or activations;
- runtime workspace and kernel allocations;
- concurrent requests;
- display use on the same GPU;
- operating-system and background-process overhead.
Leave headroom. A model that loads with a tiny prompt can still fail or thrash under the representative workload.
Hardware Decision Matrix
| Environment | What the official sources establish | What you must measure |
|---|---|---|
| NVIDIA H100 | Google reports more than 1,000 output tokens per second in its measured setup | Exact precision, runtime, sampling, batch size, prompt/output shape, power, and accepted-result quality |
| RTX 5090 | Google reports more than 700 output tokens per second in its measured setup | Whether your quantized checkpoint, runtime, thermals, and workload reproduce a useful result |
| RTX 4090/5090 class | Google describes optimized quantized consumer configurations | Actual VRAM use, kernel support, speed, context, multimodal overhead, and stability |
| Apple silicon through MLX | Google lists MLX among supported development paths | Exact model artifact, memory pressure, throughput, compatibility, and quality |
| Model Garden or NVIDIA NIM | Google links managed or containerized deployment paths | Region, quota, billing, version, SLA, concurrency, data controls, and migration plan |
| Other GPUs or CPUs | No universal performance promise follows from the launch | Whether a supported runtime exists and whether latency, memory, and cost pass your acceptance bar |
Do not interpolate one GPU’s tokens-per-second number to another by comparing theoretical compute alone.
Choose Quantization as a Quality Tradeoff
Quantization reduces numerical precision to lower memory and often improve speed. It can also alter output quality, stability, and supported kernels.
Use this checklist:
- Start from an official checkpoint link. Record repository, revision, file, hash if available, and license.
- Choose a runtime-supported format. A format label is not useful if the selected runtime lacks DiffusionGemma decoding or optimized kernels.
- Record precision. Note weight and activation precision rather than saying only “4-bit.”
- Keep a higher-precision baseline. Compare representative outputs before accepting the smaller format.
- Test non-linear tasks. Include code infilling, editing, structured formatting, or another workload that could benefit from bidirectional refinement.
- Test failure cases. Include long prompts, multimodal inputs, formatting constraints, factual checks, and adversarial or malformed input.
- Measure accepted-result cost. Faster output is not cheaper if review or regeneration rises.
Google highlights native NVFP4 support on Blackwell GPUs. Treat that as a specific hardware and kernel path, not a generic promise that every 4-bit quantization behaves the same.
Start With Google’s Sampling Configuration
Google’s developer overview publishes a recommended starting configuration:
| Parameter | Official starting value | Why it matters |
|---|---|---|
| Maximum denoising steps | 48 | Caps refinement work; adaptive stopping can finish earlier |
| Temperature schedule | Linear 0.8 → 0.4 | Starts exploratory, then reduces variation as tokens settle |
| Adaptive early stopping | Entropy threshold 0.005 | Can stop when uncertainty is low and consecutive predictions stabilize |
| Token selection | Entropy bound 0.1 | Selects lower-uncertainty tokens and leaves others for later refinement |
These are starting values, not guaranteed best settings for every task. Change one parameter at a time and keep the same prompt set, checkpoint, quantization, runtime, and hardware while evaluating.
Pick a Supported Runtime
Google’s launch materials name:
- Hugging Face Transformers;
- vLLM;
- MLX;
- fine-tuning paths through Hackable Diffusion, Unsloth, and NVIDIA NeMo;
- deployment paths through Google’s Model Garden and NVIDIA NIM.
Google described official llama.cpp support as “arriving soon” in the June 10 announcement. Do not treat that dated future statement as proof of current support. Check the live upstream repository and DiffusionGemma documentation.
Before installing, confirm:
- the runtime names DiffusionGemma explicitly;
- the checkpoint revision matches the runtime integration;
- the quantization format and GPU architecture are supported;
- required kernels are present;
- the license and notices are carried into the deployment;
- multimodal input behavior is supported if you need images or video;
- the server exposes the sampling controls you plan to test.
Run a Representative Benchmark
Build a small acceptance set from real work:
| Test class | Example | Review |
|---|---|---|
| Inline editing | Rewrite one function without changing its interface | Tests bidirectional context, correctness, and edit scope |
| Code infilling | Fill a missing block between fixed prefix and suffix | Tests a task Google identifies as a diffusion strength |
| Structured text | Produce a closed JSON, Markdown table, or schema | Checks formatting convergence and factual correctness |
| Multimodal-to-text | Explain a supplied image or short video | Checks the input path, latency, and unsupported assumptions |
| Long-form answer | Produce a verified explanation with sources | Exposes coherence, factuality, and quality differences |
| Safety and robustness | Malformed, ambiguous, or adversarial input | Tests refusal, leakage, validation, and failure handling |
Capture:
- peak and steady VRAM;
- cold start and warm start;
- time to usable output;
- total generation time;
- tokens per second under a documented definition;
- GPU utilization and power;
- errors, retries, and out-of-memory events;
- task acceptance rate;
- human review and correction time.
Run the same set on standard Gemma 4 or the current production baseline. Google says standard Gemma 4 remains the quality recommendation, so it belongs in the comparison.
Account for Concurrency
Google says DiffusionGemma’s advantage is strongest for local, consumer-facing, low-concurrency use. A dedicated accelerator can work on a large token block at once.
At high request volume, autoregressive servers can batch many sequences and use hardware efficiently. Google says the diffusion advantage can diminish and serving cost can increase under high-QPS workloads.
Test at the intended concurrency:
- one interactive user;
- a small queue;
- the expected steady concurrency;
- a safe peak;
- recovery after an out-of-memory or worker restart.
Do not publish a single-user desktop result as a cloud-throughput forecast.
Production Gate
Do not ship because the model loads or reaches a high tokens-per-second value. Require:
- accepted output quality on representative tasks;
- memory headroom under worst-case supported input;
- stable runtime and checkpoint compatibility;
- bounded latency at expected concurrency;
- content and security review;
- correct handling of multimodal inputs and unsupported audio;
- license, notice, dependency, and model-use review;
- monitoring for failures, drift, and resource exhaustion;
- an autoregressive or other fallback;
- a documented upgrade and rollback path.
FAQ
How much VRAM does DiffusionGemma need?
Google says a quantized setup can fit within an 18GB VRAM target. Measure the exact checkpoint, quantization, runtime, context, input types, and system overhead.
Can DiffusionGemma run on an RTX 4090 or 5090?
Google describes optimized quantized configurations for high-end consumer NVIDIA GPUs and reports a measured RTX 5090 result. That does not guarantee every runtime or workload. Verify compatibility, memory, thermals, and quality.
Which quantization should I use?
Use a format explicitly supported by the current DiffusionGemma runtime and GPU. Keep a higher-precision baseline and compare task acceptance, not just file size or speed.
Is DiffusionGemma faster in the cloud?
Not universally. Google says the parallel-decoding advantage is strongest at low-to-medium batch sizes on one accelerator and can diminish in high-QPS batched serving.
Should I replace Gemma 4 with DiffusionGemma?
Only after a representative test. Google says DiffusionGemma’s overall quality is lower and recommends standard Gemma 4 when maximum production quality matters.
Bottom Line
Treat the 18GB and speed statements as hypotheses for a specific local build. Verify model revision, quantization, runtime support, sampling, memory headroom, concurrency, and accepted-task quality together.
For the model record, use the DiffusionGemma entity guide. For the architecture decision, continue to DiffusionGemma vs autoregressive generation.