Architecture truth card — diffusiongemma-26B-A4B-it
← Open the interactive VRAM & cost calculator on the homepage
Exact VRAM matrix — 3 precisions × 3 context lengths
| Precision | 8192 tokens ctx | 32768 tokens ctx | 131072 tokens ctx |
|---|---|---|---|
| FP16/BF16 16-bit | 48.10 / 1.88 / 3.91 Total 53.88 GiB | 48.10 / 7.50 / 3.91 Total 59.51 GiB | 48.10 / 30.00 / 3.91 Total 82.01 GiB |
| FP8/INT8 8-bit | 24.05 / 1.88 / 2.70 Total 28.63 GiB | 24.05 / 7.50 / 2.70 Total 34.25 GiB | 24.05 / 30.00 / 2.70 Total 56.75 GiB |
| INT4/AWQ 4.5-bit | 13.53 / 1.88 / 2.18 Total 17.58 GiB | 13.53 / 7.50 / 2.18 Total 23.20 GiB | 13.53 / 30.00 / 2.18 Total 45.70 GiB |
Each cell: Weights GiB / GQA KV Cache GiB / Runtime Overhead GiB; bold line = Total VRAM GiB. Concurrency = 1, FP16 KV cache (2 bytes), overhead = 1.5 + 0.05 × weights GiB.
Terminal speed-math — no install needed
Re-run this exact sizing from your own terminal:
curl -fsSL https://gpuvram.pages.dev/vram-cli.py | python3 - diffusiongemma-26b-a4b-it --ctx 32768
Live cheapest GPU configurations
| Scenario | Cheapest fitting rig (top 3) | Total VRAM | Lowest $/hr | 730 h/mo | Deploy |
|---|---|---|---|---|---|
| INT4 @ 32K ctx | 1x RTX A5000 24GB | 24 GiB | $0.16 | $116.80 | Deploy on RunPodCheck Vast.ai Verified |
| INT4 @ 32K ctx | 2x RTX A4000 16GB | 32 GiB | $0.18 | $129.94 | Deploy on RunPodCheck Vast.ai Verified |
| INT4 @ 32K ctx | 1x RTX 3090 24GB | 24 GiB | $0.22 | $160.60 | Deploy on RunPodCheck Vast.ai Verified |
| FP8 @ 32K ctx | 2x RTX A5000 24GB | 48 GiB | $0.32 | $233.60 | Deploy on RunPodCheck Vast.ai Verified |
| FP8 @ 32K ctx | 1x RTX A6000 48GB | 48 GiB | $0.33 | $240.90 | Deploy on RunPodCheck Vast.ai Verified |
| FP8 @ 32K ctx | 1x A40 48GB | 48 GiB | $0.35 | $255.50 | Deploy on RunPodCheck Vast.ai Verified |
| FP16 @ 32K ctx | 4x RTX A4000 16GB | 64 GiB | $0.36 | $259.88 | Deploy on RunPodCheck Vast.ai Verified |
| FP16 @ 32K ctx | 1x H200 NVL 143GB | 143 GiB | $0.50 | $365.00 | Deploy on RunPodCheck Vast.ai Verified |
| FP16 @ 32K ctx | 1x MI300X 192GB | 192 GiB | $0.50 | $365.00 | Deploy on RunPodCheck Vast.ai Verified |
What over-provisioning costs you
Running diffusiongemma-26B-A4B-it production traffic at INT4 / 32K context on the cheapest fitting rig (1x RTX A5000 24GB) bills $116.80 per month (730 h). Defaulting to full FP16 forces a larger rig — 4x RTX A4000 16GB at $259.88/mo. Paying $259.88 instead of $116.80 every single month is the price of skipping 4-bit AWQ quantization; if your quality budget allows INT4, that gap is pure waste.