Data table
Model VRAM requirements: popular local LLMs × quantization
As of July 2026, an 8B model at Q4_K_M needs about 7 GB of VRAM at an 8k context, a 32B model needs about 23 GB, and a 70B model needs about 46 GB — in each case the quantized weights plus KV cache plus roughly 1.2 GB of runtime overhead.
VRAM requirement matrix
Weight sizes are the published GGUF file sizes from each model's quantization repository. The KV cache column uses an FP16 cache at an 8,192-token context; the estimated total adds a typical 1.2 GB of runtime overhead. Formula and variable definitions are on the methodology page.
| Model | Params (B) | Q4_K_M weights (GB) | Q5_K_M weights (GB) | Q8_0 weights (GB) | KV cache @8k, FP16 (GB) | Est. total @8k, Q4 (GB) | Min VRAM tier (GB) |
|---|---|---|---|---|---|---|---|
| Llama 3.1 8B Instruct | 8.0 | 4.92 | 5.73 | 8.54 | 1.07 | ≈7.2 | 8 (tight) / 12 |
| Qwen3 8B | 8.2 | 5.03 | 5.85 | 8.71 | 1.21 | ≈7.4 | 8 (tight) / 12 |
| Gemma 3 12B IT | 12.2 | 7.30 | 8.44 | 12.51 | ≈0.9 * | ≈9.4 | 12 |
| Qwen3 14B | 14.8 | 9.00 | 10.51 | 15.70 | 1.34 | ≈11.5 | 12 (tight) / 16 |
| Phi-4 (14.7B) | 14.7 | 9.05 | 10.60 | 15.58 | 1.68 | ≈11.9 | 12 (tight) / 16 |
| gpt-oss-20b (MoE) | 21.0 | 12.9 (MXFP4) † | — | — | ≈0.3 * | ≈14.4 | 16 |
| Mistral Small 3.2 24B | 24.0 | 14.33 | 16.76 | 25.05 | 1.34 | ≈16.9 | 20 |
| Gemma 3 27B IT | 27.4 | 16.55 | 19.27 | 28.71 | ≈1.1 * | ≈18.9 | 20 (tight) / 24 |
| Qwen3 30B-A3B (MoE) | 30.5 | 18.63 | 21.74 | 32.48 | 0.81 | ≈20.6 | 24 |
| Qwen3 32B | 32.8 | 19.76 | 23.21 | 34.82 | 2.15 | ≈23.1 | 24 (tight) |
| Llama 3.3 70B Instruct | 70.6 | 42.52 | 49.95 | 74.98 | 2.68 | ≈46.4 | 48 (2×24) |
| gpt-oss-120b (MoE) | 117 | 63.4 (MXFP4) † | — | — | ≈0.5 * | ≈65 | 80 |
* Gemma 3 interleaves sliding-window and global attention layers, and gpt-oss alternates dense and 128-token sliding-window attention, so their KV caches are far smaller than the full-attention formula; figures shown are estimates for those architectures. † gpt-oss models ship pre-quantized in MXFP4 (~4.25 bits/weight); no separate Q4/Q5/Q8 tiers apply.
Sources & date: weight sizes read from the published GGUF repositories on Hugging Face (Llama 3.1 8B, Llama 3.3 70B, Qwen3, Gemma 3, Mistral Small 3.2, Phi-4, gpt-oss); KV cache computed from each model's official configuration file. Retrieved 2026-07-13.
How to read this table
The number that matters is the estimated total, not the file size. A quantized model file only accounts for the weights; once loaded, the runtime also allocates a KV cache that grows linearly with context length, plus working buffers. The gap is small for 8B models (about 2.3 GB at 8k context) but it is exactly the gap that makes people's downloads fail to load: a 19.76 GB Qwen3 32B file looks like it fits on a 24 GB card with room to spare, yet at an 8k context the real footprint is about 23 GB — workable, but with almost no headroom for the desktop compositor or longer prompts.
Context length is the hidden variable. Every KV figure above assumes 8,192 tokens; doubling the context roughly doubles that column for full-attention models. Qwen3 32B at a 32k context needs about 8.6 GB of KV cache instead of 2.15 GB, pushing the total past 29 GB — beyond any single consumer card except a 32 GB RTX 5090. If you routinely run long contexts, check the methodology page for the per-token formula and size the cache for the context you actually use, not the model's advertised maximum.
Architecture choices also shift the picture in ways parameter count alone does not capture. Grouped-query attention (8 KV heads instead of 32+) is why modern 8B models cache only ~131 KB per token. Gemma 3's sliding-window layers cut its cache by roughly 70% at 8k context. MoE models such as Qwen3 30B-A3B and gpt-oss need the capacity of their full file size but read only a few billion active parameters per token, so they generate several times faster than dense models of equal size — see the measured tokens/sec table for how that plays out on real hardware, and the GPU spec table to match totals against cards. For a shortcut, the minimum GPU per model tier table collapses this matrix into buying tiers.
Frequently asked questions
How much VRAM do I need to run a 7B or 8B model?
About 7 to 7.5 GB in total at Q4_K_M with an 8k context: roughly 5 GB of weights, 1 to 1.2 GB of KV cache, and about 1.2 GB of runtime overhead. That is tight on an 8 GB card and comfortable on 12 GB.
How much VRAM does a 70B model need at Q4?
Around 46 GB at Q4_K_M with an 8k context — 42.5 GB of weights plus about 2.7 GB of KV cache and overhead. That means two 24 GB cards, a 48 GB workstation GPU, or a Mac with 64 GB or more of unified memory.
Is the GGUF file size the same as the VRAM requirement?
No. The file size covers only the weights. You also need the KV cache, which grows with context length, plus roughly 0.5 to 2 GB of runtime buffers. A 4.9 GB Q4 file typically needs about 7 GB of memory at an 8k context.
Why do Gemma 3 models need less KV cache than the formula suggests?
Gemma 3 interleaves five local sliding-window attention layers (1,024-token window) with one global layer, so most layers cache only the last 1,024 tokens. At an 8k context the 27B model uses roughly 1.1 GB of KV cache instead of the 4+ GB full attention would need.
Do MoE models like Qwen3 30B-A3B need less VRAM than dense models?
No — all expert weights must sit in memory, so the 18.6 GB Q4 file needs the same residency as a dense model of that size. The benefit is speed: only about 3.3B parameters are read per token, so generation is much faster than a dense 30B.