Data table
Minimum GPU per model tier: the quick lookup
As of July 2026, at Q4 quantization and an 8k context the minimum workable hardware is: 8 GB VRAM for 8B models (12 GB comfortable), 16 GB for 14B and gpt-oss-20b, 24 GB for dense 27–32B, 48 GB for 70B, and 80 GB for gpt-oss-120b.
Model tier → minimum hardware table
"Estimated total" is quantized Q4 weights + FP16 KV cache at an 8,192-token context + typical runtime overhead, from our model VRAM requirement table. "Minimum" means the model loads with little headroom; "comfortable" leaves ≥15% free. Mac figures use the default macOS GPU share of unified memory (≈67–75%, see the Apple Silicon table).
| Model tier | Example models | Est. total @Q4, 8k (GB) | Minimum GPU VRAM (GB) | Comfortable GPU VRAM (GB) | Minimum Mac unified memory (GB) |
|---|---|---|---|---|---|
| 7–8B dense | Llama 3.1 8B, Qwen3 8B | 7.2–7.4 | 8 (tight) | 12 | 16 |
| 12–14B dense | Gemma 3 12B, Qwen3 14B, Phi-4 | 9.4–11.9 | 12 (tight) | 16 | 24 |
| ≈20B MoE | gpt-oss-20b (MXFP4) | ≈14.4 | 16 | 16 | 24 (tight) / 32 |
| 24–27B dense | Mistral Small 3.2, Gemma 3 27B | 16.9–18.9 | 20 | 24 | 32 (tight) / 48 |
| 30–32B | Qwen3 30B-A3B (MoE), Qwen3 32B | 20.6–23.1 | 24 (tight) | 32 | 48 |
| 70B dense | Llama 3.3 70B | ≈46.4 | 48 (2×24 or workstation) | 64+ | 64 (tight) / 96 |
| ≈120B MoE | gpt-oss-120b (MXFP4) | ≈65 | 80 (datacenter) | 96+ | 96 |
Sources & date: derived from published GGUF file sizes and model configuration files (per-model links on the model VRAM requirement table), vendor GPU specifications (GPU spec table), and Apple memory documentation (Apple Silicon table). Compiled 2026-07-13. Formula on the methodology page.
How to use the tiers
Read the table with your context length in mind. Every total assumes 8,192 tokens; the KV cache grows linearly with context, so running 32k instead of 8k adds roughly 3× the listed cache on full-attention models — enough to move a model up a whole hardware tier. Qwen3 32B goes from ≈23.1 GB at 8k (fits on 24 GB, barely) to ≈29.5 GB at 32k (RTX 5090 or Mac territory). If you routinely load long documents, size for the tier above your model, or pick an architecture with sliding-window attention (Gemma 3, gpt-oss) whose cache barely grows.
"Tight" entries deserve their label. A 24 GB card running Qwen3 32B has under 1 GB spare — fine on a headless box, but a 4k-monitor desktop compositor plus a browser can claim that much on the same GPU and abort the load. The same logic applies to Macs, where the default GPU share (about two-thirds of unified memory at 36 GB and below) is the real budget: a "32 GB" Mac exposes ≈21.3 GB to Metal, which is why the 24–27B tier lists 32 GB as tight rather than comfortable.
Speed is a separate axis from fit. Minimum-tier hardware runs the model but often on a narrow bus: the 16 GB RTX 4060 Ti (288 GB/s) loads everything the 16 GB RTX 5070 Ti (896 GB/s) loads, at roughly one-third the generation speed — see the measured tokens/sec table before deciding a minimum-tier card is enough. And at the 70B tier, note that meeting the capacity bar with two consumer cards yields 16–19 t/s, not double a single card's speed, because layer-split inference pays a PCIe penalty every token.
Frequently asked questions
What GPU do I need to run a 70B model locally?
48 GB of VRAM at Q4_K_M with an 8k context — in practice two 24 GB cards (RTX 3090/4090), a 48 GB workstation card like the RTX 6000 Ada, or a Mac with 64 GB or more of unified memory. No single consumer GPU, including the 32 GB RTX 5090, can load it.
Can an 8 GB GPU run an 8B model?
Yes, tightly. An 8B model at Q4_K_M needs about 7.2 GB in total at an 8k context, leaving under 1 GB of headroom on an 8 GB card — workable if the GPU is not driving a high-resolution desktop, but 12 GB is the comfortable tier.
What is the minimum GPU for a 32B model?
A 24 GB card (RTX 3090, RTX 4090, RX 7900 XTX), and it is tight: Qwen3 32B at Q4_K_M totals about 23.1 GB at an 8k context. The 32 GB RTX 5090 runs it with comfortable headroom, and longer contexts push past 24 GB quickly.
Can a 12 GB card like the RTX 3060 run a 13B or 14B model?
Yes at Q4 with a moderate context: a 14B model totals roughly 11.5 to 11.9 GB at 8k, which fits with almost no headroom. Drop to a 4k context or an IQ4/Q3 quant for margin, or step up to a 16 GB card to run 14B comfortably.
How much memory does gpt-oss-20b need?
About 14.4 GB in total, matching its published 16 GB guidance. That means a 16 GB GPU (RTX 5060 Ti 16GB and up) or a Mac with 24 GB of unified memory, where the default ~16 GB GPU share just accommodates it.