Data table
GPU VRAM & memory bandwidth: current consumer cards
As of July 2026, consumer GPU memory runs from 8 GB (RTX 5060, RTX 4060) to 32 GB (RTX 5090), and peak bandwidth from 272 GB/s to 1,792 GB/s — the two numbers that decide which local LLMs fit and how many tokens per second they generate.
Consumer GPU memory specification table
Capacity determines whether a model fits; bandwidth determines how fast it generates once it does. Figures are vendor-published theoretical peaks, cross-checked against the TechPowerUp GPU database. Machine-readable copy: gpus.json.
| GPU | Vendor | VRAM (GB) | Peak bandwidth (GB/s) | Memory type |
|---|---|---|---|---|
| GeForce RTX 5090 | NVIDIA | 32 | 1,792 | GDDR7 |
| GeForce RTX 5080 | NVIDIA | 16 | 960 | GDDR7 |
| GeForce RTX 5070 Ti | NVIDIA | 16 | 896 | GDDR7 |
| GeForce RTX 5070 | NVIDIA | 12 | 672 | GDDR7 |
| GeForce RTX 5060 Ti 16GB | NVIDIA | 16 | 448 | GDDR7 |
| GeForce RTX 5060 Ti 8GB | NVIDIA | 8 | 448 | GDDR7 |
| GeForce RTX 5060 | NVIDIA | 8 | 448 | GDDR7 |
| GeForce RTX 4090 | NVIDIA | 24 | 1,008 | GDDR6X |
| GeForce RTX 4080 SUPER | NVIDIA | 16 | 736 | GDDR6X |
| GeForce RTX 4080 | NVIDIA | 16 | 717 | GDDR6X |
| GeForce RTX 4070 Ti SUPER | NVIDIA | 16 | 672 | GDDR6X |
| GeForce RTX 4070 SUPER | NVIDIA | 12 | 504 | GDDR6X |
| GeForce RTX 4070 | NVIDIA | 12 | 504 | GDDR6X |
| GeForce RTX 4060 Ti 16GB | NVIDIA | 16 | 288 | GDDR6 |
| GeForce RTX 4060 | NVIDIA | 8 | 272 | GDDR6 |
| GeForce RTX 3090 | NVIDIA | 24 | 936 | GDDR6X |
| GeForce RTX 3080 10GB | NVIDIA | 10 | 760 | GDDR6X |
| GeForce RTX 3060 12GB | NVIDIA | 12 | 360 | GDDR6 |
| Radeon RX 9070 XT | AMD | 16 | 645 | GDDR6 |
| Radeon RX 9070 | AMD | 16 | 645 | GDDR6 |
| Radeon RX 9060 XT 16GB | AMD | 16 | 320 | GDDR6 |
| Radeon RX 7900 XTX | AMD | 24 | 960 | GDDR6 |
| Radeon RX 7900 XT | AMD | 20 | 800 | GDDR6 |
| Radeon RX 7800 XT | AMD | 16 | 624 | GDDR6 |
| Radeon RX 7600 XT | AMD | 16 | 288 | GDDR6 |
| Arc B580 | Intel | 12 | 456 | GDDR6 |
| Arc A770 16GB | Intel | 16 | 560 | GDDR6 |
Sources & date: vendor specification pages — NVIDIA GeForce specs, AMD Radeon specs, Intel Arc specs — cross-checked against the TechPowerUp GPU database. Retrieved 2026-07-13. Bandwidth figures are theoretical peaks in decimal GB/s.
Capacity: the pass/fail spec
VRAM capacity is binary for local inference: either the model's weights, KV cache, and runtime overhead fit, or the load fails (or silently spills into system RAM at a large speed penalty). Using the totals from our model VRAM requirement table, the practical tiers are: 8 GB runs 7–8B models tightly, 12 GB runs them comfortably and reaches 12–14B, 16 GB covers 14B and MoE models like gpt-oss-20b, 24 GB covers dense 27–32B, and a 70B model needs 48 GB — which no single consumer card offers, hence dual-card and Mac setups.
Note the capacity traps in the current lineup. Two cards with the same name differ only in memory: the RTX 5060 Ti ships in 8 GB and 16 GB versions on the same 448 GB/s bus, and for local LLM use the 16 GB version is a different class of card — it holds gpt-oss-20b and 14B models that the 8 GB version cannot load at all. Meanwhile the RTX 4060 Ti 16GB and RX 7600 XT offer generous capacity on very narrow buses (288 GB/s), so they fit mid-size models but generate at roughly a third of the speed of a 24 GB flagship.
Bandwidth: the speed spec
Once a model fits, generation speed is governed almost entirely by memory bandwidth, because producing each token requires streaming the active weights through the GPU. A useful first-order estimate is tokens/sec ≈ bandwidth × efficiency ÷ bytes read per token, with measured llama.cpp efficiency on CUDA cards typically around 50–70% of the theoretical peak. That model matches observation well: the 1,008 GB/s RTX 4090 measures about 128 t/s on an 8B Q4 model while the 936 GB/s RTX 3090 measures about 112 t/s — an 8% bandwidth gap producing a 14% speed gap, with the newer architecture making up the difference. See the measured tokens/sec table for the full benchmark set.
This is also why compute-focused marketing numbers (TFLOPS, AI TOPS) are poor predictors of local-LLM generation speed. Prompt processing is compute-bound and does track shader throughput, but the part you wait on — token-by-token generation — is a memory streaming workload. When comparing two cards for local inference, compare the bandwidth column first, then capacity against the models you actually intend to run; the minimum GPU per model tier table combines both into one lookup.
Frequently asked questions
How much VRAM does the RTX 4090 have?
24 GB of GDDR6X with 1,008 GB/s of peak memory bandwidth. That fits dense models up to about 32B parameters at Q4 quantization with an 8k context, but a 70B model does not fit on a single card.
Why does memory bandwidth matter for local LLMs?
Token generation is memory-bound: every token requires reading essentially all active model weights from VRAM. Generation speed scales roughly with bandwidth ÷ model size, which is why a 1,008 GB/s RTX 4090 generates about 4x faster than a 272 GB/s RTX 4060 on the same model.
Which consumer GPU has the most VRAM in 2026?
The GeForce RTX 5090 with 32 GB of GDDR7 and 1,792 GB/s of bandwidth. Among previous-generation cards, the RTX 4090, RTX 3090, and Radeon RX 7900 XTX each offer 24 GB.
Is 8 GB of VRAM enough to run a local LLM?
Yes, for small models. An 8B model at Q4_K_M needs about 7.2 GB in total at an 8k context, which is tight but workable on an 8 GB card if the GPU is not also driving a high-resolution desktop. 12 GB is the comfortable floor for 8B-class models.
Do AMD and Intel GPUs work for local LLM inference?
Yes. llama.cpp supports AMD via ROCm or Vulkan and Intel Arc via SYCL or Vulkan. The same VRAM math applies; the practical differences are software maturity and typically lower bandwidth efficiency than CUDA cards.