Skip to content
VRAMwise

KV cache memory math: why context eats VRAM

Weights are only half the story. Context length multiplies a second memory term.

Fixed vs growing terms

GGUF weights are roughly fixed for a quant. KV cache grows with tokens. Doubling context roughly doubles that term when architecture numbers are fixed.

The formula we publish

kv_GB = kv_bytes_per_token × context / 1e9. Total ≈ (weights + kv) × (1 + overhead). Overhead covers runtime buffers; raise it for multi-batch or multimodal stacks.

Operational habit

Write down the max context you tested with FITS headroom. When a new model arrives, repeat the same context — do not assume the old number ports.

General information, not personalized advice.

Models in band
Llama 3.1 8B Instruct · Llama 3.3 70B Instruct · Qwen3 8B · Qwen3 14B · Qwen3 32B
Apple contrast
M1 · M1 Pro · M1 Max
Runtime tools
VRAM fit · Context VRAM · Quant compare
Catalog exits
Guides · Tools
Methodology paths
Context explorer · Fit checker · Context budgeting

Frequently asked questions

What is KV cache?

Key/value tensors kept per token so the model can attend over context without recomputing everything from scratch.

Why does long context break VRAM?

Cache size scales with token count. Weights stay fixed; KV grows.

How does VRAMwise estimate KV?

kv_bytes_per_token × context_tokens / 1e9 GB when architecture fields exist.

Can cache be quantized?

Some runtimes offer cache quant or sliding windows — re-measure after enabling them.

Advice?

Educational estimate framework.

KV Cache Memory Math: Why Context Eats VRAM — by VRAMwise ↗