Skip to content
VRAMwise

Context budgeting for local LLMs

Context is a budget line, not a trophy setting.

Start from the task

Customer-support style chat may live at 4–8k. Agent workflows that stuff docs need more — and more KV.

Measure twice

Set quant and GPU first. Raise context only while the verdict stays FITS with comfortable headroom.

Write it down

Record model, quant, context, GPU, and measured tok/s. That note is more valuable than another undated Reddit screenshot.

General information, not personalized advice.

Models in band
Llama 3.1 8B Instruct · Llama 3.3 70B Instruct · Qwen3 8B · Qwen3 14B · Qwen3 32B
Apple contrast
M1 · M1 Pro · M1 Max
Runtime tools
VRAM fit · Context VRAM · Quant compare
Catalog exits
Guides · Tools
Methodology paths
Context explorer · Fit checker · KV math

Frequently asked questions

How much context do I need?

From the task: short chat vs whole-repo coding differ by orders of magnitude.

What is headroom?

VRAM left after weights+KV+overhead. Low headroom breaks when you add tools or longer prompts.

Should I max the model window?

Usually no. Max windows are capability ceilings, not free resources.

How to test?

Fit checker + context explorer; raise tokens until TIGHT/WON'T FIT.

Advice?

Operational guide only.

Context Budgeting for Local LLMs — by VRAMwise ↗