How the estimate works
VRAM needed = model weights + KV cache + framework overhead. Weights depend on parameter count and quantisation — our bytes-per-parameter figures are measured from real GGUF files on Hugging Face, so Q4_K_M correctly shows ~5% larger than naive 4.5-bit maths. The KV cache depends on the model's attention architecture and your context length: dense GQA models grow linearly, while hybrid-attention models (Gemma 4, Qwen 3.5/3.6, GPT-OSS) and latent-compression models (DeepSeek V4, Mistral Large 3, GLM-5.2) cache dramatically less — this calculator uses each model's real architecture from its config.json. We then add ~10% plus 1GB for the inference framework. Full formulas on the methodology page.
Deliberately simple: one user, one machine, one clear answer. No batch sizes, no multi-GPU serving clusters — if you're planning production serving infrastructure you need a different tool.
A worked example
Take Llama 3.3 70B at Q4_K_M, the most popular quality/size trade-off, with an 8K context. The quantised weights come to about 43.1 GB, the KV cache adds about 2.7 GB, and framework overhead brings the total to roughly 51 GB — which is why "it's a 70.6B model, my card has enough for the download" regularly disappoints. The download size is only the first term of the sum. Every model in the dropdown gets this same arithmetic, tuned to its own architecture.
Quantisation, briefly
Quantisation stores weights at lower precision to shrink them: FP16 is the full-precision reference, Q8 is near-lossless at about half the size, and Q4_K_M — the community default — cuts memory to roughly a third with a quality loss most people don't notice in everyday use. Below Q4 the degradation becomes visible, which is why the calculator doesn't bother going lower. A newer wrinkle: some providers now ship weights already quantised — MXFP4 or FP8 "native" releases — in which case the shipped format is the one that matters and we mark it accordingly in the quantisation menu.
Why architecture changes the answer
Two models with identical parameter counts can need very different VRAM at long context, because the KV cache depends on the attention design. Classic dense models with grouped-query attention (DeepSeek R1 Distill Qwen3 8B, for instance) cache every layer, so memory grows linearly with context. Hybrid designs like Qwen3.5 4B give only a fraction of their layers full-context attention, keeping long conversations far cheaper. Latent-compression (MLA) designs like DeepSeek V4 Flash compress the cache itself and barely feel long contexts at all. The calculator reads each model's real attention geometry from its published config.json rather than assuming the worst case — at short contexts the difference is pocket change, at 128K it can be the difference between fitting and not.
Frequently asked questions
Can I run a 70B model on 24GB of VRAM?
Not fully on the GPU: Llama 3.3 70B needs about 48GB at Q4_K_M once you add the KV cache and overhead. On a 24GB card like an RTX 4090 or 3090 you can partially offload layers to system RAM at reduced speed, or run a smaller model — 27B–35B class models fit comfortably at Q4–Q5 on 24GB.
What is quantisation and how much quality do I lose?
Quantisation stores model weights at lower precision to shrink them. Q8 is near-lossless, Q6 and Q5_K_M lose very little, and Q4_K_M — the most popular size/quality trade-off — is still close to the original for most uses. Below Q4 degradation becomes noticeable. Our size figures use the actual measured file sizes of each quantisation format, not theoretical bit-widths.
Why does context length change how much VRAM I need?
The model keeps a KV cache — attention keys and values for every token in the conversation — in VRAM alongside the weights. For classic dense models it grows linearly with context, which can add many gigabytes at long contexts. Many 2026 models (Qwen 3.5/3.6, Gemma 4, GPT-OSS, DeepSeek V4) use hybrid or compressed attention that keeps the cache far smaller — this calculator models each architecture correctly rather than assuming the worst.
How accurate is this calculator?
Weight sizes are tuned against real GGUF files published on Hugging Face and land within about 1% of the actual downloads. KV-cache maths follows each model's real attention architecture from its config.json, and the total includes a typical inference-framework overhead of about 10% plus 1GB. Treat the result as a realistic planning figure: actual usage varies slightly by runtime and settings.
How much of my Mac's unified memory can the GPU use?
By default macOS lets Metal use roughly 75% of unified memory for the GPU — about 96GB on a 128GB machine. That is the figure this calculator uses for Apple Silicon. Advanced users can raise the limit with the iogpu.wired_limit_mb sysctl, but you still need to leave several GB for macOS itself.
Do MoE models like GPT-OSS or Qwen A3B need less VRAM?
Not for the weights: a mixture-of-experts model must hold all experts in memory, so its total parameter count decides VRAM, not the smaller "active" count. The active count is what makes them fast. GPT-OSS 120B still needs a ~80GB-class device even though only 5.1B parameters are active per token.
What if the model doesn't fit — can I still run it?
Usually, just slowly. Runtimes like llama.cpp, Ollama and LM Studio can keep some layers in system RAM and stream them through the GPU, so a model a few gigabytes over your VRAM still runs at reduced speed — but generation slows sharply as more of the model falls off the card. The better fixes are a smaller quantisation, a shorter context, or a smaller model from the same family; the calculator suggests the best quantisation that fully fits.
Does it matter whether my GPU is NVIDIA, AMD or Apple?
Not for the memory maths — a gigabyte of weights is the same size everywhere, so the fits-or-doesn't verdict holds across vendors. What differs is software support and speed: NVIDIA's CUDA has the broadest runtime support, AMD cards run through ROCm or Vulkan, and Apple Silicon uses Metal with unified memory shared between CPU and GPU. All the mainstream local runtimes support all three today; check your specific runtime before buying hardware.
Per-model VRAM guides
DeepSeek: DeepSeek V4 Flash · DeepSeek R1 Distill Qwen3 8B
Mistral: Mistral Large 3 · Mistral Small 4
Meta: Llama 3.1 8B · Llama 3.3 70B · Llama 4 Scout · Llama 4 Maverick
Alibaba: Qwen3.5 4B · Qwen3.6 27B · Qwen3.6 35B-A3B · Qwen3.5 397B-A17B · Qwen3 Coder 30B-A3B
Google: Gemma 4 E4B · Gemma 4 12B · Gemma 4 26B-A4B · Gemma 4 31B
OpenAI: GPT-OSS 20B · GPT-OSS 120B
Z.ai: GLM-5.2
Microsoft: Phi-4
More tools
Token counter: count tokens and input costs for any text. API pricing calculator: what the hosted alternatives cost per request and per month.
Last updated 2026-09-03. Model architecture verified against each model's published config.json; quantisation sizes verified against actual GGUF files; see the methodology.