Methodology

The three VRAM components

Every estimate on this site is the sum of three terms:

1. Model weights — either exact (a real synced .gguf file size from Hugging Face) or estimated as parameter_count × (bits_per_weight / 8) in generic mode.
2. KV cache — always estimated, using the formula below. This is the term most people underweight when sizing hardware.
3. Runtime overhead — a flat 10% of (weights + KV cache), a rule-of-thumb margin for CUDA context and activation buffers, not a measured figure.

The exact KV cache formula

kv_bytes = 2 × layers × kv_heads × head_dim × context_tokens × 2

The first 2 accounts for storing both keys and values. The final 2 is bytes per value at fp16 — the calculator does not model quantized KV cache, which some inference engines support as a separate setting. layers, kv_heads, and head_dim come from the matched reference architecture below, not from the specific model you selected.

The reference architectures

Four fixed architecture buckets back every KV cache estimate on this site. 7B, 13B, and 70B are real, published model specs. 120B has no real published reference model at that exact size and is extrapolated from the 70B architecture — not a measured spec.

Bucket	Layers	KV heads	Head dim
7B	32	8	128
13B	40	40	128
32B	64	8	128
70B	80	8	128
120B	88	8	128

GQA vs. MHA, and what this site actually does with it

Grouped-Query Attention (GQA) shares key/value projections across groups of query heads, which shrinks KV cache substantially versus classic Multi-Head Attention (MHA) at the same parameter count. The 7B and 70B reference buckets above use GQA-style numbers (matching Llama 3 8B and Llama 3 70B); the 13B bucket uses MHA-style numbers (matching Llama 2 13B), which is deliberately heavier on KV cache to represent an older architecture still in common use.

This site does not detect whether your specific model is actually GQA or MHA. For a model synced from Hugging Face, the calculator parses an approximate parameter count out of the model's name (for example, "8B" from "Meta-Llama-3-8B-Instruct") and matches it to the nearest bucket by size alone — not by reading the model's real config for num_attention_heads versus num_key_value_heads. If a specific model at, say, 8B parameters happens to use MHA instead of GQA, this calculator will still estimate its KV cache using the 7B bucket's GQA numbers, and the real figure would be meaningfully higher. This is a known, current limitation — the heuristic is "close for most current models at common sizes," not "verified per model."

When no estimate is shown

For some models, this site shows no total VRAM estimate at all — only the exact on-disk weight size per quantization. This happens when either of two conditions is true:

• Distance cap. If a model's parsed parameter count is more than 5B away from the nearest reference bucket, we don't force a match. A 32B model estimated with the 13B bucket's KV cache shape, for example, would produce a number that looks precise but likely understates real usage substantially — we'd rather show nothing.
• Mixture-of-experts (MoE) models. Models like Mixtral are excluded outright, regardless of parsed parameter count. MoE architectures don't scale the way the dense reference buckets assume — a name like "8x7B" parses as "7B" by our naming heuristic, but a dense 7B model's KV cache shape does not describe a MoE model's actual memory behavior.

In both cases, the exact synced weight size is still shown — only the KV cache estimate (and therefore the total) is withheld.

What's exact vs. estimated, summarized

Category	Includes
Exact	Synced model weight file size (model-specific mode only)
Estimated	Generic mode weight size; KV cache in both modes, always; runtime overhead (flat 10%); which reference architecture applies to a given synced model
Not shown	Total VRAM estimate for MoE models or models >5B from any reference bucket