About Gemma 4 12B
Gemma 4 12B is the mainstream single-device model of Google DeepMind's open Gemma 4 family, aimed at consumer hardware and streamlined local execution under the Apache 2.0 licence. Its model card highlights an encoder-free multimodal design — images and audio are projected directly into the transformer rather than through a separate encoder stack — alongside the family's interleaved local-and-global attention.
Within the family it is the balance point: substantially more capable than the phone-and-laptop tier below it, while staying within reach of a single ordinary GPU, which makes it the default recommendation for local Gemma use.
Gemma 4 12B is Google's 12B-parameter model released in April 2026, with a 256K-token context window. It uses a hybrid-attention design — only a fraction of its layers cache the full context, so long conversations cost far less VRAM than a classic dense model: its KV cache is about 0.9 GB at an 8K context, 8.9 GB at 128K, and 17.5 GB at the full 256K window (FP16 cache).
At the community-standard quantisation the totals above land in mid-range gaming-GPU territory, and the family's attention design — a handful of global layers among many short-window local ones — keeps the KV cache small enough that the long context window doesn't blow the budget. Encoder-free multimodality has a practical memory benefit too: there is no separate vision tower to load alongside the language model. For hardware below the comfortable line, the E4B tier gives up capability gracefully; above it, the dense flagship is the step up. All the usual runtimes support it.
For most people Q4_K_M is the sweet spot — the most popular quality/size trade-off — while Q8 is near-lossless if you have the memory. Totals above include the KV cache and a realistic framework overhead, so they are what you should expect to see in practice rather than just the download size. Weight sizes are calibrated against real GGUF files — see the methodology.
Frequently asked questions
How much VRAM does Gemma 4 12B need?
At Q4_K_M with an 8K context, Gemma 4 12B needs about 10 GB (weights 7 GB + KV cache + overhead). The smallest common hardware that fits is a RTX 3060 12GB.
Can an RTX 4090 (24GB) run Gemma 4 12B?
Yes. An RTX 4090's 24 GB runs Gemma 4 12B at Q8_0 (about 16 GB at 8K context).
Can a Mac run Gemma 4 12B?
Yes — Apple Silicon with 16 GB of unified memory or more (macOS lets the GPU use ~75% of it, ~12 GB) runs Gemma 4 12B at Q4_K_M.
How much VRAM does Gemma 4 12B need at its full 256K context?
The KV cache grows with context: about 0.9 GB at 8K tokens versus 17.5 GB at the full 256K window (FP16 cache). Add that to the weights (7 GB at Q4_K_M) plus overhead — long contexts can cost more than a whole quantisation step.
Related
VRAM calculator for any model ·Token counter
Similar-size local models: Phi-4 (14.7B) · DeepSeek R1 Distill Qwen3 8B (8.2B) · Llama 3.1 8B (8.03B) · Gemma 4 E4B (8B) · GPT-OSS 20B (21B)
Last updated 2026-09-03. Architecture figures from the model's published config.json; see the methodology.