About Qwen3.5 4B
Qwen3.5 4B is the edge-scale entry of Alibaba's Qwen3.5 generation, released open under Apache 2.0 with the architectural signature that defines the line: most layers use linear-attention Gated DeltaNet blocks whose memory state stays constant regardless of conversation length, with occasional full-attention layers interleaved for quality. Configurable thinking modes — deliberate reasoning on demand, direct answers otherwise — come standard even at this size.
Alibaba aims the small tiers at on-device assistants, tool calling and lightweight agent work, where the combination of tiny footprint and long usable context is the selling point.
Qwen3.5 4B is Alibaba's 4B-parameter model released in February 2026, with a 256K-token context window. It uses a hybrid-attention design — only a fraction of its layers cache the full context, so long conversations cost far less VRAM than a classic dense model: its KV cache is about 0.3 GB at an 8K context, 4.3 GB at 128K, and 8.6 GB at the full 256K window (FP16 cache).
This model runs on nearly anything — integrated graphics, several-year-old GPUs, modest laptops — and the interesting number is not the weights but the cache: with only a quarter of its layers holding conventional attention state and the rest maintaining constant-size linear-attention state, context that would swamp a classic design of this size costs almost nothing here. That makes it a genuine long-document model on hardware where that is normally impossible. Leave headroom for thinking mode, which spends context on visible deliberation, and expect support across every mainstream runtime.
For most people Q4_K_M is the sweet spot — the most popular quality/size trade-off — while Q8 is near-lossless if you have the memory. Totals above include the KV cache and a realistic framework overhead, so they are what you should expect to see in practice rather than just the download size. Weight sizes are calibrated against real GGUF files — see the methodology.
Frequently asked questions
How much VRAM does Qwen3.5 4B need?
At Q4_K_M with an 8K context, Qwen3.5 4B needs about 4 GB (weights 2 GB + KV cache + overhead). The smallest common hardware that fits is a RTX 3060 12GB.
Can an RTX 4090 (24GB) run Qwen3.5 4B?
Yes. An RTX 4090's 24 GB runs Qwen3.5 4B at FP16 / BF16 (about 10 GB at 8K context) — at full FP16 precision.
Can a Mac run Qwen3.5 4B?
Yes — Apple Silicon with 16 GB of unified memory or more (macOS lets the GPU use ~75% of it, ~12 GB) runs Qwen3.5 4B at Q4_K_M.
How much VRAM does Qwen3.5 4B need at its full 256K context?
The KV cache grows with context: about 0.3 GB at 8K tokens versus 8.6 GB at the full 256K window (FP16 cache). Add that to the weights (2 GB at Q4_K_M) plus overhead — long contexts can cost more than a whole quantisation step.
Related
VRAM calculator for any model ·Token counter
Similar-size local models: Gemma 4 E4B (8B) · Llama 3.1 8B (8.03B) · DeepSeek R1 Distill Qwen3 8B (8.2B) · Gemma 4 12B (12B) · Phi-4 (14.7B)
Last updated 2026-09-03. Architecture figures from the model's published config.json; see the methodology.