Qwen3.6-27B on an RTX 4090: Which Quant Fits

👁 117 reads

Qwen3.6-27B has been popping up in local AI threads as a coding assistant that fits on a single RTX 4090. It is an open model from Alibaba’s Qwen team under Apache 2.0, which allows commercial use, modification and redistribution. Below: what the model is, why its memory footprint at long context is unusually small, and which quantization to pick for a 24 GB card. We have not measured its speed, so there are no tokens-per-second numbers here - only file sizes, config values and our own VRAM math.

First, the date

This is not an October release. The Hugging Face repo Qwen/Qwen3.6-27B was created on April 21, 2026, and its files were last changed on April 24, 2026. The model has been around for six months, with 754 community quantizations listed under its model card and its own tag family in Ollama. What is new is that more people now use it as a local coding assistant.

Dense, not MoE

The model has 27 billion parameters (27.8B in BF16) and is dense: every parameter is active for every token. Its sibling, Qwen3.6-35B-A3B, is a mixture-of-experts model with about 3B active out of 35B. The MoE version can be faster, but it needs more memory for weights - a bad trade on a 24 GB card if you also want long context.

DeltaNet: why the KV cache stays small

Qwen3.6-27B has 64 layers arranged in 16 blocks. Each block has three Gated DeltaNet layers and one standard attention layer. DeltaNet is a form of linear attention: instead of storing every token, it keeps a fixed-size state. That means only 16 of the 64 layers carry the KV cache that grows with context length. In a model built only from standard attention layers, all 64 would.

The model reads text, images and video. Native context is 262,144 tokens, extendable to about 1.01 million with YaRN, though Qwen warns that extending can hurt results on short texts. Qwen also recommends at least 128K of context to preserve the model’s thinking. That recommendation shapes the whole VRAM question below.

Vendor benchmarks

These numbers come from Qwen’s model card. We have not seen independent replication.

The reading: in agentic coding, the 27B model catches its 397B predecessor and ties Claude Opus 4.5 on Terminal-Bench. But Opus 4.5 is an older generation; the current Opus 5.5 is not in the table. General knowledge, measured by MMLU-Pro, is flat against Qwen3.5-27B. On code it is far ahead of Gemma 4 31B, which we covered in Gemma 4 31B on an RTX 4090.

The VRAM math (our calculation)

This is a calculation from file sizes and the model config, not a measurement. Weights are the real GGUF sizes from unsloth/Qwen3.6-27B-GGUF in decimal GB. The RTX 4090’s 24 GiB is about 25.8 GB.

KV cache: 16 full-attention layers, 4 KV heads, head dimension 256. In FP16 that is 64 KiB per token: 2.15 GB at 32K, 8.59 GB at 128K and 17.18 GB at the full 262K. A q8_0 cache roughly halves that: 1.14, 4.56 and 9.13 GB. The DeltaNet fixed state adds about 0.15 GB. We assume a 1.5 GB reserve for the driver, the Windows desktop and compute buffers, which leaves about 24.3 GB usable.

The vision module (mmproj, F16) adds 0.93 GB and is only needed if you feed the model images or screenshots. For background on the quant names, see our guide to VRAM for local models.

Our pick

Q4_K_M or UD-Q4_K_XL with 128K context and a q8_0 KV cache. That meets Qwen’s context advice for thinking and leaves a few GB spare. Q5_K_M makes sense for shorter tasks. Q6_K means giving up long context, which in practice means giving up agent work on a whole repository.

Settings: Ollama and llama.cpp

In Ollama, the default tag qwen3.6:27b is 18-19 GB and qwen3.6:27b-q4_K_M is 17 GB. For long context, enable flash attention and the q8_0 cache, and set the length through an environment variable:

OLLAMA_FLASH_ATTENTION=1
OLLAMA_KV_CACHE_TYPE=q8_0
OLLAMA_CONTEXT_LENGTH=131072
ollama run qwen3.6:27b-q4_K_M

In llama.cpp, the equivalent flags are:

-c 131072 -fa on -ctk q8_0 -ctv q8_0

New to Ollama? Our Ollama install and commands guide covers the basics, and more local model write-ups live under local AI.

FAQ

Will it run on a 16 GB card?

Only in heavily compressed variants. Q3_K_M at 13.59 GB leaves little room for context. UD-IQ2_M at 10.85 GB leaves more, at a cost in quality.

Can I run the full BF16 model?

Not on a single consumer card. Ollama’s bf16 variant is 56 GB.

How fast is it on a 4090?

We have not measured it, so we will not quote a number.

Is the 35B-A3B version better for a 4090?

It can be faster because only about 3B parameters are active, but its weights take more memory, which eats into the room for long context.

Can I use it commercially?

Yes. Apache 2.0 allows commercial use, modification and redistribution.

Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →

Sources - checked on November 1, 2026

• Hugging Face: Qwen/Qwen3.6-27B model card (accessed November 1, 2026)
• Hugging Face API: Qwen3.6-27B repo metadata (accessed November 1, 2026)
• Hugging Face: Qwen3.6-27B config.json (accessed November 1, 2026)
• Hugging Face: unsloth/Qwen3.6-27B-GGUF (accessed November 1, 2026)
• Ollama: qwen3.6 tags (accessed November 1, 2026)
• Gigazine: Qwen3.6-27B (April 23, 2026)

Kinetyra