Can you run Qwen3.5 397B A17B locally?
Qwen3.5 family · 2026 · 403.4B parameters (17B activated per token) · Hugging Face model card
Qwen3.5 397B A17B is a mixture-of-experts model with 17B activated parameters per token and 397B-class total open weights (official card: Number of Parameters 397B in total and 17B activated; 512 experts, 10 routed + 1 shared activated; HF safetensors.total=403,397,928,944). It uses Gated Delta Networks combined with sparse MoE. Native context is 262,144 tokens (config.json max_position_embeddings; card: 262,144 natively, extensible to about 1,010,000), apache-2.0. Local VRAM fit uses the 17B active count, not the 397B dense-equivalent: roughly 16 GB at Q4-class (calculated). Weights still occupy ~400B-class disk, so multi-GPU serving is required. Measured speeds: not yet published here.
Minimum: 16 GB+ memory (Q4-class on 17B active, calculated) · Recommended: 24 GB+ for comfortable context headroom (Q4-class on 17B active, calculated)
How much VRAM does Qwen3.5 397B A17B need at each quantization?
Qwen3.5 397B A17B needs 266.7 GB of VRAM at Q4_K_M. The table below lists weights-only size and total VRAM including overhead for each common quantization level with 4k and 32k token contexts.
| Quantization | Bits / weight | Weights only | Total + KV @4k ctx | Total + KV @32k ctx |
|---|---|---|---|---|
| Q4_K_M | 4.8 | 242.0 GB | 266.7 GB | 270.3 GB |
| Q5_K_M | 5.7 | 287.4 GB | 316.7 GB | 320.2 GB |
| Q6_K | 6.6 | 332.8 GB | 366.6 GB | 370.1 GB |
| Q8_0 | 8.5 | 428.6 GB | 472.0 GB | 475.5 GB |
| FP16 | 16 | 806.8 GB | 888.0 GB | 891.5 GB |
Method: weights = parameters × bits-per-weight (Q4_K_M ≈ 4.8, Q5_K_M ≈ 5.7, Q6_K ≈ 6.6, Q8_0 ≈ 8.5, FP16 = 16), plus 10% loading overhead, plus KV cache from the verified architecture config (60 layers, 2 KV heads, 256 head dim). Source: HF-verified 2026-09-30 via api (gated=false params=403.4B safetensors.total=403397928944 ctx=262144 license=apache-2.0 repo=Qwen/Qwen3.5-397B-A17B@8472618112abcbd45acbcdc58436aff4233c23f7 layers=60 kv_heads=2). MoE: num_experts=512 num_experts_per_tok=10 active_params_b=17.0 (official README (397B total, 17B activated, 10 routed + 1 shared of 512)). VRAM figures are documented calculations, not measurements. No tok/s invented.
Which GPUs can run Qwen3.5 397B A17B locally?
At Q4_K_M with a 4k context, Qwen3.5 397B A17B needs 266.7 GB of VRAM. The lists below are computed live from our GPU database and grouped by how much headroom the card has. Disclosure: CompareAIHardware.com participates in the Amazon Associates program and earns from qualifying purchases through links on this page. Affiliate relationships do not influence our recommendations.
Minimum GPUs that fit Qwen3.5 397B A17B at Q4_K_M
These GPUs hold the model but leave little headroom — keep contexts short.
| GPU | VRAM | Bandwidth | Type | |
|---|---|---|---|---|
| AMD Instinct MI355X | 288 GB | 8000 GB/s | Data Center GPU | |
| NVIDIA B300 (Blackwell Ultra) | 288 GB | 8000 GB/s | Data Center GPU |
Needs 2+ GPUs to run Qwen3.5 397B A17B
One of these cards is too small on its own, but a pair (tensor or pipeline parallel, ~90% efficiency) covers the 266.7 GB requirement. See our multi-GPU guide for setup.
| GPU | VRAM | Bandwidth | Type | |
|---|---|---|---|---|
| AMD Instinct MI300X | 192 GB ×2 | 5300 GB/s | Data Center GPU | |
| NVIDIA B200 | 192 GB ×2 | 8000 GB/s | Data Center GPU | |
| AMD Instinct MI325X | 256 GB ×2 | 6000 GB/s | Data Center GPU |
Can a Mac run Qwen3.5 397B A17B?
Yes — these Apple Silicon machines fit Qwen3.5 397B A17B at Q4_K_M, since macOS lets the GPU use about 75% of unified memory. Generation speed is bound by memory bandwidth, so the GB/s column matters as much as capacity.
| Apple Silicon | Unified memory | Usable by GPU (~75%) | Bandwidth |
|---|---|---|---|
| M5 Ultra (Mac Studio) | 512 GB | 384 GB | 1200 GB/s |