Can you run Qwen3.8 2.4T A95B locally?
Qwen3.8 family · 2026 · 2.45T parameters (95B activated per token) · Hugging Face model card
Qwen3.8 2.4T A95B is a mixture-of-experts model with 95B activated parameters per token and 2.4T-class total open weights (official card: Number of Parameters 2.4T in total and 95B activated; HF safetensors.total=2,446,182,725,504). Native context is 262,144 tokens (config.json max_position_embeddings), license other (Qwen research license family). Local VRAM fit uses the 95B active count, not the 2.4T dense-equivalent: roughly 64 GB at Q4-class (calculated). Weights still occupy ~2.4T-class disk, so multi-GPU serving is required. Measured speeds: not yet published here.
Minimum: 64 GB+ memory (Q4-class on 95B active, calculated) · Recommended: 96 GB+ for comfortable context headroom (Q4-class on 95B active, calculated)
How much VRAM does Qwen3.8 2.4T A95B need at each quantization?
Qwen3.8 2.4T A95B needs 1,616.0 GB of VRAM at Q4_K_M. The table below lists weights-only size and total VRAM including overhead for each common quantization level with 4k and 32k token contexts.
| Quantization | Bits / weight | Weights only | Total + KV @4k ctx | Total + KV @32k ctx |
|---|---|---|---|---|
| Q4_K_M | 4.8 | 1,467.7 GB | 1,616.0 GB | 1,626.8 GB |
| Q5_K_M | 5.7 | 1,742.9 GB | 1,918.7 GB | 1,929.5 GB |
| Q6_K | 6.6 | 2,018.1 GB | 2,221.5 GB | 2,232.3 GB |
| Q8_0 | 8.5 | 2,599.1 GB | 2,860.5 GB | 2,871.3 GB |
| FP16 | 16 | 4,892.4 GB | 5,383.1 GB | 5,394.0 GB |
Method: weights = parameters × bits-per-weight (Q4_K_M ≈ 4.8, Q5_K_M ≈ 5.7, Q6_K ≈ 6.6, Q8_0 ≈ 8.5, FP16 = 16), plus 10% loading overhead, plus KV cache from the verified architecture config (92 layers, 4 KV heads, 256 head dim). Source: HF-verified 2026-09-21 via api (gated=false params=2446.18B safetensors.total=2446182725504 ctx=262144 license=other repo=Qwen/Qwen3.8-2.4T-A95B@207bd685a7e3696cfaff12ded7c6a7ea0f88c996 layers=92 kv_heads=4). MoE: num_experts=512 num_experts_per_tok=10 active_params_b=95.0 (official README (2.4T total, 95B activated)). VRAM figures are documented calculations, not measurements. No tok/s invented.
Which GPUs can run Qwen3.8 2.4T A95B locally?
At Q4_K_M with a 4k context, Qwen3.8 2.4T A95B needs 1,616.0 GB of VRAM. The lists below are computed live from our GPU database and grouped by how much headroom the card has. Disclosure: CompareAIHardware.com participates in the Amazon Associates program and earns from qualifying purchases through links on this page. Affiliate relationships do not influence our recommendations.
No single consumer GPU in our database runs Qwen3.8 2.4T A95B at Q4_K_M. The 1,616.0 GB requirement exceeds every listed consumer card; see the Mac and data center options above and below.