Can You Run Qwen3.8-Flash-Next Locally?
Published August 27, 2026. Model facts are from the official Hugging Face model card and Qwen's technical report; hardware specifications come from our GPU database.
Short answer: yes — this is the most self-hostable model of the August 2026 frontier releases. Qwen3.8-Flash-Next activates only 6B parameters per token, and its quantized builds start near 73 GB. A 128 GB unified-memory mini-PC runs mid quants; a 256 GB Mac Studio runs the recommended Q4_K_M with room to spare; and with expert offloading even a 16–32 GB GPU in a desktop with 64–128 GB of system RAM runs it — one community report measured 20.4 tokens per second on an RTX 5070 Ti with 85 GB of RAM. This guide gives the memory tiers, the hardware that fits each one, expected speeds, and the commands to serve it.
On this page
What is Qwen3.8-Flash-Next?
Qwen3.8-Flash-Next, published on Hugging Face on August 24, 2026, is an experimental preview of the architecture that will underpin Qwen4. It is a causal language model with a vision encoder — it accepts images and video alongside text — and it rethinks how the core components of an LLM interact at scale.
Four architecture changes define it, per Qwen's model card and technical report:
- Hybrid attention with QSA. The Gated DeltaNet and Gated Attention pairing is reworked into Gated DeltaNet plus Qwen Sparse Attention, which operates at the micro-block level rather than selecting individual tokens — cutting long-context latency for agentic workloads.
- Gated Residual. Residual streams are modulated by a data-dependent read gate and a per-branch scalar write gate across four branches, adding expressiveness while keeping inference overhead low.
- N-gram embedding. A 51B-parameter embedding table covering 20 million bigrams and trigrams, indexed at layer 2. Qwen frames this as a parameter-scaling axis that needs less compute than MoE and is more amenable to offloading — a direct benefit for memory-constrained machines.
- Tailored training recipe. Muon and AdamW applied to specific weight categories, with refitted scaling laws that eliminate batch-size warmup.
In Qwen's vendor-reported benchmarks the model beats DeepSeek-V4-Flash-0731 on DeepSWE 1.1 (58.7 vs 54.4) and posts 62.5 on SWE-bench Pro, 91.7 on GPQA Diamond, and 91.9 on LiveCodeBench v6 — competitive with far larger models given the 6B active count.
Specifications at a glance
Verified configuration details from the official model card: hidden dimension 2,560; a 12-block layout of three Gated DeltaNet→MoE groups per one Qwen Sparse Attention→MoE group; 512 experts with 10 routed plus 1 shared activated per token (expert dimension 640); Gated DeltaNet with 48 V-heads and 16 QK-heads at dimension 128; QSA with 24 query and 2 KV heads at dimension 256 and a 512-block (2,048-token) indexer budget; and a 1-layer MTP module. Local tokens-per-second figures have not been published; the 6B active count is what makes CPU-offloaded and unified-memory inference practical.
Memory requirements by quantization
Total parameter mass is 180B (125B core + 51B n-gram + 4B MTP), so the BF16 checkpoint is about 360 GB. Community GGUF builds (measured file sizes from the Unsloth and DevQuasar quantizations):
| Format | Size on disk | Realistic minimum memory | Quality |
|---|---|---|---|
| UD-IQ1_S (Unsloth) | ~73 GB | ~83 GB incl. overhead + KV | Heavy compression; testing only |
| UD-IQ1_M (Unsloth) | ~75 GB | ~85 GB | Heavy compression |
| Q2_K | ~80 GB | ~92 GB | Noticeable degradation |
| UD-Q3_K_XL (Unsloth) | ~90 GB | ~103 GB | Sweet spot for 128 GB machines |
| Q4_K_M | ~119 GB | ~136 GB | Recommended baseline quality |
| Q5_K_M | ~134 GB | ~154 GB | High quality |
| Q6_K | ~168 GB | ~193 GB | Near-lossless |
| Q8_0 | ~188 GB | ~216 GB | Lossless-ish |
| BF16 (official checkpoint) | ~360 GB | ~414 GB | Reference quality, vLLM/SGLang |
Add the vision projector (~0.9 GB F16 mmproj) for image and video input, plus KV cache for your working context. The sparse-attention indexer budget (512 blocks / 2,048 tokens) keeps long-context cache growth modest, and YaRN extends the native 262K window to 1M tokens when you need it.
Total parameter mass is 180B, but only 6B are read per token — and Qwen designed the 51B n-gram table to be offload-friendly. That splits hardware into three tiers. Tier 1 — fits entirely in VRAM or unified memory: fastest and simplest. Tier 2 — expert offload: a 16–32 GB GPU plus a desktop with enough system RAM; the GPU holds the always-active layers, active experts, and KV cache while inactive experts (and optionally the n-gram table) wait in system memory. Tier 3 — not enough total memory (VRAM + RAM combined): the only hard wall. Best for most self-hosters: a 128 GB unified-memory machine at UD-Q3_K_XL, or a Tier 2 offload box. Best for quality: a 256 GB Mac Studio at Q4_K_M or above.
| Hardware we track | Memory | Verdict for Qwen3.8-Flash-Next |
|---|---|---|
| Mac Studio, M3 Ultra (512 GB) | 512 GB unified | Tier 1 — Comfortable — Q8_0 with long context |
| Mac Studio, M3 Ultra (256 GB) | 256 GB unified | Tier 1 — Good — Q4_K_M to Q5_K_M comfortably |
| MacBook Pro 16″, M5 Max (128 GB) | 128 GB unified (614 GB/s) | Tier 1, tight — Q2_K-class (~92 GB with overhead) fits; Q3_K_XL needs n-gram table offloaded |
| Strix Halo mini-PC (128 GB, e.g. Ryzen AI Max+ 395) | 128 GB unified (~115 GB allocatable) | Tier 1 — Yes — UD-Q3_K_XL (~90 GB) fits; community-measured 12–22 tok/s on Q4-class builds |
| RTX 5090 (32 GB) + 128 GB system RAM | 160 GB total | Tier 2 — expert offload runs Q4_K_M (~136 GB with overhead); ~31 tok/s estimated |
| RTX 4090 / RTX 3090 (24 GB) + 128 GB system RAM | 152 GB total | Tier 2 — Q4_K_M fits; ~28 tok/s estimated; community-verified 20.4 tok/s on a 16 GB 5070 Ti + 85 GB RAM |
| RTX PRO 6000 Blackwell workstation (96 GB) + 64 GB RAM | 160 GB total | Tier 2 — Q3_K_XL-class with offload headroom |
| H100 / H200 node | 80-141 GB HBM per card | Production — BF16 or FP8 serving via vLLM/SGLang |
Tier 2 in practice — the consumer-GPU path: llama.cpp's --n-cpu-moe / -ot "exps=CPU" tensor overrides (or KTransformers, or the -fit flag) keep inactive experts in system RAM while the GPU processes active experts and the KV cache. This model is the cheapest frontier model to run this way: a community report measured 20.44 tok/s on an RTX 5070 Ti (16 GB) with 85 GB of system RAM at UD-IQ3_XXS (HF discussion, August 2026), and the 1-bit build has been run CPU-only at a reported 52 tok/s with the n-gram table streamed from NVMe (HF discussion, August 2026).
Actually cannot run it: any machine whose total memory (VRAM + RAM combined) is under ~85 GB — a 64 GB Mac, a gaming rig with 32 GB of RAM, a 16 GB laptop with 16 GB of RAM. The GPU badge is irrelevant: a 24 GB RTX 4090 with 128 GB of RAM runs it comfortably, while a 32 GB RTX 5090 in a 32 GB-RAM box does not. For cards working on smaller models, see our best GPU for local LLMs guide.
The n-gram embedding table is the quiet advantage here: Qwen designed it to be offload-friendly, so a machine that keeps the 125B core in fast memory can stream embedding lookups from slower storage-adjacent RAM — or even NVMe, a few KB per token — with less penalty than offloading MoE experts.
Expected speed: estimates and community reports
Community measurements exist for this model. The calculated estimates below use memory bandwidth ÷ bytes read per token, where bytes/token = GGUF file size × 6/180 (the active-parameter share of the file). Treat estimates as ceilings — the measured rows show where real systems land, typically well below the ceiling once KV reads, routing, and dequantization overhead pile up.
| Configuration | Quant | Est. tok/s (ceiling) | Community reports |
|---|---|---|---|
| RTX 5070 Ti 16 GB + 85 GB RAM (expert offload) | UD-IQ3_XXS | ~38 by the RAM-bound math | 20.44 tok/s measured — WhiteDan64, HF discussion #24, August 2026 |
| Strix Halo 128 GB (256 GB/s) | UD-Q4_XS / UD-IQ4_XS | ~82 = 256 ÷ 3.12 GB/token | 22 tok/s (dilavni) and ~12 tok/s (quimeyv) measured — HF discussion #28, August 2026 |
| Desktop CPU only, no GPU (n-gram table on NVMe) | 1-bit build | n/a (streaming-bound) | 52 tok/s reported — laxmimerit, HF discussion #29, August 2026 |
| RTX 5090 + 128 GB DDR5 (expert offload) | Q4_K_M | ~31 = ~2.9 of each token's 3.97 GB active reads from RAM at ~90 GB/s | None published yet at Q4_K_M |
| RTX 4090 / RTX 3090 + 128 GB DDR5 (offload) | Q4_K_M | ~28 (RAM-bound) | None published yet at Q4_K_M |
| MacBook Pro M5 Max 128 GB (614 GB/s) | Q2_K-class | ~229 = 614 ÷ 2.68 GB/token | None published yet on M5 Max |
Buy hardware that runs Qwen3.8-Flash-Next
Every pick below was verified live and purchasable on Amazon on August 27, 2026, and maps to a tier from the table above — including the Tier 2 offload path, where the RAM kit is the enabling component. Links are affiliate links; we may earn a commission at no extra cost to you.
| Pick | Why it runs Qwen3.8-Flash-Next | Expected speed (estimate) | Price check |
|---|---|---|---|
| ASUS ProArt RTX 5090 32 GB | Tier 2 anchor: 32 GB VRAM + 128 GB system RAM = 160 GB total, enough for Q4_K_M (~136 GB with overhead) | ~31 tok/s at Q4_K_M (estimate); 20.4 tok/s community-verified on a smaller card | Check price at Amazon |
| MSI GeForce RTX 4090 Gaming X Trio 24 GB | Tier 2: 24 GB VRAM + 128 GB system RAM = 152 GB total; Q4_K_M fits with offload | ~28 tok/s at Q4_K_M (estimate) | Check price at Amazon |
| NVIDIA RTX 3090 24 GB (renewed) | Budget Tier 2: same 24 GB + 128 GB RAM math; in offload mode the system RAM does the work | ~28 tok/s at Q4_K_M (estimate) | Check price at Amazon |
| Apple MacBook Pro 16″ M5 Max, 128 GB (Early 2026) | Tier 1 laptop: 128 GB unified memory holds Q2_K-class builds (~92 GB with overhead); 614 GB/s per Apple's spec page | ~229 tok/s ceiling at Q2_K-class (614 ÷ 2.68); expect less in practice | Check price at Amazon |
| GMKtec mini PC, Ryzen AI Max+ 395, 128 GB (Strix Halo) | Tier 1 desk box: 128 GB unified holds UD-Q3_K_XL (~103 GB with overhead); the community-measured Strix Halo row above is this platform | 12–22 tok/s community-measured on Q4-class builds; ~85 ceiling at Q3_K_XL | Check price at Amazon |
| TEAMGROUP T-Create Expert 128 GB (2×64 GB) DDR5-6400 | The Tier 2 enabler: turns an RTX 5090/4090/3090 desktop into an offload box (~102 GB/s dual-channel) | Enables the ~28–31 tok/s offload estimates above | Check price at Amazon |
| NEMIX RAM 256 GB (4×64 GB) DDR5-5600 ECC RDIMM | Climbs Tier 2 to Q8_0 (~216 GB with overhead) — needs a workstation platform (Threadripper/Xeon/W790) plus a 32 GB GPU | ~17 tok/s at Q8_0 with a 32 GB GPU (estimate) | Check price at Amazon |
Want the maximum-quality single-box route? A Mac Studio M3 Ultra (256–512 GB) runs Q4_K_M–Q8_0 without offload — but no Mac Studio configuration was listed on Amazon when we last checked (August 27, 2026), so we link nothing rather than guess. See our Mac Studio vs PC for AI comparison for the trade-offs.
How do I run Qwen3.8-Flash-Next locally?
Qwen lists Transformers, vLLM, SGLang, TokenSpeed, and KTransformers as supported frameworks, with production serving on SGLang, KTransformers, or vLLM. Community GGUF builds run on llama.cpp. Thinking is on by default; reasoning_effort accepts xhigh (default), medium, and low.
Option 1 — llama.cpp (GGUF, unified memory or CPU offload)
# 128 GB unified-memory machine (Strix Halo class)
llama-server -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q3_K_XL \
--ctx-size 16384 --threads -1 --flash-attn
# With the vision projector for image input
llama-server -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q3_K_XL \
--mmproj mmproj-F16.gguf --ctx-size 16384 --flash-attnOllama support has not been listed on the model card as of August 27, 2026 — check its library before assuming availability.
Option 2 — vLLM or SGLang (official weights, long context)
# Native 262K context
vllm serve Qwen/Qwen3.8-Flash-Next --tensor-parallel-size 2
# 1M context via YaRN (per the official model card)
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve Qwen/Qwen3.8-Flash-Next \
--hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' \
--max-model-len 1000000Qwen recommends temperature 1.0 / top-p 0.95 / top-k 20 in thinking mode, and allocating generous output budgets (up to 262K reasoning tokens) for agentic tasks. Framework cookbooks are linked from the official model card.
API alternative
The production API version, qwen3.8-flash on Qwen Cloud, ships with a 1M context window by default and built-in tools. Pricing is $0.15 per million input tokens, $0.47 per million output tokens, and $0.016 per million cached-input tokens (explicit cache creation $0.20, reads $0.016). With a 6B-active local build competing against a $0.15/$0.47 API, self-hosting makes sense mainly for privacy, offline use, fine-tuning, or heavy sustained volume.
Frequently Asked Questions
Can you run Qwen3.8-Flash-Next locally?
Yes. Quantized GGUF builds start near 73 GB, so 128 GB unified-memory machines (Strix Halo mini-PCs, 128 GB MacBook Pro) run mid quants, a 256 GB Mac Studio runs Q4_K_M comfortably, and a 16–32 GB GPU with 64–128 GB of system RAM runs it via expert offloading — community-measured at 20.4 tokens per second on an RTX 5070 Ti with 85 GB of RAM.
Can an RTX 4090 or RTX 5090 run Qwen3.8-Flash-Next?
Yes, with expert offloading. The GPU holds the always-active layers, active experts, and KV cache; inactive experts and the n-gram table sit in system RAM. A 24 GB card with 128 GB of RAM runs Q4_K_M at an estimated ~28 tokens per second; a 32 GB card reaches ~31. The community-verified reference point is 20.44 tok/s on an RTX 5070 Ti (16 GB) with 85 GB of RAM.
How much RAM or VRAM does Qwen3.8-Flash-Next need?
About 73 GB for the smallest UD-IQ1_S quant, 90 GB at UD-Q3_K_XL, 119 GB at Q4_K_M, and 188 GB at Q8_0, plus roughly 0.9 GB for the vision projector and KV-cache headroom. The official BF16 checkpoint is about 360 GB.
What is Qwen3.8-Flash-Next?
An experimental preview of the architecture that will underpin Qwen4: a 125B-parameter model with 6B active per token, 51B n-gram embedding parameters, hybrid Gated DeltaNet and Qwen Sparse Attention layers, 512-expert MoE, and native vision input.
What is the n-gram embedding in Qwen3.8-Flash-Next?
A 51B-parameter embedding table covering 20 million bigrams and trigrams, indexed at layer 2. Qwen describes it as a parameter-scaling axis that needs less compute than MoE and is more amenable to offloading — useful on memory-constrained machines.
Can I use Qwen3.8-Flash-Next commercially?
Yes under the Qwen Community License 1.0: free to use, modify, and sell, with two conditions — products with over 100 million monthly active users or $20 million monthly revenue must display the model name, and MaaS or AI-work-assistant businesses need a separate license.
Sources
Model facts are from the official Hugging Face model card and Qwen's technical report; quantization sizes are measured file sizes from the published GGUF repositories; hardware figures come from our GPU specification database. Checked August 27, 2026.
- Official model card Qwen/Qwen3.8-Flash-Next — architecture, license, context handling, benchmarks, serving guides
- Qwen blog and technical report — qwen.ai/blog?id=qwen3.8-flash-next, "On the Design of Qwen3.8-Next Architecture" (Alibaba Group, August 2026)
- GGUF repositories: Unsloth (UD quants), DevQuasar (Q2_K–Q8_0)
- Community speed reports: RTX 5070 Ti + 85 GB RAM, 20.44 tok/s; Strix Halo 12–22 tok/s; CPU-only 1-bit, 52 tok/s reported (all August 2026)
- Amazon product availability and affiliate links — checked live August 27, 2026 via our affiliate proxy (api.petergedeon.com)
- Qwen Cloud API pricing — qwen3.8-flash model page
- Our GPU database — Mac Studio M3 Ultra, Mac mini M4 Max, Strix Halo, RTX PRO 6000 Blackwell, RTX 6000 Ada, H100/H200 records
Affiliate Disclosure: CompareAIHardware.com earns commissions from purchases made through links on this page. This does not affect our editorial content or recommendations.