Can you run K2 Horizon 375B A23B locally?
K2-Horizon family · 2026 · 379.2B parameters (23B activated per token) · Hugging Face model card
K2 Horizon 375B A23B is a mixture-of-experts model with 23B activated parameters per token and 375B-class total open weights (official card: stores 375B parameters and runs 23B per token; HF safetensors.total=379,167,159,168 — the multi-format dump total exceeds the 375B designation, so the designation is kept for the announced count). Native context is 524,288 tokens (config.json max_position_embeddings; card: 512K), apache-2.0. Local VRAM fit uses the 23B active count, not the 375B dense-equivalent: roughly 16 GB at Q4-class (calculated). Weights still occupy ~375B-class disk. Measured speeds: not yet published here.
Minimum: 16 GB+ memory (Q4-class on 23B active, calculated) · Recommended: 24 GB+ for comfortable context headroom (Q4-class on 23B active, calculated)
How much VRAM does K2 Horizon 375B A23B need at each quantization?
K2 Horizon 375B A23B needs 251.3 GB of VRAM at Q4_K_M. The table below lists weights-only size and total VRAM including overhead for each common quantization level with 4k and 32k token contexts.
| Quantization | Bits / weight | Weights only | Total + KV @4k ctx | Total + KV @32k ctx |
|---|---|---|---|---|
| Q4_K_M | 4.8 | 227.5 GB | 251.3 GB | 258.4 GB |
| Q5_K_M | 5.7 | 270.2 GB | 298.2 GB | 305.4 GB |
| Q6_K | 6.6 | 312.8 GB | 345.1 GB | 352.3 GB |
| Q8_0 | 8.5 | 402.9 GB | 444.2 GB | 451.4 GB |
| FP16 | 16 | 758.3 GB | 835.2 GB | 842.4 GB |
Method: weights = parameters × bits-per-weight (Q4_K_M ≈ 4.8, Q5_K_M ≈ 5.7, Q6_K ≈ 6.6, Q8_0 ≈ 8.5, FP16 = 16), plus 10% loading overhead, plus KV cache from the verified architecture config (61 layers, 8 KV heads, 128 head dim). Source: HF-verified 2026-09-21 via api (gated=false params=379.17B safetensors.total=379167159168 ctx=524288 license=apache-2.0 repo=IFM/K2-Horizon-375B-A23B@4b84fb40cbe7ffe103e3a9e7a1bc2addb9c7dd0b layers=61 kv_heads=8). MoE: num_experts=192 num_experts_per_tok=8 active_params_b=23.0 (official README (375B stored, 23B runs per token)). VRAM figures are documented calculations, not measurements. No tok/s invented.
Which GPUs can run K2 Horizon 375B A23B locally?
At Q4_K_M with a 4k context, K2 Horizon 375B A23B needs 251.3 GB of VRAM. The lists below are computed live from our GPU database and grouped by how much headroom the card has. Disclosure: CompareAIHardware.com participates in the Amazon Associates program and earns from qualifying purchases through links on this page. Affiliate relationships do not influence our recommendations.
Minimum GPUs that fit K2 Horizon 375B A23B at Q4_K_M
These GPUs hold the model but leave little headroom — keep contexts short.
| GPU | VRAM | Bandwidth | Type | |
|---|---|---|---|---|
| AMD Instinct MI325X | 256 GB | 6000 GB/s | Data Center GPU | |
| AMD Instinct MI355X | 288 GB | 8000 GB/s | Data Center GPU | |
| NVIDIA B300 (Blackwell Ultra) | 288 GB | 8000 GB/s | Data Center GPU |
Needs 2+ GPUs to run K2 Horizon 375B A23B
One of these cards is too small on its own, but a pair (tensor or pipeline parallel, ~90% efficiency) covers the 251.3 GB requirement. See our multi-GPU guide for setup.
| GPU | VRAM | Bandwidth | Type | |
|---|---|---|---|---|
| H200 SXM | 141 GB ×2 | 4800 GB/s | Data Center GPU | |
| AMD Instinct MI300X | 192 GB ×2 | 5300 GB/s | Data Center GPU | |
| NVIDIA B200 | 192 GB ×2 | 8000 GB/s | Data Center GPU |