Can you run Kolibri 1 locally?

Kolibri family · 2026 · 78.1B parameters (3.5B activated per token) · Hugging Face model card

How much VRAM does Kolibri 1 need at each quantization?

Kolibri 1 needs 52.0 GB of VRAM at Q4_K_M. The table below lists weights-only size and total VRAM including overhead for each common quantization level with 4k and 32k token contexts.

Sliding-window attention: window covers 512 preceding tokens plus the current token (config sliding_window=513); most attention layers use the window, keeping long-context KV cost low.

Method: weights = parameters × bits-per-weight (Q4_K_M ≈ 4.8, Q5_K_M ≈ 5.7, Q6_K ≈ 6.6, Q8_0 ≈ 8.5, FP16 = 16), plus 10% loading overhead, plus KV cache from the verified architecture config (50 layers, 4 KV heads, 128 head dim). Source: HF-verified 2026-10-05 via plain api (gated=false safetensors.total=78103074560 params=78.1B ctx=262144 license=apache-2.0 repo=Aleph-Alpha/Kolibri-1@e52eb4627d11516b0c01de49210ab5a4e4061444 layers=50 hidden=2560 kv_heads=4 heads=48 vocab=128000). MoE: num_experts=384 num_experts_per_tok=6 active_params_b=3.46 (vendor model card (Total 78B, Active 3.46B per token); config num_experts=384 routed 6 per token). VRAM figures are documented calculations (Q4-class ~0.6 GB per B params, on active params for MoE rows), not measurements. release_date is the HF repo lastModified date observed on the verification pass. No tok/s or benchmark scores invented.

Which GPUs can run Kolibri 1 locally?

At Q4_K_M with a 4k context, Kolibri 1 needs 52.0 GB of VRAM. The lists below are computed live from our GPU database and grouped by how much headroom the card has. Disclosure: CompareAIHardware.com participates in the Amazon Associates program and earns from qualifying purchases through links on this page. Affiliate relationships do not influence our recommendations.

Runs Kolibri 1 comfortably (25%+ VRAM headroom)

Needs 2+ GPUs to run Kolibri 1

One of these cards is too small on its own, but a pair (tensor or pipeline parallel, ~90% efficiency) covers the 52.0 GB requirement. See our multi-GPU guide for setup.

Can a Mac run Kolibri 1?

Yes — these Apple Silicon machines fit Kolibri 1 at Q4_K_M, since macOS lets the GPU use about 75% of unified memory. Generation speed is bound by memory bandwidth, so the GB/s column matters as much as capacity.

Can Kolibri 1 run on mini PCs, Jetson, or NPU devices?

These edge and NPU devices from our database have enough memory for Kolibri 1 at Q4_K_M. Their memory bandwidth is far below discrete GPUs, so expect a fraction of desktop generation speed.

Which pre-built systems can run Kolibri 1?

These mini PCs and workstations from our database fit Kolibri 1 at Q4_K_M. Usable-memory figures are conservative: Windows shares about half of system RAM with the GPU by default (Linux can expose more), macOS lets Apple Silicon GPUs use about 75% of unified memory, and Linux unified-memory systems such as GB10 expose roughly 90%. Generation speed is bound by memory bandwidth, so compare the GB/s column before buying.

Frequently asked questions