Can you run Nemotron 3 Ultra 550B A55B locally?
Nemotron 3 family · 2026 · 560.5B parameters (55B activated per token) · Hugging Face model card
NVIDIA Nemotron 3 Ultra is a 550B-total / 55B-active base model (vendor card: Number of model parameters 550B Total / 55B Active). The HF safetensors total is 560,524,578,816 in bfloat16, about 2 percent above the announced 550B. config.json declares 512 routed experts with 22 active per token plus 1 shared expert (shared intermediate 10240), hidden size 8192, 64 attention heads, only 2 KV heads, head dim 128, vocab 131072, on the nemotron_h hybrid Mamba/attention architecture; the config does not publish a layer count, so that field is left empty here instead of estimated. Native context is 262,144 tokens. Licensed under NVIDIA's model license (other). It is a base (pre-trained, not instruction-tuned) checkpoint. Local VRAM fit uses the 55B active count: roughly 32 GB at Q4-class quantization (calculated). Measured speeds: not yet published here.
Minimum: 32 GB+ memory (Q4-class on 55B active, calculated) · Recommended: 48 GB+ for comfortable context headroom (Q4-class, calculated)
How much VRAM does Nemotron 3 Ultra 550B A55B need at each quantization?
Nemotron 3 Ultra 550B A55B needs 369.9 GB of VRAM at Q4_K_M. The table below lists weights-only size and total VRAM including overhead for each common quantization level with 4k and 32k token contexts.
| Quantization | Bits / weight | Weights only | Total + KV @4k ctx | Total + KV @32k ctx |
|---|---|---|---|---|
| Q4_K_M | 4.8 | 336.3 GB | 369.9 GB | 369.9 GB |
| Q5_K_M | 5.7 | 399.4 GB | 439.3 GB | 439.3 GB |
| Q6_K | 6.6 | 462.4 GB | 508.7 GB | 508.7 GB |
| Q8_0 | 8.5 | 595.6 GB | 655.1 GB | 655.1 GB |
| FP16 | 16 | 1,121.0 GB | 1,233.1 GB | 1,233.1 GB |
config.json omits num_hidden_layers for this base checkpoint, so the layer count is left NULL rather than guessed. Hybrid nemotron_h stack with only 2 KV heads.
Method: weights = parameters × bits-per-weight (Q4_K_M ≈ 4.8, Q5_K_M ≈ 5.7, Q6_K ≈ 6.6, Q8_0 ≈ 8.5, FP16 = 16), plus 10% loading overhead, plus KV cache from the verified architecture config (not applicable to this architecture). Source: HF-verified 2026-09-30 via plain api (gated=false safetensors.total=560524578816 params=560.52B ctx=262144 license=other repo=nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16@9f5b80e4845884eb8da03353b0b176503631e034 layers=None hidden=8192 kv_heads=2 heads=64 vocab=131072). MoE: num_experts=512 num_experts_per_tok=22 active_params_b=55.0 (vendor model card (550B total, 55B active); config n_routed_experts=512, 22 per token). VRAM figures are documented calculations (Q4-class ~0.6 GB per B params, on active params for MoE rows), not measurements. release_date is the HF repo lastModified date observed on the verification pass. No tok/s or benchmark scores invented.
Which GPUs can run Nemotron 3 Ultra 550B A55B locally?
At Q4_K_M with a 4k context, Nemotron 3 Ultra 550B A55B needs 369.9 GB of VRAM. The lists below are computed live from our GPU database and grouped by how much headroom the card has. Disclosure: CompareAIHardware.com participates in the Amazon Associates program and earns from qualifying purchases through links on this page. Affiliate relationships do not influence our recommendations.
Needs 2+ GPUs to run Nemotron 3 Ultra 550B A55B
One of these cards is too small on its own, but a pair (tensor or pipeline parallel, ~90% efficiency) covers the 369.9 GB requirement. See our multi-GPU guide for setup.
| GPU | VRAM | Bandwidth | Type | |
|---|---|---|---|---|
| AMD Instinct MI325X | 256 GB ×2 | 6000 GB/s | Data Center GPU | |
| AMD Instinct MI355X | 288 GB ×2 | 8000 GB/s | Data Center GPU | |
| NVIDIA B300 (Blackwell Ultra) | 288 GB ×2 | 8000 GB/s | Data Center GPU |
Can a Mac run Nemotron 3 Ultra 550B A55B?
Yes — these Apple Silicon machines fit Nemotron 3 Ultra 550B A55B at Q4_K_M, since macOS lets the GPU use about 75% of unified memory. Generation speed is bound by memory bandwidth, so the GB/s column matters as much as capacity.
| Apple Silicon | Unified memory | Usable by GPU (~75%) | Bandwidth |
|---|---|---|---|
| M5 Ultra (Mac Studio) | 512 GB | 384 GB | 1200 GB/s |