NVIDIA B200
NVIDIA · Data Center GPU · 2025
Performance Benchmarks
LLM Inference all sourced rows →
| Model | Metric | Value | Source |
|---|---|---|---|
|
Llama-2-70B
FP8 MLPerf v5.1 server scenario, 8x B200 HGX host (throughput is per-host, not per-GPU) |
system tokens per second server |
101,611
tok/s
TensorRT-LLM (MLPerf v5.1) · 2025-09-10
|
Independent benchmark MLPerf Inference Results ↗ |
|
Llama-2-70B
FP8 MLPerf v5.1 offline scenario, 8x B200 HGX host (throughput is per-host, not per-GPU) |
system tokens per second offline |
101,246
tok/s
TensorRT-LLM (MLPerf v5.1) · 2025-09-10
|
Independent benchmark MLPerf Inference Results ↗ |
|
Llama-3.1-405B
FP8 MLPerf v5.1 offline scenario, 8x B200 HGX host (throughput is per-host, not per-GPU) |
system tokens per second offline |
1,660
tok/s
TensorRT-LLM (MLPerf v5.1) · 2025-09-10
|
Independent benchmark MLPerf Inference Results ↗ |
|
Llama-3.1-405B
FP8 MLPerf v5.1 server scenario, 8x B200 HGX host (throughput is per-host, not per-GPU) |
system tokens per second server |
1,280
tok/s
TensorRT-LLM (MLPerf v5.1) · 2025-09-10
|
Independent benchmark MLPerf Inference Results ↗ |
Full Specifications
Related GPUs
Common Questions
Can the NVIDIA B200 run local LLMs?
Yes, if the model fits in its 192 GB of memory — VRAM capacity decides which models load at all, while its 8000 GB/s of bandwidth decides how fast tokens generate. With 24 GB or more it can hold large quantized models such as 70B-class LLMs in reduced precision, and our model pages list exact VRAM fits per model.
How much VRAM does the NVIDIA B200 have?
NVIDIA B200 has 192 GB of video memory with 8000 GB/s of memory bandwidth and a 1000 W TDP , per the specification values in our hardware database.
Is the NVIDIA B200 worth it for AI workloads?
Its value depends on whether your models fit in 192 GB. Compare its VRAM and bandwidth against your target models with our VRAM calculator, or see how it stacks up in head-to-head comparisons.
Guides & Tools
CompareAIHardware.com participates in the Amazon Associates program. As an Amazon Associate, we earn from qualifying purchases.