H200 SXM
NVIDIA · Data Center GPU · 2024
Performance Benchmarks
LLM Inference all sourced rows →
| Model | Metric | Value | Source |
|---|---|---|---|
|
Llama-3-70B
Q4 141GB HBM3e — editorial estimate pending article-level source verification |
tokens per second |
140
tok/s
est llama.cpp · 2025-06-01
|
Estimated NVIDIA Official Specs ↗ |
Full Specifications
Related GPUs
Common Questions
Can the H200 SXM run local LLMs?
Yes, if the model fits in its 141 GB of memory — VRAM capacity decides which models load at all, while its 4800 GB/s of bandwidth decides how fast tokens generate. With 24 GB or more it can hold large quantized models such as 70B-class LLMs in reduced precision, and our model pages list exact VRAM fits per model.
How much VRAM does the H200 SXM have?
H200 SXM has 141 GB of video memory with 4800 GB/s of memory bandwidth and a 700 W TDP , per the specification values in our hardware database.
Is the H200 SXM worth it for AI workloads?
Its value depends on whether your models fit in 141 GB. Compare its VRAM and bandwidth against your target models with our VRAM calculator, or see how it stacks up in head-to-head comparisons.
Guides & Tools
CompareAIHardware.com participates in the Amazon Associates program. As an Amazon Associate, we earn from qualifying purchases.