Compare GPUs for Local AI
Find the right GPU for LLMs, image generation, video generation, fine-tuning, and more. Compare VRAM, benchmarks, software compatibility, and value — then buy with confidence.
GPU Comparison Selector
What will you use it for?
All GPUs (15)
Click a row to add to comparison, or use the selector above.
| GPU | Mfr | VRAM | BW | TDP | MSRP | Year | Status |
|---|---|---|---|---|---|---|---|
| M3 Ultra (Mac Studio) | Apple | 512 GB | 819 GB/s | 270W | $3,999 | 2025 | available |
| GeForce RTX 5090 | NVIDIA | 32 GB | 1792 GB/s | 575W | $1,999 | 2025 | available |
| GeForce RTX 5080 | NVIDIA | 16 GB | 960 GB/s | 360W | $999 | 2025 | available |
| Radeon RX 7600 XT | AMD | 16 GB | 288 GB/s | 190W | $329 | 2024 | available |
| Arc B580 | Intel | 12 GB | 456 GB/s | 190W | $249 | 2024 | available |
| H200 SXM | NVIDIA | 141 GB | 4800 GB/s | 700W | — | 2024 | available |
| M4 Max (MacBook Pro) | Apple | 128 GB | 546 GB/s | — | $3,199 | 2024 | available |
| GeForce RTX 4080 SUPER | NVIDIA | 16 GB | 736 GB/s | 320W | $999 | 2024 | available |
| GeForce RTX 4060 Ti 16GB | NVIDIA | 16 GB | 288 GB/s | 160W | $499 | 2023 | available |
| H100 SXM | NVIDIA | 80 GB | 3350 GB/s | 700W | — | 2023 | available |
| RTX 6000 Ada | NVIDIA | 48 GB | 960 GB/s | 300W | $6,800 | 2023 | available |
| Radeon RX 7900 XTX | AMD | 24 GB | 864 GB/s | 355W | $999 | 2022 | available |
| GeForce RTX 4090 | NVIDIA | 24 GB | 1008 GB/s | 450W | $1,599 | 2022 | available |
| RTX A6000 | NVIDIA | 48 GB | 768 GB/s | 300W | $4,500 | 2021 | available |
| GeForce RTX 3090 | NVIDIA | 24 GB | 936 GB/s | 350W | $1,499 | 2020 | eol |
💾 Why VRAM matters most
For local AI, VRAM determines which models you can run at all. A GPU with 8GB simply cannot load a 14B model, regardless of speed. Bandwidth then determines how fast inference runs. Prioritize VRAM capacity first, then bandwidth.
⚡ Measured vs theoretical
TFLOPS numbers from spec sheets are theoretical. Actual LLM inference depends on memory bandwidth, quantization support, software optimization, and thermal behavior. We label every data point: Measured, Vendor spec, Calculated, Community, or Estimated.
🔧 Software ecosystem
NVIDIA (CUDA) has the broadest support: PyTorch, vLLM, Ollama, ComfyUI, TensorRT all work out of the box. AMD (ROCm) works on Linux for most tools. Apple (MLX) is excellent for LLMs but limited for some video/image tools. Intel is improving but has less community support.
💰 New vs used considerations
Used RTX 3090 (24GB) cards offer exceptional value for LLM work. Check: memory condition (mining wear), fan health, thermal pad condition, and warranty status. Verify VRAM capacity matches (some models have variants).
🔗 Multi-GPU limitations
Most local AI tools split models across GPUs via tensor parallelism or pipeline parallelism. Performance scales well for inference but NOT linearly. NVLink helps but is absent on consumer RTX 40/50 series. Expect ~80% scaling with 2 GPUs for inference.
❓ How to choose
1. What\\'s your largest model? → Minimum VRAM
2. What\\'s your workload? → LLM/Image/Video
3. What\\'s your software preference? → CUDA/ROCm/MLX
4. What\\'s your budget? → New + used options
5. What\\'s your PSU/case? → Physical fit