← All GPUs
GeForce RTX 4090
NVIDIA · Consumer GPU · 2022
Check Latest Price
$1,599 MSRP
Check Price on Amazon →
We earn a commission from qualifying purchases · Prices may vary
Price checked Jul 23, 2026
Performance Benchmarks
Image Generation all sourced rows →
| Model | Metric | Value | Source |
|---|---|---|---|
|
FLUX.1-dev
BF16 Requires memory-efficient attention (xFormers/SDPA) within 24GB; approximate |
images per minute |
4
img/min
est Diffusers · 2026-05-03
|
Estimated Spheron GPU Benchmark Blog ↗ |
| SDXL Turbo FP16 | images per minute |
80
img/min
ComfyUI · 2025-06-01
|
Independent benchmark Tom's Hardware GPU Benchmarks 2025 ↗ |
|
Stable Diffusion 1.5
FP16 Standardized UL Procyon suite; same lab platform (ThreadRipper 7980X, driver 571.86); 1.188 s/image |
procyon image overall score |
5,260
score
UL Procyon AI Image Generation · 2025-01-29
|
Independent benchmark StorageReview GPU Reviews ↗ |
|
Stable Diffusion 1.5 INT8
INT8 Standardized UL Procyon suite; same lab platform (ThreadRipper 7980X, driver 571.86); 0.503 s/image |
procyon image overall score |
62,160
score
UL Procyon AI Image Generation · 2025-01-29
|
Independent benchmark StorageReview GPU Reviews ↗ |
|
Stable Diffusion XL
FP16 Standardized UL Procyon suite; same lab platform (ThreadRipper 7980X, driver 571.86); 7.461 s/image |
procyon image overall score |
5,025
score
UL Procyon AI Image Generation · 2025-01-29
|
Independent benchmark StorageReview GPU Reviews ↗ |
LLM Inference all sourced rows →
| Model | Metric | Value | Source |
|---|---|---|---|
|
Llama-2-13B
Standardized UL Procyon suite; same lab platform (ThreadRipper 7980X, driver 571.86); output rate 92.853 tok/s |
procyon text overall score |
5,013
score
UL Procyon AI Text Generation (TensorRT) · 2025-01-29
|
Independent benchmark StorageReview GPU Reviews ↗ |
|
Llama-3-70B
Q4 24GB VRAM, very tight, heavy swapping — editorial estimate pending article-level source verification |
tokens per second |
18
tok/s
est llama.cpp · 2025-06-01
|
Estimated Tom's Hardware GPU Benchmarks 2025 ↗ |
|
Llama-3-8B
Standardized UL Procyon suite; same lab platform (ThreadRipper 7980X, driver 571.86); output rate 150.039 tok/s |
procyon text overall score |
4,849
score
UL Procyon AI Text Generation (TensorRT) · 2025-01-29
|
Independent benchmark StorageReview GPU Reviews ↗ |
|
Llama-3.1-8B
FP16 Batch-serving throughput, ~18GB VRAM used; published llama.cpp/vLLM benchmarks (approximate) |
batch tokens per second |
2,550
tok/s
est vLLM · 2026-05-03
|
Estimated Spheron GPU Benchmark Blog ↗ |
|
Llama-3.1-8B
Q4_K_M Token generation: single user, batch 1, 4k context, mean of 3 runs |
tokens per second |
125
tok/s
llama.cpp b3500 (CUDA) · 2026-05-22
|
Unverified community MyAIHardware llama.cpp Benchmarks ↗ |
|
Llama-3.1-8B
Q4_K_M Prompt processing: 512-token prompt, batch 1, mean of 3 runs |
prompt tokens per second |
4,800
tok/s
llama.cpp b3500 (CUDA) · 2026-05-22
|
Unverified community MyAIHardware llama.cpp Benchmarks ↗ |
|
Mistral-7B
Standardized UL Procyon suite; same lab platform (ThreadRipper 7980X, driver 571.86); output rate 183.266 tok/s |
procyon text overall score |
5,094
score
UL Procyon AI Text Generation (TensorRT) · 2025-01-29
|
Independent benchmark StorageReview GPU Reviews ↗ |
|
Phi-3.5-mini
Standardized UL Procyon suite; same lab platform (ThreadRipper 7980X, driver 571.86); output rate 244.343 tok/s |
procyon text overall score |
4,958
score
UL Procyon AI Text Generation (TensorRT) · 2025-01-29
|
Independent benchmark StorageReview GPU Reviews ↗ |
|
Qwen3-8B
Q4_K_XL Token generation: 16K context, batch 1, Ubuntu 24.04, on-hardware lab measurement |
tokens per second |
104.3
tok/s
llama.cpp llama-bench, CUDA 12.8 · 2025-12-09
|
Unverified community Hardware Corner LLM GPU Rankings ↗ |
Full Specifications
Vram gb
24 GB
Vram type
GDDR6X
Memory bus
384 bit
Memory bandwidth gbps
1008 GB/s
Tdp watts
450 W
Cuda cores
16384
Pcie interface
PCIe 4.0 x16
Architecture
Ada Lovelace
Related GPUs
Common Questions
Can the GeForce RTX 4090 run local LLMs?
Yes, if the model fits in its 24 GB of memory — VRAM capacity decides which models load at all, while its 1008 GB/s of bandwidth decides how fast tokens generate. With 24 GB or more it can hold large quantized models such as 70B-class LLMs in reduced precision, and our model pages list exact VRAM fits per model.
How much VRAM does the GeForce RTX 4090 have?
GeForce RTX 4090 has 24 GB of video memory with 1008 GB/s of memory bandwidth and a 450 W TDP , per the specification values in our hardware database.
Is the GeForce RTX 4090 worth it for AI workloads?
At a $1,599 MSRP, its value depends on whether your models fit in 24 GB. Compare its VRAM and bandwidth against your target models with our VRAM calculator, or see how it stacks up in head-to-head comparisons.
Guides & Tools
CompareAIHardware.com participates in the Amazon Associates program. As an Amazon Associate, we earn from qualifying purchases.