Best GPU for Local LLMs in 2026
Updated October 27, 2026. All prices are launch MSRPs from our GPU database; we do not track street prices.
How much GPU you need depends on which model classes you want to run, and this guide maps that decision to specific cards.
Want the full tier-by-tier breakdown — every current GPU from 12 GB to 96 GB, minimum vs recommended configs per model size, and power/price brackets? See the Best GPU for AI Inference in 2026 pillar guide.
For running local LLMs in 2026, the NVIDIA GeForce RTX 5090 is the best overall GPU because its 32 GB of GDDR7 memory holds large quantized models that smaller cards cannot fit. The best value pick is a used GeForce RTX 3090 with 24 GB of VRAM, and the cheapest usable new card is the Intel Arc B580 at a $249 MSRP. This guide ranks the best GPU for each budget using verified specification data and third-party benchmark results.
How do the best GPUs for local LLMs compare?
The table below compares seeded specifications and Llama-3.1-8B inference speed for the GPUs most often used with local LLMs. Tok/s figures are decode rates measured at Q4_K_M quantization in llama.cpp.
| GPU | VRAM | Memory type | Bandwidth | TDP | MSRP | Llama-3.1-8B Q4_K_M (tok/s) |
|---|---|---|---|---|---|---|
| GeForce RTX 5090 | 32 GB | GDDR7 | 1,792 GB/s | 575 W | $1,999 | 215 |
| GeForce RTX 4090 | 24 GB | GDDR6X | 1,008 GB/s | 450 W | $1,599 | 125 |
| Radeon RX 7900 XTX | 24 GB | GDDR6 | 960 GB/s | 355 W | $999 | 76 |
| GeForce RTX 3090 | 24 GB | GDDR6X | 936 GB/s | 350 W | $1,499 | 85 |
| GeForce RTX 5080 | 16 GB | GDDR7 | 960 GB/s | 360 W | $999 | 132 * |
| GeForce RTX 4060 Ti 16GB | 16 GB | GDDR6 | 288 GB/s | 160 W | $499 | — |
| Arc B580 | 12 GB | GDDR6 | 456 GB/s | 190 W | $249 | 41 * |
Benchmark attribution: the RTX 5090 (215), RTX 4090 (125), RTX 3090 (85) and RX 7900 XTX (76) figures all come from a single test run in our benchmark database — Llama-3.1-8B at Q4_K_M in llama.cpp, decode rate, recorded 22 May 2026, with no row flagged as an estimate. The two starred figures come from different runs and are not directly comparable with those four: the RTX 5080's 132 tok/s was recorded 1 August 2026 through Ollama, and the Arc B580's 41 tok/s on 25 May 2026 through Vulkan. We hold no Llama-3.1-8B Q4_K_M row for the RTX 4060 Ti 16GB, so that card shows no figure. This column replaces an earlier Llama-3-8B Q4 column whose values were not backed by database rows; the two Llama-3-8B Q4 rows we do hold (the RTX 4060 Ti 16GB at 95 tok/s and the H100 SXM at 500 tok/s) are both flagged as estimates and are not used above.
Which GPU is best for 70B-class models?
Best for 70B-class models: GeForce RTX 5090. Its 32 GB of GDDR7 memory is the largest consumer VRAM pool in our database, and its 1,792 GB/s of bandwidth is the highest of any consumer card listed here. The class is defined by weight size: a 70B model at Q4 quantization occupies roughly 40 GB before any KV cache, so no single 24 GB card can hold it outright.
Our database holds two Llama-3-70B Q4 rows for these cards: 42 tokens per second for the RTX 5090 and 18 for the GeForce RTX 4090, both recorded 1 June 2025 against Tom's Hardware as the source. Both rows carry an estimate flag, and our own notes on them read "editorial estimate pending article-level source verification" — so read them as a floor, not as measured throughput. The ratio between the two tracks bandwidth rather than raw compute: the 5090's 1,792 GB/s is 1.78x the 4090's 1,008 GB/s, and its 42 tokens per second is 2.3x the 4090's 18. Decode on a memory-bound model rises with bandwidth, hence the wider margin here than on the 8B test above.
What the notes do support is the fit. The 5090's 32 GB is a tight but workable allocation for a 70B model at Q4 — the weights alone consume most of the pool, leaving little headroom for a long context window. The 4090's 24 GB is tighter still: our notes record heavy memory swapping there, because the model does not fit and layers must stream from system memory over PCIe. Swapping does not stop the model running, but it moves the bottleneck off the GPU's bandwidth onto a much slower link.
Among the consumer cards in the table above, the ranking is short. The RTX 5090 is first at 32 GB. The RTX 4090, RTX 3090 and RX 7900 XTX tie at 24 GB, where our notes record the same heavy-swapping behaviour; the RX 7900 XTX adds its ROCm configuration overhead on top. The RTX 5080, RTX 4060 Ti 16GB and Arc B580 sit below the quantized weights' size and are not realistic candidates for this class.
For buyers who need to run 70B models comfortably rather than occasionally, the practical options sit above the consumer tier. Workstation cards with 48 GB such as the RTX 6000 Ada — listed at a $6,800 launch MSRP in our database — hold the quantized weights with room for context. Apple's M3 Ultra is the other route, with up to 512 GB of unified memory per our specification database: it fits far larger models, though models that fit both platforms run faster on the NVIDIA card.
The decision rule is narrow. If 70B at Q4 is the ceiling of your workload and you can accept a tight fit, the RTX 5090 is the consumer card that gets closest to holding it in VRAM. If you need 70B with long context or larger models, budget for 48 GB or unified memory instead — or rent, as the section below explains.
Which GPU is best on a mid-range LLM budget?
Best mid-range pick: GeForce RTX 4060 Ti 16GB. It carries 16 GB of GDDR6 memory at a $499 MSRP, which is the cheapest new NVIDIA card in our database with that much VRAM.
We hold no measured 8B-class decode row for the RTX 4060 Ti 16GB, so the table above leaves it blank. The one Llama-3-8B Q4 row we do hold for this card is 95 tokens per second in llama.cpp, recorded 1 June 2025 from TechPowerUp — but that row is flagged as an estimate rather than a measurement, so we treat it as an indication and not a result. What we can state without qualification is the hardware: its 288 GB/s of bandwidth is the lowest in this comparison, so it will be slower than pricier cards, while its 160 W TDP means it runs in most desktop systems without a power supply upgrade. It is a good match for buyers who mostly run models in the 7B to 14B class and want a new card with a warranty.
The mid-range bracket is where the buying decision gets genuinely close, because our database holds several cards within $200 of each other whose measured decode rates differ by more than their prices do. The clearest comparison we hold is the GeForce RTX 4070 SUPER: on the same 22 May 2026 common run as the cards in the table above — Llama-3.1-8B at Q4_K_M in llama.cpp, no estimate flag — it generated 92 tokens per second. That is faster than the RTX 3090's 85 and the RX 7900 XTX's 76 on the identical run, from a card listed at a $599 launch MSRP. It is now end-of-life in our database, so it is a used-market proposition, but the throughput figure is measured rather than estimated.
The trade-off against the RTX 4060 Ti 16GB is memory, not speed. The 4070 SUPER carries 12 GB of GDDR6X against the 4060 Ti's 16 GB, so it wins on decode — 504 GB/s of bandwidth against 288 GB/s — while losing the headroom that keeps 14B-class models fully resident. Buyers who stay at 8B or below and want the fastest decode per dollar should weigh the 4070 SUPER; buyers who expect to grow into larger quantized models should keep the 16 GB card. Both draw under 225 W, so neither forces a power supply change.
For a current-generation card with a warranty at a similar price, the GeForce RTX 5060 Ti 16GB is the closest match in our database: 16 GB of GDDR7, 448 GB/s of bandwidth, and a $429 launch MSRP — less than the 4060 Ti's $499 for newer memory and more bandwidth, though we hold no measured 8B decode row for it. The GeForce RTX 5070 pairs 12 GB of GDDR7 with 672 GB/s at $549, and the RTX 3060 12GB remains the floor at $329, where our database records 38 tokens per second on the common run.
Check the GeForce RTX 4060 Ti 16GB on Amazon →
Which GPU is the cheapest way to run local LLMs?
Best budget pick: Intel Arc B580. At a $249 MSRP with 12 GB of GDDR6 memory, it is the least expensive new card in our database that can hold 7B-class models at Q4 quantization.
Our database records the Arc B580 at 41 tokens per second on Llama-3.1-8B at Q4_K_M through the Vulkan backend, measured 25 May 2026 on a run three days newer than the common test used for the NVIDIA and AMD figures, so treat it as indicative beside them rather than as directly comparable. The card runs local LLM software through Vulkan rather than CUDA, so some tooling that assumes NVIDIA hardware will need configuration. For buyers testing local LLMs on a tight budget, it is a reasonable starting point despite those caveats.
Check current Intel Arc B580 prices on Amazon →
Which AMD GPU is best for local LLMs?
Best AMD pick: Radeon RX 7900 XTX. It pairs 24 GB of GDDR6 memory with an $999 MSRP, which is the lowest launch price in our database for any card with 24 GB.
On the common test run in our benchmark database — Llama-3.1-8B at Q4_K_M in llama.cpp, recorded 22 May 2026 with no estimate flag — the RX 7900 XTX generated 76 tokens per second through the ROCm stack, against the GeForce RTX 3090's 85 on the same run. That is roughly 89% of the 3090's decode rate from a card launched at a $999 MSRP, which is the core of its value case. The trade-off is software support: CUDA remains the default target for most local AI tooling, while AMD relies on ROCm and the Vulkan backend, which our benchmark notes describe as functional but less widely supported. Buyers who already prefer AMD for other workloads can run local LLMs on this card without major compromises.
See the Radeon RX 7900 XTX on Amazon →
Why do LLM buyers keep choosing the used RTX 3090?
Best value pick: used GeForce RTX 3090. It offers 24 GB of GDDR6X memory, which matches the VRAM of the RTX 4090 at a much lower typical used price.
Our database lists the RTX 3090 as an end-of-life 2020 card with a $1,499 launch MSRP, 936 GB/s of bandwidth, and a 350 W TDP. On the 22 May 2026 common run it still generated 85 tokens per second on Llama-3.1-8B at Q4_K_M, which outpaces both new cards under $1,000 in the table above — the $999 RTX 5080 on its own later run and the $999 RX 7900 XTX on the same common run. The caveats are the risks of any used purchase: no warranty, unknown treatment by previous owners, and a power draw that is high for a card of its age. For buyers whose priority is VRAM capacity per dollar, it remains the classic recommendation, and our used RTX 3090 buying guide covers how to evaluate individual cards.
How much VRAM do I need for local LLMs?
VRAM capacity decides which models fit at all, while memory bandwidth decides how fast they run. A slower card with enough VRAM beats a faster card that cannot hold the model. The model-fit rows in our database map each model size to a minimum card and a recommended card.
The pattern is consistent across the 7B to 14B band. Models such as Llama 3.1 8B, Qwen 2.5 7B, and Qwen 3 8B carry a minimum of 8 GB — an RTX 4060 class card — and a recommendation of 12 GB, which our database names as the RTX 3060 12GB or RTX 4070. The 12B to 14B models step up one tier: Gemma 3 12B, Mistral NeMo 12B, Phi-4 14B, and Qwen 3 14B all list 12 GB as the minimum and 16 GB as the recommendation, the latter naming the RTX 4060 Ti 16GB. Our own notes for Llama 3.1 8B put the arithmetic behind the 8 GB floor: about 5.8 GB at Q4_K_M with a 4k context, rising to about 9.6 GB at 32k. That is why 12 GB is the comfortable target rather than the minimum, and why a 6 GB card must offload layers to system RAM.
The 32B class changes the answer. DeepSeek R1-Distill 32B, Qwen 3 32B, and Qwen 2.5 Coder 32B list a minimum of 24 GB — the RTX 3090 or RTX 4090 — with the R1-Distill notes giving about 22.7 GB at Q4_K_M and a 4k context. That is a floor rather than a sweet spot: reasoning traces of 8k tokens or more raise the KV cache, so the recommendation climbs to 32 GB or more for long contexts.
At 70B, a single consumer card stops being the target. Llama 3.1 70B and Llama 3.3 70B list 48 GB as the minimum — the RTX A6000, RTX 6000 Ada, or L40S — against about 47.9 GB at Q4_K_M with a 4k context, of which 42.36 GB is weights. Our notes state plainly that a single RTX 4090 cannot hold the model. The recommendation is an 80 GB card such as the H100 SXM, and our Llama-3-70B Q4 rows describe the model as a tight fit even on the RTX 5090's 32 GB, with heavy swapping on the RTX 4090's 24 GB.
For larger models still, the practical options are workstation cards with 48 GB, multi-GPU setups, or Apple Silicon: the M3 Ultra Mac Studio supports up to 512 GB of unified memory per our specification database, which no consumer NVIDIA or AMD card approaches.
Two rules follow from those rows. Buy for the model class you actually run, not the largest one you might try once: the minimum column is what avoids CPU offload, and the recommended column keeps longer contexts comfortable. And because VRAM is the one specification you cannot upgrade later without replacing the card, the VRAM calculator is the fastest way to check any model and quantization.
What are the trade-offs between NVIDIA, AMD, and Intel for LLMs?
NVIDIA's CUDA is the default software target for local AI tools, so NVIDIA cards have the fewest compatibility problems. AMD cards run llama.cpp well through ROCm and Vulkan but need more configuration, and Intel's Arc cards work through Vulkan with the youngest software ecosystem of the three. Our benchmark database records which backend produced every figure it holds, and that column states the difference more plainly than any summary can: each NVIDIA row we hold was measured through CUDA, the Radeon RX 7900 XTX through ROCm, and the Arc B580 through Vulkan.
Because the database holds one test run across all three vendors on the same model and quantization — Llama-3.1-8B at Q4_K_M in llama.cpp, recorded on 22 May 2026 — the stack difference can be priced rather than asserted. On that identical run the RX 7900 XTX generated 76 tokens per second against the RTX 3090's 85 and the RTX 4090's 125. All three come from the same tier-3 community source, recorded under identical settings — single user, batch 1, 4k context, mean of 3 runs — with no estimate flag set.
- CUDA (RTX 4090): 125 tok/s on the common test, the fastest card in this guide on that run.
- ROCm (RX 7900 XTX): 76 tok/s — 89% of the RTX 3090's 85, but 61% of the RTX 4090's 125. The AMD penalty is modest against a same-generation card and steep against a flagship.
- Vulkan (Arc B580): 41 tok/s. That figure is weaker evidence than the other two: it was recorded three days later, on 25 May 2026, on a different llama.cpp build, and from a different source entirely — so treat it as indicative beside the 22 May rows rather than as directly comparable.
Prompt processing tells a different story from token generation. On the same common test the RX 7900 XTX sustained 2,100 prompt tokens per second against the RTX 3090's 2,400 — 88% of it — where its decode rate was 89% of that same card's. Against the RTX 4090 both gaps widen sharply: 44% of its 4,800 on prompt processing, and 61% of its 125 on decode. A buyer who mostly ingests long documents and generates short answers will feel the AMD shortfall less than the decode figure on its own suggests.
The practical summary: CUDA buys compatibility and, against the flagship, roughly 1.6x the throughput. ROCm costs a modest fraction of speed against a mid-range NVIDIA card and returns more VRAM per dollar — 24 GB at a $999 MSRP against 15 GB per $1,000 on the RTX 4090 — which is why the RX 7900 XTX stays competitive in our rankings even where it trails on speed. Intel's Arc cards remain the cheapest way in, but note that our Arc decode evidence is thinner than the CUDA and ROCm evidence: the only measured row we hold for the B580 is the 41 tok/s Vulkan figure above, and we hold no SYCL or IPEX-LLM result, so confirm your backend before buying. Buyers who enjoy tinkering get better VRAM per dollar outside NVIDIA; buyers who want everything working on the first attempt should stay on NVIDIA.
What are the pros and cons of the top picks?
Each pick trades something away. Here is the short version of what you gain and what you accept with each recommendation.
- RTX 5090: most consumer VRAM (32 GB) and highest bandwidth (1,792 GB/s); costs $1,999 at MSRP and draws 575 W.
- Used RTX 3090: 24 GB of VRAM at a low used price with 85 tok/s on 8B models on our 22 May 2026 test run; end-of-life, no warranty, 350 W draw.
- RTX 4060 Ti 16GB: cheapest new 16 GB NVIDIA card at $499 with a 160 W TDP; only 288 GB/s of bandwidth limits speed.
- Arc B580: $249 entry price with 12 GB of VRAM; estimate-tier performance and Vulkan-only software path.
- RX 7900 XTX: 24 GB at a $999 MSRP with competitive inference speed; ROCm and Vulkan require more setup than CUDA.
The pattern is consistent: within one brand, you pay mainly for VRAM capacity and bandwidth, and the steepest prices buy capacity that only matters for the largest models.
Who should NOT buy a flagship GPU for local LLMs?
Buyers who only run 7B-class and 8B-class models should not buy a flagship card for LLMs alone. On our 22 May 2026 common run — Llama-3.1-8B at Q4_K_M in llama.cpp, single user, batch 1, 4k context, mean of 3 runs — the RTX 3090 reached 85 tokens per second and the RX 7900 XTX 76. The RTX 3090 carries a $1,499 launch MSRP and the RX 7900 XTX $999, both under the RTX 5090's $1,999, and neither is a current flagship. The RTX 5080, at a $999 MSRP, recorded 132 tokens per second on its own 1 August 2026 run. Decode at that level already outruns comfortable reading speed, so a 32 GB flagship buys little for models that fit comfortably in 12 GB.
That is not a claim that flagship cards are wasted. Our benchmark notes record the RTX 5090 at 215 tokens per second on the same 8B test, the highest figure we hold, so it is measurably the fastest card here. The point is narrower: for 8B-class work the extra speed sits well past the threshold where it changes what you can do, while the $1,999 MSRP and 575 W TDP are costs you pay from day one.
Buyers who need 70B-class models only occasionally should not buy a flagship either, and should compare against renting first. Our cloud pricing database lists a single RTX 4090 instance at $0.34 per hour on RunPod, on-demand, seeded June 2025 — the same rate our deep learning GPU guide publishes. For infrequent large-model sessions that hourly figure avoids the hardware purchase entirely, and the 4090's 24 GB is a workable but tight fit for a 70B model at Q4, as our notes record. Buying wins when large models run daily, or when data must stay offline and cannot leave your machine.
A third group should wait before buying: buyers who are not yet sure which model sizes they will standardise on. VRAM capacity is the one specification you cannot upgrade later without replacing the card, and the jump from 12 GB to 24 GB to 32 GB is what the RTX 3060 12GB, the RTX 3090, and the RTX 5090 each represent in our database. Starting on the cheapest card that runs your current models, then moving up once your workload is stable, costs less than guessing the ceiling on day one.
Finally, buyers who need maximum throughput on a large model across many concurrent users should look past consumer flagships altogether. Cards such as the H100 SXM, with 80 GB and a much wider memory bus, or the RTX 6000 Ada with 48 GB, exist for that tier. Our database lists the RTX 6000 Ada at a $6,800 launch MSRP and the RTX A6000 at $4,500, both far above the consumer flagship. If your workload is serving rather than personal use, the honest advice is to price the professional tier or rent it, not to buy a gaming flagship and hope it scales.
Can these GPUs also fine-tune LLMs, or are they inference-only?
Every NVIDIA and AMD card on this page can fine-tune LLMs with QLoRA — the method that quantizes the frozen base model to 4-bit and trains only small low-rank adapter weights, published by Dettmers et al. as fine-tuning “a 65B parameter model on a single 48GB GPU” [s]. What changes per card is which model sizes you can train. Unsloth’s published QLoRA VRAM floors put 7B–8B models at roughly 5–6 GB and 14B at about 8.5 GB [s], so even the 12 GB Arc B580 clears 8B fine-tuning through its Vulkan path, and every 16 GB card here trains 14B models. The 24 GB cards — the used RTX 3090, RTX 4090, and RX 7900 XTX — cover the 27B–32B class, and the RTX 5090’s 32 GB is the consumer ceiling for QLoRA on 70B-class models, matching the paper’s 48 GB result with today’s 4-bit tooling headroom.
Two caveats keep expectations honest. First, fine-tuning is a compute job, not a bandwidth job: generation speed — the metric this page ranks on — says little about training throughput, which is why our training-focused picks live in the deep learning GPU guide. Second, prompt-processing speed still matters after fine-tuning, because a custom model is only useful if you can actually feed it prompts. Our benchmark database records prompt-processing throughput from the same 22 May 2026 test run that backs the decode column above: the RTX 5090 processes 9,800 tokens per second on Llama-3.1-8B, versus 4,800 for the RTX 4090, 2,400 for the RTX 3090, 2,100 for the RX 7900 XTX, and 950 for the RTX 3060 12GB (512-token prompt, batch 1, mean of 3 runs). Cards that feel similar at chat speed separate sharply when you paste a 4,000-token document into a fine-tuned model.
Full-parameter fine-tuning is a different league entirely. A widely used estimate puts naive full fine-tuning at roughly 4.2× the model size in memory — 1× weights, 1× gradients, 2× optimizer state, plus activations [s] — which pushes even 8B full fine-tuning past 32 GB and is exactly why adapter methods like QLoRA dominate on consumer cards.
How did we rank these GPUs?
How did we rank these GPUs?
We ranked the cards using the specification set in our GPU database, which stores manufacturer datasheet values for VRAM, memory type, bandwidth, TDP, and launch MSRP. Speed claims come from our benchmark database, each row labeled with its source, test date, and estimate status — and only from rows we actually hold. The 8B-class decode column in the comparison table above is drawn from measured Llama-3.1-8B Q4_K_M rows: the RTX 5090, RTX 4090, RTX 3090 and RX 7900 XTX from one shared run recorded 22 May 2026, the RTX 5080 from a separate 1 August 2026 run, and the Arc B580 from a 25 May 2026 run. Where we hold no row for a card we say so instead of substituting a figure: that is why the RTX 4060 Ti 16GB shows no decode number. The 70B-class figures quoted elsewhere on this page are separate Llama-3-70B Q4 rows from June 2025, and those rows carry an estimate flag, which is why we describe them as indicative rather than measured.
We deliberately report MSRP only, never street or used prices, because those change constantly. Rankings weigh VRAM capacity first for LLM use, then bandwidth, then power draw, then price. Affiliate links do not influence any ranking on this site.
Frequently Asked Questions
How much VRAM do I need for local LLMs?
For 7B-class and 8B-class models at Q4 quantization, 12 GB is enough according to our benchmark notes. For 70B-class models, our notes record heavy memory swapping even on a 24 GB card, so 32 GB or more is the safer target.
Is the RTX 3090 still good for local LLMs in 2026?
Yes. It carries 24 GB of VRAM and generated 85 tokens per second on Llama-3.1-8B at Q4_K_M on our 22 May 2026 test run, faster than every new card under $1,000 in our comparison. Its drawbacks are age, a 350 W TDP, and the lack of any warranty when bought used.
Can I run local LLMs on AMD or Intel GPUs?
Yes. On the same 22 May 2026 test run the RX 7900 XTX reached 76 tokens per second on Llama-3.1-8B at Q4_K_M through ROCm, about 89% of the RTX 3090, and the Arc B580 recorded 41 tokens per second through Vulkan on a run three days later. Expect more configuration effort than on CUDA-based cards.
Does memory bandwidth matter for LLM inference?
Yes, bandwidth is the main speed limiter once a model fits in VRAM. The RTX 5090 combines 1,792 GB/s with 32 GB, which is why it leads every 8B-class decode row in our table at 215 tokens per second.
Is Apple Silicon a good alternative for local LLMs?
For very large models, yes. The M3 Ultra Mac Studio supports up to 512 GB of unified memory per our database, which exceeds any consumer GPU. Its 819 GB/s of bandwidth is below the RTX 5090's 1,792 GB/s, so models that fit both platforms run faster on the NVIDIA card.
Sources
Specifications are manufacturer datasheet values from our GPU database; benchmark figures come from the sources below, named here in the order they appear on the page.
- Tom's Hardware GPU Benchmarks 2025 — https://www.tomshardware.com/pc-components/gpus
- TechPowerUp GPU Reviews — https://www.techpowerup.com/reviews/
- MyAIHardware llama.cpp Benchmarks — https://www.myaihardware.com/llama-cpp-benchmarks/
- LocalAI Master GPU Benchmarks — https://localaimaster.com/blog/rtx-5090-vs-5080-local-ai
- RunAIHome Local AI Benchmarks — https://runaihome.com/blog/intel-arc-b580-local-ai-2026/
- QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers et al., arXiv 2305.14314) — https://arxiv.org/abs/2305.14314
- Unsloth fine-tuning VRAM requirements — https://unsloth.ai/docs/get-started/fine-tuning-for-beginners/unsloth-requirements
- Modal — Efficient LLM Finetuning with Unsloth (full fine-tuning memory estimate) — https://modal.com/docs/examples/unsloth_finetune
Related reading: our VRAM calculator, the deep learning training GPU guide, and the used RTX 3090 buying guide.
Disclosure: CompareAIHardware.com participates in the Amazon Associates program and earns from qualifying purchases through links on this page. Affiliate relationships do not influence our recommendations.
CompareAIHardware.com participates in the Amazon Associates program. As an Amazon Associate, we earn from qualifying purchases. This guide reflects our independent analysis — affiliate relationships do not influence our recommendations.