Jetson edge devices for local AI: which Orin runs your model?

Specs pulled live from our hardware database, with a model-fit matrix computed at Q4_K_M using our documented VRAM formulas.

The NVIDIA Jetson AGX Orin 64GB is the only Jetson that runs 70B-class LLMs locally, because it is the only module with 64 GB of unified memory (204.8 GB/s bandwidth, 275 INT8 TOPS). The Jetson Orin Nano 8GB Super is the budget entry at 67 INT8 TOPS and $249 launch price, and it runs 8B-class models such as Llama 3.1 8B at Q4_K_M. Every Jetson runs the same CUDA software stack as desktop NVIDIA GPUs, so any GGUF model that fits in memory works.

What are the NVIDIA Jetson Orin specs?

All current Jetson modules share the Ampere GPU architecture and LPDDR5 unified memory. INT8 TOPS, power, and memory figures below are from NVIDIA's Jetson Orin product pages.

Launch MSRPs are NVIDIA module/dev-kit launch prices from our sourced database; blank cells are SKUs NVIDIA sells only through partners. See each product page for sources.

Which Jetson runs which AI model?

The matrix below is computed, not guessed: each cell compares the model's Q4_K_M VRAM requirement (weights + 10% loading overhead + KV cache or activation overhead at 4k context, per our documented methodology) against the module's unified memory. A check mark means the requirement fits in full unified memory.

Q4_K_M requirement per model shown in each green cell. Unified memory is shared with the OS on Jetson — plan for 1-2 GB of system overhead, or run headless. Full quantization tables (Q4 to FP16, 4k/32k contexts) are on each model page.

How much slower is a Jetson than a desktop GPU for LLM inference?

Token generation speed is governed by memory bandwidth. The AGX Orin's 204.8 GB/s is about one-eighth of a GeForce RTX 5090's 1792 GB/s, so the same 70B model generates tokens roughly eight times slower on the Jetson. The Jetson's advantage is power: 25-60 W against 115-600 W for desktop cards, which is why Jetson wins in robots, drones, kiosks, and offline appliances. For pure price-performance on a desk, a used RTX 3090 or the RTX 3060 12GB is faster per dollar — see our best GPU for LLMs guide for desktop picks.

What can the 67-TOPS Orin Nano Super actually do?

The Jetson Orin Nano 8GB Super runs Llama 3.1 8B at Q4_K_M (5.8 GB requirement), Whisper large-v3 transcription, and small vision models — enough for voice assistants, document processing, and sensor analytics at 25 W. It cannot load 27B-class models, which need 16 GB or more. The 157-TOPS Orin NX 16GB Super handles 14B-class models at Q4_K_M (about 10.5 GB) but not 27B models, which need roughly 18 GB; the AGX Orin 64GB is the only module that fits 70B-class models like Llama 3.3 70B (about 48 GB at Q4_K_M).

What else should I read?

Pair this hub with our model VRAM guides (every major LLM, image, and audio model with quantization tables), the VRAM calculator to run your own numbers, and our mini PC finder if you want quiet desktop edge boxes with laptop-class NPUs instead of modules.