You know that moment when you’re three prompts into something you actually care about, maybe pasting in a client contract or a chunk of your own code, and you pause. Just for a second. Because you’re about to send that text to a server you don’t control, run by a company whose terms of service you skimmed once in 2023.
That pause is why local AI exists. Not because cloud models are bad. Because control matters, and because a $20 a month subscription turns into $240 a year turns into “wait, why am I renting a brain.”
So you start looking into building your own machine. And within about four minutes you hit the wall everyone hits: GPU shopping for AI is nothing like GPU shopping for gaming. The specs that sell a graphics card in a game benchmark barely matter here. There’s one number that runs the whole show, and almost nobody explains it clearly before you’ve already spent the money.
VRAM Is the Only Spec That Actually Decides What You Can Run
Here’s the blunt version. A language model is a pile of numbers, billions of them, and every single one has to sit in memory the GPU can reach instantly. If the model doesn’t fit in VRAM, one of two things happens. It spills into system RAM and slows to a crawl, or it just refuses to load.
Related reading:
CUDA cores, boost clocks, ray tracing hardware, none of that decides whether a 70B model runs on your machine. VRAM capacity decides that. Memory bandwidth decides how fast the tokens come out once it’s loaded. Everything else is a rounding error for this specific job.
A few things worth knowing before you buy anything:
- Models are usually described by parameter count (7B, 14B, 70B) and by quantization level (Q4, Q8, FP16)
- Quantization shrinks the model’s memory footprint by storing each parameter with fewer bits, at a small quality cost
- Q4 quantization is the practical default for most home setups. It roughly cuts memory use in half compared to Q8
- Context window (how much conversation the model remembers) eats into your VRAM budget too, so don’t buy right up to the edge
VRAM Requirement Lookup Chart
Bookmark this table. It’s the thing you’ll actually reference when you’re staring at a GPU listing trying to figure out if it’s enough.
| Model size | Minimum VRAM (Q4) | Comfortable VRAM | What fits on it |
|---|---|---|---|
| 7B to 8B | 6 GB | 8 to 10 GB | Llama 3.3 8B, Qwen 3.5 8B, small coding assistants |
| 13B to 14B | 8 to 11 GB | 14 to 16 GB | Phi 4, Qwen 3.5 14B, general chat |
| 30B to 32B | 18 GB | 20 to 24 GB | Qwen 3.5 32B, strong reasoning and coding models |
| 70B | 40 GB | 48 GB and up | Llama 3.3 70B, the practical ceiling for one consumer GPU |
| MoE giants (DeepSeek R1, similar) | 24 GB and up (fewer active params per token) | 48 GB plus | Frontier level output without needing full dense VRAM |
| 400B and beyond | Not viable on one consumer GPU | 128 GB plus unified memory | Multi GPU rigs or Mac Studio class hardware only |
That 70B row is where most people’s plans fall apart. The minimum VRAM for a 70B LLM at Q4 quantization sits right around 40 GB, and that’s tight, with barely any headroom for context length. This is exactly why the 70B tier forces a choice between a single high end GPU running at the edge of its limit, or unified memory hardware that simply has room to spare.
Budget Build: $800, Best Budget PC for Local LLM
This tier isn’t about running 70B models. It’s about getting a real, responsive local AI machine that handles 7B to 14B models well, which covers most day to day chat, coding help and summarization tasks without ever touching the cloud.
- GPU: RTX 4060 Ti 16 GB (around $449) or a used RTX 3060 12 GB if you find one cheap
- CPU: A current gen budget 6 core chip (Ryzen 5 or Intel Core i5 tier), around $150
- RAM: 32 GB DDR5, roughly $90
- Storage: 1 TB NVMe SSD, around $70
- PSU, case, cooling: Budget but reliable, around $150 combined
What it runs: Llama 3.3 8B and Qwen 3.5 8B at full quality, 14B models at Q4 comfortably. It’s not a powerhouse, but it’s a genuinely useful daily driver, and it’s the honest answer to “best budget PC for local LLM” rather than some fantasy build that quietly needs a $1,500 GPU.
Enthusiast Build: $2,500, the Sweet Spot
This is where most serious hobbyists land, and for good reason. It’s the tier where 32B models run beautifully and 70B becomes possible, even if you’re working near the edge.
- GPU: Used RTX 4090 24 GB (around $1,400 to $1,600 on the secondhand market) or a new RTX 5080 if you prefer warranty coverage
- CPU: Ryzen 9 or Core i7 tier, 12 plus cores, around $350
- RAM: 64 GB DDR5, around $180
- Storage: 2 TB NVMe SSD, around $130
- PSU: 850W to 1000W gold rated, around $150
- Case and cooling: around $150
What it runs: Qwen 3.5 32B at Q8 with room to spare, and mixture of experts models like Qwen 3 30B A3B fly on this card since only a fraction of parameters activate per token. 70B models run at Q4, tight but workable if you’re not chasing huge context windows. This is genuinely the best GPU for local LLM use for most people who aren’t running a business on it.
Workstation Build: $6,000 and Up
Past this point you’re not buying speed anymore. You’re buying headroom, and there are two very different ways to get it.
Path one, the discrete GPU route: An RTX 5090 32 GB build. Expect to pay $4,000 to $5,000 for the card alone right now given ongoing supply constraints, plus another $1,500 to $2,000 for a serious CPU, 128 GB of RAM, dual NVMe drives and a 1200W platinum PSU. This gets you the fastest token generation available on consumer hardware and enough VRAM to run 70B at Q5 or Q6 with real context length.
Path two, the unified memory route: A Mac Studio configured with the M5 Ultra chip, starting at $5,499 and scaling to 512 GB of unified memory. Slower per token than the GPU route, but it can load models that no single consumer GPU will ever touch, and it does it while pulling a fraction of the power and making almost no noise.
Neither path is wrong. They’re answering different questions, which brings us to the comparison everyone actually wants.
Mac Studio vs RTX 4090 for AI: Which One Should You Buy
This is the question that stalls more local AI builds than any other, so let’s just lay it out plainly.
The RTX 4090 is a discrete GPU with 24 GB of dedicated GDDR6X memory running at over 1,000 GB/s bandwidth. It’s fast, it’s proven, it runs on CUDA which is still the default for nearly every AI tool and framework, and used units sit around $1,400 to $1,600. The catch is that 24 GB is a hard ceiling. Nothing you do changes that number.
The Mac Studio uses unified memory, meaning the CPU and GPU share one large memory pool instead of the GPU getting its own dedicated chunk. A Mac Studio with the M5 Max chip starts at $2,499 with up to 128 GB of unified memory. Step up to the M5 Ultra and you’re looking at $5,499 and up, with a ceiling of 512 GB. That’s more memory than four RTX 4090s combined, in one quiet box pulling under 200 watts.
Here’s how they actually stack up:
- Raw speed: RTX 4090 wins clearly, generally 40 to 60 percent faster tokens per second than an equivalent Mac Studio configuration
- Maximum model size: Mac Studio wins by a wide margin once you’re past 24 GB of VRAM need
- Power draw: Mac Studio at 50 to 200 watts under load versus 350 to 450 watts for the 4090 alone
- Noise: Mac Studio is close to silent, the RTX 4090 build is audibly a computer under load
- Software ecosystem: RTX 4090 and CUDA support essentially everything, Mac Studio relies on MLX and llama.cpp style tools which have caught up a lot but still trail
- Upgrade path: RTX 4090 systems can add a second card later, Mac Studio memory is fixed at purchase
If your work fits inside 24 GB, buy the RTX 4090. It’s faster, cheaper, and more flexible with software. If you need to run 70B models comfortably, or you want headroom for whatever ships next year, the Mac Studio’s unified memory is doing something a GPU physically cannot do at any price under ten thousand dollars.
Building a Home Lab AI Server
If you’re thinking past a personal workstation toward something that runs 24 hours a day, serving requests to your household or a small team, the calculus shifts a little.
A home lab AI server build should prioritize:
- Headless operation. No need for a monitor or a flashy case, but you do need remote access set up properly (SSH, or a tool like Tailscale)
- Sustained cooling. A server running inference jobs around the clock needs airflow rated for continuous load, not burst gaming sessions
- A UPS. Losing power mid inference on a long running task is annoying, losing it during a fine tuning job is expensive
- Redundant storage if you’re storing anything you can’t easily redownload, since model files themselves are always replaceable
One thing worth knowing if you go this route: idle GPU time on a home AI server doesn’t have to sit unused. Decentralized compute networks let you rent out spare capacity when you’re not actively running inference, which can offset part of the electricity cost over a year. We’ve covered how to monetize your spare GPU for AI in a separate guide if that’s interesting to you.
RAM and Storage: The Part People Forget
VRAM gets all the attention, but system RAM and storage still matter more than people expect.
A rough rule that holds up well: your system RAM should be at least double your VRAM. This gives the operating system, the inference engine, and any partial CPU offloading enough breathing room. For a 24 GB GPU build, 64 GB of system RAM is the sane minimum, not the ceiling.
Storage matters because model files are large. A single 70B model at Q4 can run 40 GB or more, and if you’re experimenting with several models you’ll eat through storage fast. A 2 TB NVMe drive isn’t a luxury on a serious build, it’s the thing that stops you from deleting a model you’ll want again next week.
Frequently Asked Questions
What is the minimum VRAM for a 70B LLM? Around 40 GB at Q4 quantization, though that leaves little room for context length. 48 GB or more is the comfortable target if you want a decent context window alongside the model itself.
Is Mac Studio better than a PC for local AI? It depends on the model size you need. Mac Studio wins on maximum model capacity and power efficiency. A PC with an RTX 4090 wins on raw speed and software compatibility. Most people are better served by matching the hardware to the model size they actually use, not the biggest number on a spec sheet.
Can I run local AI without a dedicated GPU? Technically yes, small models under 4B parameters can run on CPU with enough system RAM, but it’s slow enough that most people give up on it within a week. A budget GPU changes the experience completely.
How much RAM do I need for local AI beyond VRAM? At least double whatever your GPU’s VRAM is. A 24 GB GPU build wants 64 GB of system RAM as a baseline, more if you’re multitasking or running multiple models.
Is a used RTX 3090 still worth it for AI in 2026? Yes, and it’s one of the best value picks available. It matches the RTX 4090’s 24 GB VRAM capacity for roughly half the price, at somewhat slower speeds. For anyone building the best budget PC for local LLM use on a tighter number, a used 3090 is often the smarter buy over a brand new 16 GB card.
Can I combine two GPUs for more VRAM? Yes, most inference tools support splitting a model across multiple GPUs, and it’s a legitimate way to reach 48 GB or more without buying a single expensive card. The tradeoff is added complexity in cooling, power delivery, and software configuration.
