Local LLM PC Builds by Model Size
The first local-LLM build question is not “which GPU is fastest?” It is “what must stay in memory?” A rough lower bound for weights is parameters × bits ÷ 8. An 8-billion-parameter model at 4-bit quantization is roughly 4 GB before runtime overhead and context; a 32B model is roughly 16 GB; a 70B model is roughly 35 GB. Real runs need additional headroom, so buying exactly the raw weight size is a recipe for offloading or out-of-memory errors.
The practical tiers
| Workload | Sensible GPU memory | System RAM | What it is good for |
|---|---|---|---|
| Small models, 7B–14B at 4-bit | 8–12 GB | 32 GB | Chat, summarization, light code completion, learning local runtimes |
| Mid-size models, roughly 20B–32B at 4-bit | 16 GB | 64 GB | Better coding/reasoning models, longer sessions, one substantial agent |
| 70B-class at 4-bit | 48 GB+ combined | 128 GB | Full or near-full GPU residency with context headroom |
| Large-model experimentation with CPU offload | 24–32 GB | 128 GB | Slower 70B-class runs without buying a multi-GPU workstation |
These are planning bands, not universal compatibility guarantees. Architecture, quantization format, runtime, context length, and cache precision all move the actual number.
Starter: small local models
An 8 GB card such as the RTX 5060 is enough to learn the workflow and run compact quantized models, but it leaves little room for long context. The 12 GB Intel Arc B580 offers more capacity for less money when the chosen runtime supports Intel GPUs well. Pair either with 32 GB of system RAM and a 1 TB NVMe drive.
This is the right tier for local privacy, document summaries, and occasional code assistance. It is not a future-proof “run anything” box.
Sweet spot: 16 GB and 64 GB of RAM
The RTX 5080 16GB or Radeon RX 9070 XT 16GB gives a 4-bit 30B-class model a plausible home, but context and runtime overhead can still push it over the edge. NVIDIA generally has the least-friction software path; AMD can be a strong value only after confirming support in the exact runtime and operating system you intend to use.
Use 64 GB of system RAM. It gives the runtime somewhere to offload, keeps an IDE and browser from fighting the model, and is cheap compared with replacing the GPU later.
High end: a 32 GB card is not a full 70B guarantee
The RTX 5090 32GB is extremely fast, but a 70B model at 4-bit starts around 35 GB for weights alone. It therefore remains a partial-offload machine for many 70B-class configurations. It can still be a good workstation—32 GB handles smaller models comfortably and makes offloaded 70B runs less painful—but capacity and speed are separate questions.
For full GPU residency with usable context headroom, plan around 48 GB or more of aggregate VRAM and verify that the runtime can split the model across the chosen GPUs. That usually means professional cards or a carefully engineered multi-GPU system, not a normal gaming build with a second card casually added.
What to buy first
Choose the model class and runtime, calculate the weight floor, add context headroom, then choose the GPU. After that, buy at least as much system RAM as the largest model you expect to load—preferably twice as much—and enough NVMe storage to keep multiple quantizations without constant deletion. CPU upgrades come after memory capacity unless your plan relies heavily on CPU offload.