Buying RAM

RAM for local AI models: how much and how fast?

For running AI models on your own PC, how much memory you have decides which models you can run, and how fast that memory is decides how quickly they answer. If the model fits in your graphics card's memory (VRAM), the GPU does the work and system RAM matters less. If it doesn't, the model runs from system RAM, and then 32GB handles small models (7 to 14 billion parameters), 64GB handles mid-size ones (around 30B), and 96 to 128GB is needed for 70B models, which will run slowly on a normal desktop.

How much memory does a model need?

The rough math: number of parameters × bits per parameter ÷ 8, plus extra for the conversation history (the "context").

Most people run "quantized" models, compressed versions that use about 4 to 5 bits per parameter instead of 16, with a small loss in quality. Corsair gives real examples:

  • An 8B model at the popular Q4_K_M quality drops from about 16GB to around 5 to 6GB.
  • A 70B model (like Llama 3.3 70B) at 4-bit needs about 42GB just for the model.
  • Context adds up: about 1GB per 10 billion parameters for 8,000 tokens of context, and long contexts can double the total.

So, roughly:

Model size Memory at 4-bit (model only) System RAM to run it from RAM
7B to 8B About 5 to 6GB 16GB works; 32GB is comfortable
14B About 9 to 10GB 32GB
30B to 32B About 18 to 20GB 64GB
70B About 42GB 96GB to 128GB

Leave room for Windows and your other apps on top of the model, typically 8GB or more.

Why speed is about bandwidth

To produce each word (token), the computer reads the model's weights from memory. So the speed limit is roughly:

Tokens per second ≈ memory bandwidth ÷ model size

Pasquale Pillitteri explains it the same way: fewer bytes to read per token means more tokens per second. Dual-channel DDR5-6000 gives about 96 GB/s (6000 × 8 bytes × 2 channels). That means, at best:

  • An 8B model at about 5GB: up to roughly 19 tokens per second.
  • A 70B model at about 42GB: up to roughly 2.3 tokens per second.

Real numbers come in below those ceilings. That's why 70B models on a normal desktop feel slow: usable for a single question, painful for a chat.

Memory speed helps directly here, much more than in games. Corsair reports that moving from DDR5-4800 to DDR5-6000 improved token generation by 20 to 23% for 7B and 8B models running on the CPU.

Graphics card vs system RAM

A graphics card's memory is several times faster than system RAM. Corsair notes that a model fitting entirely in VRAM can generate 40 or more tokens per second, while the same model running from system RAM might manage 8 to 15 on a fast CPU setup.

So:

  • If your model fits in VRAM: system RAM mainly needs to hold the model while it loads, plus your other apps. 32GB covers most single-GPU setups.
  • If it doesn't fit: tools like llama.cpp, Ollama and LM Studio can put part of the model on the GPU and the rest in system RAM. Speed drops sharply as more runs from RAM, but bigger models become possible. This is where 64 to 128GB of fast system RAM pays off.
  • No graphics card: everything runs from system RAM, and bandwidth sets the pace.

What about "mixture of experts" models?

Some large models are built as a "mixture of experts": only part of the model is used for each token. They still need enough memory to hold the whole model, but they only read the active part each time, so they run much faster than their size suggests. If you have lots of RAM but a modest GPU, these models are often the best fit.

Unified memory machines

Some systems share one large pool of fast memory between the CPU and GPU: Apple's M-series Macs, and AMD's Ryzen AI Max (Strix Halo) mini PCs and laptops with up to 128GB. They can hold big models and read them faster than a dual-channel desktop. Mini PC Lab notes that even mid-range mini PCs with soldered LPDDR5X beat most DDR5 SO-DIMM setups on bandwidth. The trade-off: the memory is soldered, so you buy it all at once. See LPDDR5X, CAMM2 and LPCAMM2 explained.

Buying RAM for a local AI desktop

  1. Pick the largest model you want to run and use the table above to size your RAM.
  2. Buy two sticks, not four. Desktop CPUs have two memory channels either way, so four sticks add capacity but not bandwidth, and they usually force lower speeds. 2x48GB (96GB) or 2x64GB (128GB) is better than four smaller sticks. See 48GB and 96GB kits.
  3. Get the fastest speed your platform runs reliably: DDR5-6000 on AMD, 6400 or more on Intel. Here, unlike in games, it shows.
  4. Turn on XMP or EXPO. Running at the default 4800 costs you about a fifth of your speed.
  5. Consider the GPU first if you're choosing between upgrades. More VRAM speeds up models that fit far more than any RAM upgrade.

The short version

  • Memory size decides which models fit; memory bandwidth decides how fast they run.
  • At 4-bit, an 8B model needs about 5 to 6GB; a 70B model about 42GB, plus context and Windows.
  • 32GB for small models, 64GB for about 30B, 96 to 128GB for 70B from system RAM.
  • Faster RAM helps AI far more than games: DDR5-6000 vs 4800 is about 20% more tokens per second on CPU.
  • If the model fits in your graphics card's memory, that's much faster than any system RAM.

Sources