the IT Hustle
ToolsField ManualAbout

Local AI Checker

Wondering if your machine can run a local model? Enter your RAM and GPU (or Apple Silicon) and get a straight answer for every model size from 3B to 70B, at each quantization level, with the memory math shown.

On Apple Silicon the GPU shares system memory, so VRAM doesn't apply. On PCs, check your GPU model's VRAM (e.g. RTX 4060 = 8 GB, RTX 4090 = 24 GB).

Model sizeQ4Q8FP16
3BparamsSlow (CPU)~3 GBSlow (CPU)~5 GBSlow (CPU)~8 GB
7BparamsSlow (CPU)~5 GBSlow (CPU)~9 GBWon't fit~17 GB
13BparamsSlow (CPU)~9 GBWon't fit~16 GBWon't fit~30 GB
34BparamsWon't fit~20 GBWon't fit~39 GBWon't fit~76 GB
70BparamsWon't fit~40 GBWon't fit~79 GBWon't fit~156 GB
How to read this
  • Runs well — Fits in GPU/unified memory — interactive speeds with runtimes like Ollama or LM Studio.
  • Slow (CPU) — Fits in system RAM only — runs on CPU at a few tokens per second. Fine for testing, painful for daily use.
  • Won't fit — Needs more memory than this machine has, even at Q4 quantization.

Planning estimates for llama.cpp-style local inference, including runtime overhead. Actual usage varies with context length and runtime. Q4/Q8/FP16 = quantization levels: smaller numbers mean smaller, slightly less accurate models.

How to Check If Your Machine Can Run Local AI

Find out which local models your hardware can run, from 3B to 70B parameters.

  1. 1
    Enter your RAMTotal system memory in GB. On Windows: Task Manager → Performance. On macOS: About This Mac.
  2. 2
    Enter your GPU VRAMDedicated GPU memory in GB (e.g. RTX 4060 = 8 GB, RTX 4090 = 24 GB). Skip this on Apple Silicon and tick the unified-memory box instead.
  3. 3
    Read the matrixEach model size (3B–70B) gets a verdict per quantization level: runs well on GPU, slow on CPU, or won't fit — with the approximate memory needed.
  4. 4
    Install a runtimeIf you get green cells, install Ollama or LM Studio, pull a model that fits, and you're running AI locally with zero API cost.

Frequently Asked Questions

Quantization levels — how many bits store each model weight. Q4 shrinks a model to roughly a quarter of its full size with a small quality loss; FP16 is full quality at full size. Most local users run Q4 or Q8 for the memory savings.

Apple Silicon Macs share one pool of unified memory between CPU and GPU, so a 32 GB MacBook can GPU-accelerate models that would need a 24 GB discrete graphics card on a PC.

No — frontier hosted models are clearly stronger. But a 7B–13B local model handles summarization, drafting, and coding help well, costs nothing per token, works offline, and keeps sensitive data on your machine.

They're planning figures for llama.cpp-style runtimes, including typical runtime overhead. Long contexts, big batch sizes, and specific runtimes shift real usage — treat a narrow green verdict as a maybe.

Related Tools

AI Cost PlannerToken CounterNetwork Tools