L LocalNodeOps

Know what your hardware can actually run.

LocalNodeOps estimates what your hardware can actually run: VRAM needs, tokens-per-second figures and quantization trade-offs for consumer and prosumer hardware, with every figure labeled by where it comes from.

Hardware benchmarks

Each card is tagged verified, community-reported, or estimated. Right now every card is estimated: the figures come from third-party measurements, published papers or bandwidth calculations, and each card says which. None has been reproduced on LocalNodeOps hardware yet.

GPU Estimated — not yet benchmarked

RTX 4090 24GB

40 Tokens/sec (Qwen3 32B, Q4_K)

24GB of GDDR6X at about 1,008 GB/s. Fits 32B-class Q4 models at 8K context; a 70B at Q4 needs about 40GB and still requires CPU offload.

View benchmark
GPU Estimated — not yet benchmarked

RTX 4060 Ti 16GB

34 Tokens/sec ceiling (Llama 3.1 8B, Q8_0)

16GB at about 288 GB/s. Roomy for 8B models at Q8_0 and long context, but memory bandwidth, not VRAM, caps speed.

View benchmark
CPU Estimated — not yet benchmarked

Ryzen 9 7950X

16 Tokens/sec (Llama 2 7B, Q4_0, CPU-only)

A CPU-only fallback for 7B models. Memory bandwidth, not core count, sets the speed.

View benchmark
Memory Estimated — not yet benchmarked

DDR5-6000 Dual Channel

96 GB/s Theoretical peak bandwidth

Theoretical peak for dual-channel DDR5-6000. Real sustained bandwidth is lower, and it sets the speed of any model layers that run from system RAM.

View benchmark
Quantization Estimated — not yet benchmarked

GGUF Q4_K_M

+0.18 Perplexity delta vs FP16 (Llama 3 8B)

A common default for consumer GPUs: about 3.3x smaller than FP16 on Llama 3 8B, for a small perplexity increase.

View benchmark
Quantization Estimated — not yet benchmarked

AWQ 4-bit

+0.40 Perplexity delta vs FP16 (Llama 3 8B)

GPU-focused 4-bit weight quantization. The perplexity cost varies by model, from about +0.07 to +0.40 in published tables.

View benchmark
GPU Estimated — not yet benchmarked

Mac Mini M4 16GB

24 Tokens/sec ceiling (Llama 3.1 8B, Q4_K_M)

16GB unified memory shared between CPU, GPU, and OS. By default macOS lets the GPU use about 75% of it (around 12GB), not the full 16GB.

View benchmark
GPU Estimated — not yet benchmarked

RTX 5090 32GB

61 Tokens/sec (Qwen3 32B, Q4_K_M)

32GB GDDR7 at 1,792 GB/s bandwidth. Fits 32B-class Q4 models fully on the GPU; a 70B at Q4 still needs CPU offload.

View benchmark

Latest from the blog

Guides on quantization, VRAM budgeting, and diagnosing common local inference errors.

Hardware

Best Local LLMs for 16GB and 24GB GPUs in Late 2026

Three local models, which GPU each one fits, and the exact VRAM math behind it, using real GGUF file sizes.

LocalNodeOps
5 min read
Analysis

Local LLM Trends: MoE Models and the KV Cache Problem

Why KV cache size, not parameter count, decides which new MoE models fit a 16GB or 24GB GPU. Real GGUF sizes and the math behind each result.

Haseeb
4 min read
Guides

Building a Private RAG Pipeline with Qdrant, Ollama, and Docker

Learn how to orchestrate a fully containerized Retrieval-Augmented Generation (RAG) stack locally using Ollama, Qdrant, and Open WebUI.

Haseeb
6 min read