Blog
Guides and write-ups on local LLM orchestration, hardware sizing, and inference troubleshooting.
Best Local LLMs for 16GB and 24GB GPUs in Late 2026
Three local models, which GPU each one fits, and the exact VRAM math behind it, using real GGUF file sizes.
Local LLM Trends: MoE Models and the KV Cache Problem
Why KV cache size, not parameter count, decides which new MoE models fit a 16GB or 24GB GPU. Real GGUF sizes and the math behind each result.
Building a Private RAG Pipeline with Qdrant, Ollama, and Docker
Learn how to orchestrate a fully containerized Retrieval-Augmented Generation (RAG) stack locally using Ollama, Qdrant, and Open WebUI.
Qwen 3.8 27B vs. Gemma 4 31B Dense: VRAM Benchmarks & Hardware Guide
Comparing Qwen 3.8 27B and Gemma 4 31B Dense. Exact VRAM requirements with our 10% overhead formula to see what fits on your GPU.
VRAM Requirements & Benchmark Guide for DeepSeek R1 Distills (8B, 14B, 32B)
A complete hardware guide to running DeepSeek R1 reasoning distills locally. Find out exactly how much VRAM you need for the 8B, 14B, and 32B models.
Why Your 24GB VRAM Isn't Enough: Context Spillover and Layer Offloading Explained
Why does an 18.49GB 32B GGUF crash a 24GB RTX 4090? A deep dive into KV cache scaling, CUDA context overhead, and layer offloading math.
The Baseline Architecture for Local LLM Orchestration: Avoiding VRAM Bottlenecks
What actually consumes VRAM in local LLM serving, and how to architect a baseline setup that doesn't blow past it.
How to budget VRAM for a 70B model
A walkthrough of how quantization level, context length, and batch size trade off against the VRAM you actually have.
Ollama vs llama.cpp server: when to use which
Ollama trades control for convenience. Here's where that trade stops being worth it.
Diagnosing CUDA out-of-memory errors
The five most common causes of CUDA OOM during local inference, and how to tell them apart.