TECHNICAL RESEARCH & EMPIRICAL CPU ABLATIONS

Systems Engineering Papers & Benchmarks

Controlled ablation studies, CPU quantization benchmarks, and architectural deep-dives on running 0.8B–8B Small Language Model agents deterministically on commodity hardware.

10-Plane Systems Harness $0 Cost · 100% CPU GBNF & Cascade Router
November 2026 · 10 min read

Building a 10-Plane Local AI Agent Harness at $0 Cost: How We Made 0.8B–8B SLMs Reliable on Pure CPU

Small models do not fail because they lack reasoning capacity for everyday tasks; they fail because we wrap them in lazy harnesses designed for 200B cloud models. Here is the complete systems breakdown of slm_harness (./slm)—combining a <35ms 4-stage Cascade Router, GBNF JSON-Schema decoding, content-addressed artifact:// handles, a Watermark LRU MemoryArena, and self-healing SQL/Python/Math verifiers at $0 cost.

$0.00
API & GPU Cloud Cost
24.6 tok/s
Qwen3-4B CPU + GBNF
<35 ms
4-Stage Cascade Router
3,272 MB
Peak Arena RSS (13 Models)
SP
Suryaprakash C V
Creator, SLMAgents · Pure-CPU Systems Engineering
Controlled Benchmark 100% CPU RAG Sub-3B Models
August 2026 · 8 min read

Context Distillation for CPU RAG: A Controlled Study on Sub-3B Models

Most local RAG pipelines retrieve the top 5–8 chunks from a vector database and dump 2,500+ tokens into the prompt. When run on sub-3B models on CPU, prefill latency spikes and hallucination rates surge. We ran a controlled 120-query ablation to demonstrate how cross-encoder distillation to Top-K=2 maximizes grounded accuracy and cuts TTFT by 75%.

95.8%
Grounded Accuracy (±1.8% SE)
1.15s
Total Latency p50 (1.23s p95)
0.10s
TTFT p50 (75% Faster)
3,049 MB
Peak Memory Footprint
SP
Suryaprakash C V
Creator, SLMAgents · Published on Medium