TECHNICAL RESEARCH & EMPIRICAL CPU ABLATIONS
Systems Engineering Papers & Benchmarks
Controlled ablation studies, CPU quantization benchmarks, and architectural deep-dives on running 0.8B–8B Small Language Model agents deterministically on commodity hardware.
10-Plane Systems Harness
$0 Cost · 100% CPU
GBNF & Cascade Router
November 2026 · 10 min read
Small models do not fail because they lack reasoning capacity for everyday tasks; they fail because we wrap them in lazy harnesses designed for 200B cloud models. Here is the complete systems breakdown of slm_harness (./slm)—combining a <35ms 4-stage Cascade Router, GBNF JSON-Schema decoding, content-addressed artifact:// handles, a Watermark LRU MemoryArena, and self-healing SQL/Python/Math verifiers at $0 cost.
$0.00
API & GPU Cloud Cost
24.6 tok/s
Qwen3-4B CPU + GBNF
<35 ms
4-Stage Cascade Router
3,272 MB
Peak Arena RSS (13 Models)
SP
Suryaprakash C V
Creator, SLMAgents · Pure-CPU Systems Engineering
Controlled Benchmark
100% CPU RAG
Sub-3B Models
August 2026 · 8 min read
Most local RAG pipelines retrieve the top 5–8 chunks from a vector database and dump 2,500+ tokens into the prompt. When run on sub-3B models on CPU, prefill latency spikes and hallucination rates surge. We ran a controlled 120-query ablation to demonstrate how cross-encoder distillation to Top-K=2 maximizes grounded accuracy and cuts TTFT by 75%.
95.8%
Grounded Accuracy (±1.8% SE)
1.15s
Total Latency p50 (1.23s p95)
0.10s
TTFT p50 (75% Faster)
3,049 MB
Peak Memory Footprint
SP
Suryaprakash C V
Creator, SLMAgents · Published on Medium