slm_rag · Productivity & RAG · 10-Plane Pure-CPU Agent

SLM RAG

Local Retrieval-Augmented Generation engine for offline document indexing, dense vector search, and grounded Q&A.

01 / Overview & Capabilities

Local Retrieval-Augmented Generation engine for offline document indexing, dense vector search, and grounded Q&A.

Key Features

  • Offline 1024-d (mxbai-embed-large) & 384-d CPU embedding generation + cross-encoder reranking via the 10-Plane Harness
  • Local vector similarity indexing
  • Grounded factual document question answering
  • Strict context window enforcement

02 / Standalone OS Harness Installers, CLI & Python SDK

In SLMAgents 2.0, SLM RAG executes inside the 10-Plane Pure-CPU Agent Harness (slm_harness). Download a self-extracting Individual Agent + Harness OS Installer for macOS, Windows, or Linux (bundling slm_rag + slm_harness + the ./slm CLI), install the full 26-agent desktop suite, or install via Python pip:

1. Individual SLM RAG + 10-Plane Harness Installers (All OSes)

2. Full 26-Agent Studio & Desktop OS Installers

3. 10-Plane Harness CLI (./slm) & Python pip Installation

Terminal — 10-Plane Harness CLI & Pip
# Option A: Install via PyPI or editable source with the 10-Plane Harness
$ pip install slm-rag
$ pip install -e ./slm_harness -e ./slm_rag

# Option B: Run directly through the Unified 10-Plane Pure-CPU Harness CLI (./slm)
$ ./slm ask "What is the total Q3 net revenue in @financial_report.pdf?"
$ ./slm chat   # Interactive Rich TUI with 18 slash commands (/trace, /models, /arena, /undo)
$ ./slm serve  # 100% Offline Web AI Studio + OpenAI /v1 API at http://127.0.0.1:7860

03 / 10-Plane Pure-CPU Harness Integration (slm_harness)

Rather than executing as an isolated prompt wrapper, SLM RAG is natively orchestrated by the SLMAgents 2.0 10-Plane Pure-CPU Harness (via slm_harness.runtime.legacy_bridge and CascadeRouter). Every call benefits from hybrid llama.cpp GGUF (Q4_K_M) + Quantized INT4/INT8 CPU execution, watermarked LRU memory governance, grammar-constrained decoding, and deterministic verification at $0 API cost:

Harness PlaneSubsystemHow It Powers SLM RAG on Pure CPU
Plane 6 CascadeRouter (<35ms) Stage 2 (Document/Query Signature) & Stage 3 (1024-d Embedding Margin)
Plane 2 & 5 MemoryArena & Hybrid CPU Runtime Specialist Role: embeddings (mxbai-embed-large INT8) + reranker (bge-reranker-base) + primary (Qwen3-4B GGUF Q4_K_M / Llama-3.2-3B INT4). Managed inside Watermark LRU MemoryArena (lite 2.6 GB / balanced 4.7 GB / power 8.2 GB) with zero-copy session sharing and automatic OOM tier step-down (8B -> 4B -> 3B -> 0.8B).
Plane 7 & 8 Deterministic Verifier & Governance Plane 7 Citation Grounding Verifier ([1], [2] span-level verification against retrieved chunks) + Context Distillation (Top-K=2)
Plane 1, 3 & 4 Kernel, Artifact Scratchpad & 4-Tier Memory SQLite WAL CheckpointStore (/fork, /rewind, /undo), content-addressed artifact://<sha12> handles to prevent prompt bloat, and hybrid SQLite FTS5 BM25 + HNSW vector memory.

04 / Source Checkout (Git Sparse-Checkout)

Clone this agent alongside the 10-Plane Pure-CPU Harness (slm_harness) from the monorepo using Git sparse-checkout — no need to download all 26 agents:

Option 1 — Sparse Checkout with 10-Plane Harness (Recommended)

Terminal — Git Sparse Checkout
# 1. Create and enter a new directory
$ mkdir slm_rag_workspace && cd slm_rag_workspace

# 2. Initialise empty git repo and add remote
$ git init
$ git remote add origin https://github.com/t00114218-stack/SLMAgents.git

# 3. Enable sparse-checkout for both slm_harness and slm_rag
$ git sparse-checkout init --cone
$ git sparse-checkout set slm_harness slm_rag

# 4. Pull the 10-Plane Harness and agent source
$ git pull origin main

Option 2 — Full Repository Clone

Terminal — Full Clone
$ git clone https://github.com/t00114218-stack/SLMAgents.git
$ cd SLMAgents
$ ./slm doctor

Note: After checkout, install the harness and agent in editable mode with pip install -e ./slm_harness -e ./slm_rag or download the pre-packaged OS Harness Installer above.

05 / Configuration & API Reference

Constructor Parameters

Instantiate SLMRag with performance and runtime options:

ParameterType / DefaultDescription
model_pathstr | NoneLocal GGUF (.gguf) or quantized ONNX model path managed by MemoryArena. Default: None (auto-resolved via slm_harness).
temperaturefloat | 0.0Sampling temperature. Default: 0.0.
top_pfloat | 0.9Nucleus sampling threshold. Default: 0.9.
max_tokensint | 256Maximum output token limit. Default: 256.
n_threadsint | 4CPU threads. Default: 4.

Methods

Method SignatureReturn TypeDescription
answer(question, chunks, ...)str | dictRetrieves matching context chunks and generates factual response.

Execution Parameters

Complete list of execution parameters accepted by the primary agent method:

ParameterType / DefaultDescription
questionstrUser question string.
chunkslist[str] | strDocument context text or chunk list.
instructionstr | NoneExtraction instruction guideline. Default: None.
system_promptstr | NoneSystem prompt instruction. Default: None.
user_inputstr | NoneContextual key values. Default: None.
temperaturefloat | 0.0Sampling temperature. Default: 0.0.
top_pfloat | 0.9Nucleus sampling probability. Default: 0.9.
max_tokensint | 256Maximum token limit. Default: 256.

Quick Start

Python Example
from slm_rag import SLMRag

rag = SLMRag(temperature=0.0, top_p=0.9, max_tokens=256)
res = rag.answer(
    question="What is total Q3 net revenue?",
    chunks="Q3 net revenue reached $1.25M., Due date: Sept 2026",
    instruction="Extract exact numerical totals",
    system_prompt="Factual document extraction",
    user_input="Currency: USD",
    temperature=0.0,
    top_p=0.9,
    max_tokens=256
)
print(res)

06 / Verified Execution Logs

Diagnostic execution console output running locally on CPU:

Output Console
→ INPUT:
Question: What is total Q3 net revenue?

← OUTPUT:
{
  'status': '200 OK',
  'answer': 'Based on documents: Q3 net revenue is $1.25M USD.',
  'retrieved_chunks': 2
}