slm_embeddings · Data & Utilities · 10-Plane Pure-CPU Agent

SLM Embeddings Server

Starts a local CPU-optimized embedding server to compute dense document and query vectors on standard hardware.

01 / Overview & Capabilities

Starts a local CPU-optimized embedding server to compute dense document and query vectors on standard hardware.

Key Features

  • Loads quantized mini-LM or BGE embeddings locally
  • High-speed cosine similarity index built directly in memory
  • Provides local HTTP API endpoint for integration
  • Under 200 MB RAM memory usage footprint during idle states

02 / Standalone OS Harness Installers, CLI & Python SDK

In SLMAgents 2.0, SLM Embeddings Server executes inside the 10-Plane Pure-CPU Agent Harness (slm_harness). Download a self-extracting Individual Agent + Harness OS Installer for macOS, Windows, or Linux (bundling slm_embeddings + slm_harness + the ./slm CLI), install the full 26-agent desktop suite, or install via Python pip:

1. Individual SLM Embeddings Server + 10-Plane Harness Installers (All OSes)

2. Full 26-Agent Studio & Desktop OS Installers

3. 10-Plane Harness CLI (./slm) & Python pip Installation

Terminal — 10-Plane Harness CLI & Pip
# Option A: Install via PyPI or editable source with the 10-Plane Harness
$ pip install slm-embeddings
$ pip install -e ./slm_harness -e ./slm_embeddings

# Option B: Run directly through the Unified 10-Plane Pure-CPU Harness CLI (./slm)
$ ./slm serve --port 7860  # Exposes /v1/embeddings (1024-d INT8 mxbai-embed-large)
$ ./slm chat   # Interactive Rich TUI with 18 slash commands (/trace, /models, /arena, /undo)
$ ./slm serve  # 100% Offline Web AI Studio + OpenAI /v1 API at http://127.0.0.1:7860

03 / 10-Plane Pure-CPU Harness Integration (slm_harness)

Rather than executing as an isolated prompt wrapper, SLM Embeddings Server is natively orchestrated by the SLMAgents 2.0 10-Plane Pure-CPU Harness (via slm_harness.runtime.legacy_bridge and CascadeRouter). Every call benefits from hybrid llama.cpp GGUF (Q4_K_M) + Quantized INT4/INT8 CPU execution, watermarked LRU memory governance, grammar-constrained decoding, and deterministic verification at $0 API cost:

Harness PlaneSubsystemHow It Powers SLM Embeddings Server on Pure CPU
Plane 6 CascadeRouter (<35ms) Stage 2 (Vectorization / Cosine Similarity / /v1/embeddings Request)
Plane 2 & 5 MemoryArena & Hybrid CPU Runtime Specialist Role: embeddings (mxbai-embed-large-v1 1024-d INT8) + reranker (bge-reranker-base INT8). Managed inside Watermark LRU MemoryArena (lite 2.6 GB / balanced 4.7 GB / power 8.2 GB) with zero-copy session sharing and automatic OOM tier step-down (8B -> 4B -> 3B -> 0.8B).
Plane 7 & 8 Deterministic Verifier & Governance Plane 5 MemoryArena Warm Pinning (0.03ms warm latency) + L2 Normalization & Dimension (1024-d) Verifier
Plane 1, 3 & 4 Kernel, Artifact Scratchpad & 4-Tier Memory SQLite WAL CheckpointStore (/fork, /rewind, /undo), content-addressed artifact://<sha12> handles to prevent prompt bloat, and hybrid SQLite FTS5 BM25 + HNSW vector memory.

04 / Source Checkout (Git Sparse-Checkout)

Clone this agent alongside the 10-Plane Pure-CPU Harness (slm_harness) from the monorepo using Git sparse-checkout — no need to download all 26 agents:

Option 1 — Sparse Checkout with 10-Plane Harness (Recommended)

Terminal — Git Sparse Checkout
# 1. Create and enter a new directory
$ mkdir slm_embeddings_workspace && cd slm_embeddings_workspace

# 2. Initialise empty git repo and add remote
$ git init
$ git remote add origin https://github.com/t00114218-stack/SLMAgents.git

# 3. Enable sparse-checkout for both slm_harness and slm_embeddings
$ git sparse-checkout init --cone
$ git sparse-checkout set slm_harness slm_embeddings

# 4. Pull the 10-Plane Harness and agent source
$ git pull origin main

Option 2 — Full Repository Clone

Terminal — Full Clone
$ git clone https://github.com/t00114218-stack/SLMAgents.git
$ cd SLMAgents
$ ./slm doctor

Note: After checkout, install the harness and agent in editable mode with pip install -e ./slm_harness -e ./slm_embeddings or download the pre-packaged OS Harness Installer above.

05 / Configuration & API Reference

Constructor Parameters

Instantiate SLMEmbeddingsServer with performance options:

ParameterType / DefaultDescription
model_pathstr | NoneExplicit path to GGUF (Q4_K_M) or quantized ONNX model weights. If omitted, managed automatically by the 10-Plane Harness MemoryArena.
cache_dirstr | NoneDirectory to store model weights offline. Defaults to ~/.cache/slm-embeddings/. Also settable via SLM_EMBEDDINGS_SERVER_CACHE_DIR.
n_threadsint | 4CPU thread count for 10-Plane Harness inference (auto-tuned to physical P-cores). Optimize for CPU core count. Also settable via SLM_EMBEDDINGS_SERVER_N_THREADS.

Methods

Method SignatureReturn TypeDescription
embed(texts, system_prompt=None, user_input=None)list[float]Generates dense vectors from standard string lists.

Method Parameters (Execution Customization)

All main execution methods accept optional system routing parameters:

ParameterType / DefaultDescription
system_promptstr | NoneOptional custom system prompt instruction to override the default system template response parameters.
user_inputstr | NoneOptional additional user-supplied target text variables or contextual keys.

Quick Start

from slm_embeddings import SLMEmbeddingsServer

server = SLMEmbeddingsServer()
vector = server.embed(
    ["sample test"],
    system_prompt="Calculate semantic weights",
    user_input="Normalized cosine distance"
)
print(vector)

Environment Variables

Configure agent parameters globally using environment values:

Environment VariableDefaultPurpose
SLM_EMBEDDINGS_SERVER_N_THREADS4Sets CPU inference execution threads.
SLM_EMBEDDINGS_SERVER_CACHE_DIR~/.cache/slm-embeddings/Default directory to store local GGUF (Q4_K_M) and quantized INT4/INT8 model weights (shared with ~/.slm_harness/models).

CPU Performance Tuning

To run the SLMEmbeddingsServer engine efficiently on CPU under 1.5 GB memory footprint:

  • Match Threads to Core Count: Set n_threads or SLM_EMBEDDINGS_SERVER_N_THREADS to match the physical CPU core count.
  • Sequential Processing: Avoid concurrent processing when batch files are large.
  • Garbage Collection: Clear variables and run gc.collect() to release model RAM blocks after execution.

Verified Input & Output Logs

Diagnostic execution console response running locally on CPU:

→ INPUT:
"sample test"

← OUTPUT:
"Vector dimension check: 1024"