SLM Vision Parser
Extract text nodes, bounding boxes, chart labels, and table grids locally on CPU using local CPU-optimized visual neural models.
01 / Overview & Capabilities
The SLM Vision Parser uses local multimodal neural models to translate image structures directly. It is designed to run offline on CPU under 1.5 GB memory footprint, making it ideal for visual PDF OCR, diagram relationship mappings, and whiteboard extractions.
02 / Standalone OS Harness Installers, CLI & Python SDK
In SLMAgents 2.0, SLM Vision Parser executes inside the 10-Plane Pure-CPU Agent Harness (slm_harness). Download a self-extracting Individual Agent + Harness OS Installer for macOS, Windows, or Linux (bundling slm_vision_parser + slm_harness + the ./slm CLI), install the full 26-agent desktop suite, or install via Python pip:
1. Individual SLM Vision Parser + 10-Plane Harness Installers (All OSes)
slm_vision_parser + slm_harness + ./slm CLIslm_vision_parser + slm_harnessslm_vision_parser + slm_harness + ./slm CLI2. Full 26-Agent Studio & Desktop OS Installers
3. 10-Plane Harness CLI (./slm) & Python pip Installation
# Option A: Install via PyPI or editable source with the 10-Plane Harness $ pip install slm-vision-parser $ pip install -e ./slm_harness -e ./slm_vision_parser # Option B: Run directly through the Unified 10-Plane Pure-CPU Harness CLI (./slm) $ ./slm ask "Extract all nodes and arrow connections from @flowchart.png" $ ./slm chat # Interactive Rich TUI with 18 slash commands (/trace, /models, /arena, /undo) $ ./slm serve # 100% Offline Web AI Studio + OpenAI /v1 API at http://127.0.0.1:7860
03 / 10-Plane Pure-CPU Harness Integration (slm_harness)
Rather than executing as an isolated prompt wrapper, SLM Vision Parser is natively orchestrated by the SLMAgents 2.0 10-Plane Pure-CPU Harness (via slm_harness.runtime.legacy_bridge and CascadeRouter). Every call benefits from hybrid llama.cpp GGUF (Q4_K_M) + Quantized INT4/INT8 CPU execution, watermarked LRU memory governance, grammar-constrained decoding, and deterministic verification at $0 API cost:
| Harness Plane | Subsystem | How It Powers SLM Vision Parser on Pure CPU |
|---|---|---|
| Plane 6 | CascadeRouter (<35ms) |
Stage 1 (Image MIME / Extension Inspection: .png, .jpg, .webp, |
| Plane 2 & 5 | MemoryArena & Hybrid CPU Runtime |
Specialist Role: vision (Florence-2-base Vision Encoder + Decoder INT4/FP16 CPU). Managed inside Watermark LRU MemoryArena (lite 2.6 GB / balanced 4.7 GB / power 8.2 GB) with zero-copy session sharing and automatic OOM tier step-down (8B -> 4B -> 3B -> 0.8B). |
| Plane 7 & 8 | Deterministic Verifier & Governance | Plane 5 MemoryArena Sequential Vision Pinning + Plane 7 Bounding-Box & Task-Tag ( |
| Plane 1, 3 & 4 | Kernel, Artifact Scratchpad & 4-Tier Memory | SQLite WAL CheckpointStore (/fork, /rewind, /undo), content-addressed artifact://<sha12> handles to prevent prompt bloat, and hybrid SQLite FTS5 BM25 + HNSW vector memory. |
04 / Source Checkout (Git Sparse-Checkout)
Clone this agent alongside the 10-Plane Pure-CPU Harness (slm_harness) from the monorepo using Git sparse-checkout — no need to download all 26 agents:
Option 1 — Sparse Checkout with 10-Plane Harness (Recommended)
# 1. Create and enter a new directory $ mkdir slm_vision_parser_workspace && cd slm_vision_parser_workspace # 2. Initialise empty git repo and add remote $ git init $ git remote add origin https://github.com/t00114218-stack/SLMAgents.git # 3. Enable sparse-checkout for both slm_harness and slm_vision_parser $ git sparse-checkout init --cone $ git sparse-checkout set slm_harness slm_vision_parser # 4. Pull the 10-Plane Harness and agent source $ git pull origin main
Option 2 — Full Repository Clone
$ git clone https://github.com/t00114218-stack/SLMAgents.git $ cd SLMAgents $ ./slm doctor
Note: After checkout, install the harness and agent in editable mode with pip install -e ./slm_harness -e ./slm_vision_parser or download the pre-packaged OS Harness Installer above.
🤖 Truly Agentic Workflow
The Vision Parser maps visual coordinate dimensions into clean layout logs:
- OCR Node Extraction: Translates raster text strings from diagrams.
- Region Mapping: Generates bounding box parameters mapping visual coordinates for web rendering.
- Semantic Image Captioning: Extracts visual flow structures (e.g., arrows, shapes, labels) into a raw description.
- Agentic LLM Refinement: Pipe raw visual description nodes directly into the local neural reasoning model. By guiding the text model with a template prompt, you can synthesize exact custom descriptions (e.g., describing step hierarchies and the exact count of connecting arrows).
CPU Performance Tuning Guidelines
Local vision models utilize visual transformers. To run model prediction tasks efficiently on CPU under 2.0 GB memory:
- Sequential execution: Process page scans sequentially rather than in batches to avoid CPU memory thrashing.
- Garbage Collection: Clear cache blocks after processing high-resolution images:
import gc gc.collect()
Accuracy Improvement Tips
1024px and scaling contrast values using Pillow before parsing.
"<OCR>" or "<DETAILED_CAPTION>") to trigger correct task headers.
05 / Configuration & API Reference
SLMVisionParser Initialization
from slm_vision_parser.vision_parser import SLMVisionParser
parser = SLMVisionParser()
| Parameter | Type | Description |
|---|---|---|
| model_path | str | Local directory containing vision model checkpoints. |
| system_prompt | str | None | Optional custom system prompt instructions overriding the default template. |
| user_input | str | None | Optional additional user-supplied target parameters or variables. |
parse_image Method
Run coordinate mappings or caption generation by supplying task tags. Below is the input flowchart diagram processed by the vision model:
from slm_vision_parser.vision_parser import SLMVisionParser
parser = SLMVisionParser()
# Describe the flowchart diagram in text using the local vision LLM
result = parser.parse_image("flowchart.png", task="<DETAILED_CAPTION>")
print(result)
"A flowchart showing a start step ('Start Process') and a next step ('Next Step Link') connected with two arrows from the start step to the next step."