At 2am, a machine throws an alarm the night operator has never seen. The answer is in the maintenance manual: page 612 of a 900-page PDF, in a table whose header is on page 611, in a binder someone borrowed. So the machine sits until day shift. Multiply that by every control, every option, every parameter question, and you've found one of the most useful jobs a language model can do on a shop floor. It isn't writing G-code. It's finding the right page and quoting it accurately.
That's what retrieval-augmented generation (RAG) is: search your documents for the relevant passages, hand only those passages to a language model, and make it answer from them, with citations. The model doesn't need to have memorized FANUC's alarm list. It needs to read the right two pages and not make anything up. This post builds that system locally, on one PC, with no data leaving the building.
Why local, and why that's realistic now
Machine manuals are copyrighted and often under NDA, and plenty of shop networks are deliberately air-gapped. A 7–8B-parameter open model running under Ollama on a desktop with a mid-range GPU is plenty for "read these passages and answer," because retrieval does the hard part. You're not asking the model to know anything, only to read.
Why naive RAG fails on machine manuals
The 15-minute tutorial version (split PDFs into 1,000-character chunks, embed them, retrieve the top 5) works on blog posts and falls apart on technical manuals. I've watched it fail in exactly three ways:
- Codes defeat embeddings. Semantic embeddings are great at meaning and bad at identifiers. Ask about
SV0436and vector search happily returnsSV0437, or a paragraph about servo alarms in general. Alarm numbers and parameter numbers need exact keyword matching. - Fixed-size chunks cut entries in half. An alarm entry is code, title, cause, remedy. A 1,000-character window splits it mid-remedy and glues on half of the next alarm. The model then confidently mixes the two.
- Tables lose their headers. Plain text extraction flattens a parameter table into a soup of numbers. The row survives; the column meaning doesn't.
Every fix below targets one of those three. Get them right and a modest local model gives answers you can trust. Get them wrong and no model size will save you.
Step 1: parse, keeping page numbers
Use PyMuPDF (pip install pymupdf) for fast, layout-aware text extraction, and keep the page number with every piece of text: it's your citation. For table-heavy manuals (parameter lists), a structure-aware parser like [Docling](https://github.com/docling-project/docling) or pdfplumber keeps rows and headers together. Test both on your worst manual; what works depends on how the PDF was produced.
Step 2: chunk by structure, not by size
Manuals are already chunked for you: one alarm per entry, one parameter per entry. Split on those boundaries with a regex, and fall back to paragraph-sized chunks only for prose sections. Each chunk carries its manual, page, and the code it describes as metadata:
import re
import fitz # PyMuPDF
# Alarm entries usually start a line with a code like "SV0436" or "PS0010".
# Adjust this pattern per manual; it's the most important line in the file.
ENTRY = re.compile(r"^(?P<code>(?:SV|PS|OT|SW|DS|IO|SP|OH|EX)\d{3,4})\s+(?P<title>.+)$", re.M)
def chunks_from_pdf(path, manual_name):
doc = fitz.open(path)
for page_no, page in enumerate(doc, start=1):
text = page.get_text("text")
starts = list(ENTRY.finditer(text))
if not starts: # prose page: paragraph chunks
for para in re.split(r"\n\s*\n", text):
if len(para.strip()) > 80:
yield dict(manual=manual_name, page=page_no, code=None, text=para.strip())
continue
for i, m in enumerate(starts): # one chunk per alarm entry
end = starts[i + 1].start() if i + 1 < len(starts) else len(text)
yield dict(manual=manual_name, page=page_no, code=m["code"],
text=text[m.start():end].strip())Entries that continue onto the next page need one more pass: if a page's first lines don't start with a code, append them to the last chunk of the previous page. Ten minutes spent on chunking beats ten hours of prompt tuning.
Step 3: hybrid search, keywords plus meaning
This is the fix for problem #1, and it's the one most tutorials skip. Store every chunk twice: in a SQLite FTS5 full-text index for exact matches (alarm codes, parameter numbers, part numbers), and as an embedding for meaning ("spindle won't orient," "Z axis drifts when warm"). At query time, run both and merge them. If the question contains a code, the keyword hit goes first, always.
import json, re, sqlite3
import numpy as np
import requests
OLLAMA = "http://localhost:11434"
EMBED_MODEL = "nomic-embed-text" # pick one and never mix models in one index
def embed(texts):
r = requests.post(f"{OLLAMA}/api/embed", json={"model": EMBED_MODEL, "input": texts})
r.raise_for_status()
v = np.array(r.json()["embeddings"], dtype=np.float32)
return v / np.linalg.norm(v, axis=1, keepdims=True) # normalize for cosine
db = sqlite3.connect("manuals.db")
db.execute("CREATE VIRTUAL TABLE IF NOT EXISTS chunks USING fts5(manual, page, code, text)")
def search(question, k=6):
hits = {}
# 1. exact codes in the question win outright
for code in re.findall(r"\b[A-Z]{2}\d{3,4}\b", question.upper()):
for row in db.execute("SELECT rowid, manual, page, code, text FROM chunks WHERE code = ?", (code,)):
hits[row[0]] = (10.0, row)
# 2. keyword relevance (BM25) for everything else
terms = " OR ".join(f'"{w}"' for w in re.findall(r"\w{3,}", question)) # quoted: safe FTS syntax
if terms:
for row in db.execute(
"SELECT rowid, manual, page, code, text, bm25(chunks) FROM chunks "
"WHERE chunks MATCH ? ORDER BY bm25(chunks) LIMIT 20", (terms,)):
hits.setdefault(row[0], (1.0, row[:5]))
# 3. semantic similarity (row i of vectors.npy = chunk with rowid i + 1, built at ingest)
vecs = np.load("vectors.npy")
sims = vecs @ embed([question])[0]
for rowid in np.argsort(sims)[-20:][::-1] + 1:
if rowid not in hits:
row = db.execute("SELECT rowid, manual, page, code, text FROM chunks WHERE rowid = ?",
(int(rowid),)).fetchone()
hits[rowid] = (float(sims[rowid - 1]), row)
return [row for _, row in sorted(hits.values(), key=lambda h: -h[0])][:k]Building the index is a one-off script: insert every chunk, then embed them in order so row i of the vector file lines up with SQLite rowid i + 1 on a fresh table:
from pathlib import Path
rows = [c for pdf in sorted(Path("manuals").glob("*.pdf"))
for c in chunks_from_pdf(pdf, pdf.stem)]
db.executemany("INSERT INTO chunks (manual, page, code, text) VALUES (?, ?, ?, ?)",
[(r["manual"], r["page"], r["code"], r["text"]) for r in rows])
db.commit()
vecs = np.vstack([embed([r["text"] for r in rows[i:i + 64]]) # batches of 64
for i in range(0, len(rows), 64)])
np.save("vectors.npy", vecs)A numpy matrix in memory is plenty for a few thousand pages: a dot product over 20,000 vectors takes milliseconds. Reach for a vector database (sqlite-vec, Chroma, LanceDB) when you're indexing every manual in a multi-site company, not before.
Step 4: answer only from the sources, and cite them
The prompt is where you buy trustworthiness. Number the retrieved passages, tell the model to answer only from them, cite by number, and say so plainly when the answer isn't there. Then show the operator the citations as clickable manual, page links, so they read the real page before they act:
CHAT_MODEL = "qwen3:8b" # any local instruct model that follows instructions well
SYSTEM = """You answer questions about CNC machines using ONLY the numbered sources.
Cite every claim like [2]. If the sources do not contain the answer, reply exactly:
"Not found in the indexed manuals." Never guess parameter values or alarm causes.
Never tell the user to change a parameter without quoting the source text for it."""
def ask(question):
sources = search(question)
context = "\n\n".join(
f"[{i}] {m}, p.{p}{' (' + c + ')' if c else ''}\n{t}"
for i, (_, m, p, c, t) in enumerate(sources, start=1))
r = requests.post(f"{OLLAMA}/api/chat", json={
"model": CHAT_MODEL, "stream": False,
"messages": [{"role": "system", "content": SYSTEM},
{"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {question}"}]})
r.raise_for_status()
answer = r.json()["message"]["content"]
refs = [f"[{i}] {m} p.{p}" for i, (_, m, p, _, _) in enumerate(sources, start=1)]
return answer, refsAdvisory, never actuating
This assistant reads manuals; it never writes to a machine. Parameter changes, alarm resets, and anything that moves an axis stay with a qualified person who has read the cited page. Make that a hard rule in the system prompt and in the architecture: the RAG service shouldn't have network access to the controls at all.
Step 5: measure it before you trust it
Before anyone uses it at 2am, build a test set: 30–50 real questions from maintenance tickets and operator logs, each with the manual page that answers it. Then measure two numbers separately:
- Retrieval hit rate: is the correct page in the top 6 sources? If this is below ~90%, fix chunking and search first; no prompt will rescue missing context.
- Answer faithfulness: does every claim match the cited source? Read them yourself. It's tedious, and it's the only check that counts.
Re-run the test set every time you change the chunker, the embedding model, or the chat model. It turns "it seems better" into a number.
What it won't do (and shouldn't)
- It doesn't know your machine's history. The manual says what alarm 436 can mean; your maintenance log says what it usually means on machine 7. Index your own repair notes as a second source and the assistant gets dramatically more useful.
- It can't read diagrams. Wiring diagrams and ladder screenshots are images. Index their captions and figure numbers so at least the right page comes back.
- It's only as current as the index. Re-index when a manual revision arrives, and show the revision in the citation.
The model doesn't need to know the manual. It needs to find the right page and refuse to improvise beyond it.
Once it works in a terminal, wrap ask() as a tool in an MCP server so any assistant on your network can call it, or put a single text box in front of it with Flask or a small desktop app. New to the idea? Start with what an MCP server is and why it matters for manufacturing.


