"Chat with your PDF" sounds like magic: drop in a 150-page contract, ask "what's the termination penalty?", get the answer with a page number. Under the hood it's a pattern every AI engineer should understand — because it's retrieval-augmented generation (RAG) in miniature, and it's the architecture behind most serious enterprise AI systems.
This post walks through the pipeline using our free Chat with PDF tool as the working example. You can follow along with any document.
The problem: models have a context budget
A language model can only read so much text per request, and every character costs money and latency. A 150-page contract is ~300k characters; sending all of it for every question is wasteful — and the naive alternative most simple tools use is worse: truncate at a fixed size and silently ignore the rest. Ask about page 100 of a tool that reads 25 pages and you get a confident "the document doesn't cover that." Wrong, and unverifiable.
The fix is a three-stage pipeline: extract → chunk → retrieve.
Stage 1: Extraction (in your browser)
First, the PDF becomes text. Our tool does this client-side with pdf.js — the file never leaves your device; only extracted text is sent for analysis. Crucially, extraction is page-aware: each page's text keeps its page number, because citations later depend on it.
(Scanned PDFs are images, not text — they need OCR, which is a different pipeline. If extraction comes back nearly empty, that's why.)
Stage 2: Chunking
The document is split into chunks of a few paragraphs each, aligned to paragraph boundaries so ideas aren't cut mid-sentence, each chunk remembering its page. Chunk size is a real design decision: too big and retrieval gets imprecise; too small and chunks lose the context that makes them answerable.
Stage 3: Retrieval — the interesting part
When you ask a question, the system doesn't send the whole document. It ranks every chunk against your question and sends only the most relevant ones, within a fixed budget the model can handle.
There are two families of ranking:
- Lexical — score chunks by overlapping terms with the question. Fast, free, runs anywhere (ours runs in your browser). Weakness: "how much do I owe if I quit early?" may miss a clause worded as "termination penalty" if no words overlap.
- Semantic (embeddings) — turn the question and chunks into vectors and rank by meaning, not wording. Catches synonyms; costs an extra model call and vector infrastructure.
Production RAG systems usually combine both. For a free tool, lexical retrieval with well-chosen chunks covers the large majority of real questions at zero added cost — and the summary path uses a different strategy entirely, sampling chunks across the whole document (including the end!) so long documents get summarized fairly rather than "summarize the first N pages."
Citations: the honesty layer
Because chunks carry page numbers, the model can be instructed to cite — "the termination penalty is $50,000 (p. 100)" — and you can verify instead of trust. Just as important is what the model is told about what it can't see: when it only received excerpts, it must say "the excerpts don't cover that" rather than "the document doesn't say," because it hasn't read the whole document. That one prompt rule is the difference between a trustworthy tool and a confident liar.
Try it, then build it
Drop a long PDF into the free analyzer — you'll see the coverage note on long documents, page citations in the answers, and honest refusals for questions the text doesn't answer. Then ask yourself how you'd scale it: embeddings for retrieval, a vector store, reranking, evaluation sets…
That path — from this exact pattern to production RAG systems — is what we teach at MindloomHQ: real agent projects, including document Q&A over real data, portfolio-ready. The free tool is the demo; building it yourself is the skill.