GenAI 2026 · Module 10 · RAG

Five notebooks in.
Here is what we actually built.

A quick revision before we open notebook six today. No new code in this part, just the shape of the pipeline so far.

01Why RAG

02+03Ingest & chunk

04Embeddings

05Vector DB

06Sparse retrieval · today

Notebook 01 · Why RAG

A model that never opened your files

We asked gpt-4o-mini about a company we made up, a phone that does not exist, and a private engineering decision. It answered all three, confidently, and got every single one wrong.

Closed book

"What is the home internet stipend at Northwind Logistics?"

"...a stipend of $75 per month..."

With retrieval

Same question, after we hand it the real policy document

"...a stipend of $45 per month... [HR-2024-17 §7.3]"

Hallucination Knowledge cutoff Private data invisible

Grounded

Answers come from retrieved evidence, not memory

Fresh

Add a document, it is answerable on the next query

Traceable

Every claim carries a citation back to its source

The RAG contract

One loop, repeated in every notebook since

Notebook 01 used keyword overlap as a toy retriever. Everything from notebook 02 onward is about building each of these three stages properly.

Stage 1

Retrieve

Find the documents relevant to the question

→

Stage 2

Augment

Place them into the model's context

→

Stage 3

Generate

Answer from that evidence, and cite it

Notebook 02 + 03 · Ingestion

Parsing decides what is even possible to retrieve

Before any chunking, a document has to be read correctly. Here is the same table from the same PDF, read two different ways.

PyPDFLoader · naive

Region
Shipments
On-Time %
Northwest
128,400
97.2%

PyMuPDF4LLM · layout aware

Region	Ship.	On-Time
Northwest	128,400	97.2%

Left side: you cannot tell if 128,400 is a shipment count or a revenue figure. Right side: the grid survives, so a retrieved chunk still means something.

Tier 1

Tesseract / RapidOCR

Free, local, handles most clean scans

→

Tier 2

Cloud document AI

Textract, Document Intelligence: tables and form fields

→

Tier 3

Vision model

Slow and costly, only for the hard minority

We built this twice, once in LangChain and once in LlamaIndex. Same lesson both times: the parser matters, the framework does not.

Notebook 02 + 03 · Chunking

Six ways to cut text, each one a bit smarter

1Fixed sizeCuts blindly, even through a table

2RecursivePrefers paragraph and line breaks

3SemanticCuts where the topic actually shifts

4Sentence windowMatch small, hand back the neighbours

5Parent / small-to-bigMatch a child, return its parent section

6Document-type awareCode, Markdown and HTML each split on their own structure

Framework note: strategies 4 and 5 needed manual wiring in LangChain. In LlamaIndex, SentenceWindowNodeParser and HierarchicalNodeParser do the same job in one line. That is where LlamaIndex earns its keep.

Notebook 04 · Embeddings

Meaning, turned into distance

"How do plants make food using light" and "Photosynthesis converts sunlight into glucose" share almost no words, yet they sit close together once embedded. An unrelated sentence sits far away.

The one non-negotiable rule

Index and query must use the exact same embedding model. Swap models mid-way and the "nearest neighbour" stops meaning anything.

language coverage context window dimensions vs cost domain fit

Notebook 05 · Vector databases

Same search, three levels of who runs it

FAISS

library, not a database

We built the HNSW graph and the IVF clusters ourselves, to see the mechanics.

Chroma

embedded database

Same HNSW core, plus metadata filters and persistence, zero infrastructure.

Pinecone

managed service

Someone else runs the infrastructure. You pay for that in cost and lock-in.

efSearch (HNSW) and nprobe (IVF) are the dials. Turn them up: recall climbs toward the exact-search line, latency climbs with it. Every vector database is tuning this same trade-off underneath.

Where we stand

The pipeline, five notebooks in

Ask

02·03

Parse

02·03

Chunk

Embed

Store

Retrieve

Answer

Every retrieval step so far has been either plain keyword overlap or nearest neighbour on meaning. Neither one alone is enough for real queries, and that gap is exactly where notebook six starts.

Today · Notebook 06

What happens when the words do not match

Dense embeddings find meaning. But do they ever lose to plain keyword search? And can keyword search fail completely? Let's measure it.

query: "what is a heart attack?"

gold document: "Myocardial infarction results from coronary artery occlusion..."

BM25 score: 0.000 Dense: hit

Notebook 06, sparse retrieval →

Open the notebook and we will find out exactly where BM25 wins, where it goes blind, and why production systems run it alongside dense search rather than picking one.

Five notebooks in.Here is what we actually built.