Teaching Deck: Notebook 02, Ingestion & Chunking (LangChain)

Teaching Deck · Notebook 02

Ingestion & Chunking, the LangChain way

This is the point where a PDF stops being a PDF and turns into text you can actually retrieve. Most of the RAG session gets spent on vector DBs and rerankers, but honestly, if you mess this part up nothing downstream saves you.

~45–55 min session 6 chunking strategies 4 document formats Twin of Notebook 03 (LlamaIndex)

01 / 13

Where this fits

We covered why retrieval last week. Today's the how.

Notebook 01 was the pitch: a raw LLM hallucinates because it has nothing to ground it in. Fine, but grounded in what? And how do you actually get from a folder of PDFs to something an LLM can use? That's today.

Documents

→

Parse

→

Chunk

→

Embed

→

Store

→

Retrieve

→

Generate

For the trainer

Draw this on the whiteboard before you even open the notebook. Ask them straight: "if step 2 or 3 is broken, can step 5, the vector database, fix it for you?" Let it go quiet. Nobody answers yes, and that silence is basically the whole session.

02 / 13

Part 1 · Ingestion

The money demo: one PDF, two extractors

PyPDFLoader extracts raw text. PyMuPDF4LLMLoader reconstructs layout. Same file, same table, and the table barely survives one of them.

Naive: PyPDFLoader

Region
Shipments
On-Time %
Revenue ($M)
Northwest
128,400
97.2%
14.6
West
204,910
98.1%
...

Layout-aware: PyMuPDF4LLMLoader

|Region|Shipments|On-Time %|Revenue|
|---|---|---|---|
|Northwest|128,400|97.2%|14.6|
|West|204,910|98.1%|23.9|
...

For the trainer

Cover the right panel first. Show them just the naive output and ask "which number is Northwest's on-time rate?" Nobody will be sure. Rows and columns are gone, it's just a list of numbers now. Then uncover the Markdown version. That's it, that's the whole demo, don't over-explain it.

03 / 13

Part 1 · Ingestion

Four formats, four different failure modes

Every format quietly destroys something different if you use the wrong loader.

PDF

Tables flatten to number soup

→ layout-aware parser (pymupdf4llm)

HTML

Nav/boilerplate mixed with content

→ structure-aware loader (BSHTMLLoader)

Word

Clause numbering / structure lost

→ Docx2txtLoader preserves paragraph order

Scanned image

It's just pixels. No text underneath at all.

→ OCR (RapidOCR), imperfect but real text

For the trainer

Run the OCR cell live, don't just show a screenshot. Let them find the garbage themselves, "causedby", "INTERNALMEMO", stuff like that. Point out that this noise doesn't stay contained, it flows straight into your chunks and then your embeddings. Some pipelines add a confidence gate before chunking for exactly this reason.

04 / 13

Part 2 · Chunking

Six strategies, and they build on each other

Don't treat these as six separate options to pick from. It's more of a ladder. Start at the bottom, only climb when your retrieval quality actually needs it.

1

Fixed-size

2

Recursive

3

Semantic

4

Sentence-
window

5

Parent /
small-to-big

6

Doc-type-
aware

For the trainer

Tell them straight up: when someone says "our RAG is broken," the fix is almost never a bigger model. Nine times out of ten it's one step up this ladder. I say this line at least twice a session, it's worth repeating.

05 / 13

Strategy 1 & 2

Fixed-size vs. Recursive

Same chunk size target, two very different boundary decisions.

TokenTextSplitter, naive

Cuts blindly every N tokens. Watch a boundary land mid-table.

chunk_size=80, chunk_overlap=10
→ 4 chunks | tokens 58/70/78

RecursiveCharacterTextSplitter, default

Tries \n\n, then \n, then a plain space, in that order. Whichever one gives a natural break and still fits, wins.

chunk_size=400, chunk_overlap=60
→ 3 chunks | tokens 46/83/132

For the trainer

Run both live, print the previews side by side in the terminal so it's not abstract.
Ask which one's table still looks intact. Let them find it themselves, don't just tell them.
The takeaway to land: recursive is the sane default for general text. Start there 9 times out of 10.

06 / 13

Strategy 3

Semantic chunking: cut at meaning, not length

Embed adjacent sentences, cut where the similarity between them drops off. You're finding an actual topic change instead of just counting characters and cutting blindly.

langchain_experimental.text_splitter

semantic = SemanticChunker(embeddings, breakpoint_threshold_type="percentile")
sem_chunks = semantic.create_documents([contract])
→ 2 chunks | tokens 31/141/252

For the trainer

Be honest about the cost here. It's calling an embedding model during indexing, not just at query time, so it's slower and pricier than recursive splitting. I tell students to reach for this only when one document mixes wildly different topics, not as a default choice. Also worth mentioning: it lives in langchain_experimental still, not core LangChain, so expect API drift.

07 / 13

Strategy 4

Sentence-window: retrieve small, expand at answer time

Index one precise sentence for matching; hand the model its neighbors so it has enough context to actually answer.

Matched sentence: "...home-internet stipend of $45/month..."

→

Expanded window: "...Remote-Work Reimbursement... $45/month... Contractors are not eligible."

For the trainer

Honestly the best "aha" in the whole notebook, don't rush it. Ask the class: if we only returned the matched sentence, what fact would the model never even see? Answer: contractors aren't eligible, and that's a compliance detail you really don't want silently dropped. Worth mentioning LangChain has no built-in for this, it's a manual metadata trick. Notebook 03 turns it into a one-liner in LlamaIndex, so tease that for tomorrow.

08 / 13

Strategy 5

Parent-document: small-to-big retrieval

Embed the small child chunks for precise matching. When one hits, hand back its whole parent section instead, so the model isn't working off a fragment.

Parent chunk · 600 chars

full Remote-Work Reimbursement clause

Child (150 chars) matched: "...home-internet stipend..."

→

Returned to the model: the entire parent clause, not just the child

For the trainer

Put this right next to strategy 4 and make them compare. Sentence-window fixes context with a fixed number of neighboring sentences. Parent-document fixes it using the document's actual structure, section boundaries. Ask which one a legal-contract system would want and why. Clause boundaries mean something, an arbitrary sentence count doesn't.

09 / 13

Strategy 6

Document-type-aware: stop using one splitter for everything

Code

Split on function/class boundaries (from_language)

Markdown

Split on #/## headers

HTML

Split on tag structure (h1/h2/table)

For the trainer

Make sure your Markdown demo doc actually has more than one heading. If it only has one, the splitter quietly returns 1 chunk and looks completely broken. We built this exact demo against a single-heading doc the first time and spent ten minutes convinced the splitter was buggy. It wasn't, the test doc was just wrong. Tell students this story, it's a real lesson: check that your demo is actually demonstrating what you think it is before you trust it.

10 / 13

Bonus · 2024 technique, now near-standard

Contextual Retrieval: a 7th lever you can stack on top

All six strategies so far are about where you cut. This one's different, it's about what you add to a chunk before you embed it, and you can bolt it onto any strategy above.

An LLM writes a short blurb (50 to 100 tokens), something like: "this chunk is from Northwind's Q2 report, Northeast region." That blurb gets prepended before embedding and BM25 indexing.

~35% fewer failed retrievals (context alone) ~49% (+ contextual BM25) ~67% (+ reranking)

For the trainer

Yes, it's one more LLM call per chunk, but it happens at indexing time, not on every query, so it's cheap and you can cache it. I don't build this live in class, just flag that it exists. Pairs well with reranking, which we get to in Notebook 08.

11 / 13

Suggested session flow

Roughly 50 minutes, here's how it splits

5m

10m

20m

7m

8m

Time	Segment
0–5 min	Recap Notebook 01, show the pipeline diagram, tell them today is parsing plus chunking
5–15 min	Ingestion: run the PDF naive vs. layout-aware demo live, then a quick pass on HTML/Word/OCR
15–35 min	Chunking strategies 1 through 6, live in the notebook, in order. Don't skip the ramp framing, it's the thread that ties everything together
35–42 min	Contextual Retrieval as a bonus concept, no live coding needed here
42–50 min	Summary table, then hand off to Notebook 03 (same ideas, different library)

12 / 13

Anticipate these questions

What students usually ask at this point

"Why not just always use strategy 6, the most advanced one?"

Cost and complexity climb with the ramp. Semantic and small-to-big both need extra embedding calls and more infrastructure to run. Start simple, escalate only once retrieval quality actually demands it.

"Can't I just make the chunk size bigger and be done with it?"

No. Go too large and you blur several topics into one embedding, which hurts precision. There's no free lunch here, chunk size is a real knob you have to tune, not something you max out and forget.

"Should I always turn overlap on?"

Overlap cuts down on information lost at boundary cuts, but it costs you storage and you'll retrieve redundant text. 10 to 20% is the usual range. Anything much bigger is just waste.

"Which one should I actually use in my own project?"

Recursive by default, basically always. Add sentence-window if answers feel starved of context. Add parent-document for long structured docs like contracts or manuals. Save semantic chunking for corpora that actually mix unrelated topics.

13 / 13