Notebook 08 · Retrieval

Fusion gets it into the list. Reranking puts it on top.

Notebook 07 proved hybrid fusion beats either retriever alone. But "beats" isn't "perfect", the right document can still land at #2 or #3. Today's job is a second pass that fixes exactly that.

BM25 + Dense
notebook 06
→
RRF fusion
notebook 07
→
Rerank
today

The setup

We rebuild notebook 07's pipeline, then add one stage

Same 96 passages, same 16 gold-labeled queries. BM25 and bge-small retrieve, RRF fuses them, exactly as before. Reranking slots in right after.

BM25
+
Dense
→
RRF fusion
→
rerank, today

Why does fusion need a second stage at all? Because RRF and weighted fusion are still built from fast, shallow scorers. They're excellent at deciding which 20 documents deserve a look. They're not always right about which one of those 20 is best.

Back up one step first

Cosine similarity doesn't know what a word means

Every retriever so far, notebook 04's embeddings, notebook 05's vector database, notebook 07's dense scores, ends with the same tiny bit of arithmetic: cosine similarity, comparing two lists of numbers. That arithmetic has no idea what "HTTP logs" means. It just measures an angle. So the real question is: who turns the sentence into a list of numbers in the first place?

"How do I reduce noisy HTTP logs?"
→
TRANSFORMER
→
[0.12, −0.87, 0.33, ...]

That's the missing piece. The transformer, the same self-attention architecture from module 0, GPT's own family, has been running quietly since notebook 04, every single time you called encoder.encode(text).

encoder = SentenceTransformer("bge-small-en-v1.5")
encoder.encode("How do I reduce noisy HTTP logs?") # this line runs a transformer
Cosine similarity was never a replacement for the transformer. It's the tiny bit of plain math that happens after the transformer already did the hard work of turning words into numbers.

Now map that onto bi-encoder vs cross-encoder

Same transformer. Two completely different jobs.

query text→transformer→vector A
doc text→transformer→vector B→cosine(A, B)

That's the bi-encoder you already know, run the same transformer twice, separately, then compare the two outputs with cosine similarity. The cross-encoder below is the exact same kind of neural network, used differently: run once, on both texts glued together, no cosine similarity step at all.

Think of hiring for one job with 1,000 resumes. You can't read every resume next to the job description, so you summarize the job once, and summarize each resume once, separately, then compare summaries. Fast, but a bit shallow. For your final shortlist of 20, you actually sit down and read the job description and the resume together. Slower, but far more accurate. That's the whole idea below.
Bi-encoder, first stage
Two separate forward passes, at two separate times. The transformer never has both texts in front of it at once, so it never gets to compare them word-by-word.
pass 1, done ahead of time: "Raise the log level of the HTTP client library..."
pass 2, done at search time: "How do I reduce noisy per-request HTTP logs?"
Each pass alone produces one vector. The two vectors are compared afterward, outside the model, with plain cosine similarity.
fast, shallow, no query-document interaction
Cross-encoder, second stage
One forward pass. Query and document are glued into a single input, so every word of the query and every word of the document sit inside the same self-attention computation.
[CLS] How do I reduce noisy per-request HTTP logs from our service? [SEP] Raise the log level of the HTTP client library... [SEP]
one sequence in, self-attention lets "HTTP logs" and "log level" attend directly to each other
Out comes not a vector, but a single number: how well this exact document answers this exact question.
slow, deep, judges the actual pair
1 encode
the query, once. Documents were already encoded ahead of time and just get reused
20 encodes
query re-paired with every one of the top 20 candidates, redone for every single query

That's why the cross-encoder only ever touches the shortlist fusion already built, never the whole 96-document corpus. Doing the careful read 20 times per query is fine. Doing it 96 million times would never finish.

A real query where hybrid gets it wrong

Watch the two rounds happen, in order

query"How do I reduce noisy per-request HTTP logs from our service?"
gold document: doc 1, "Raise the log level of the HTTP client library..."
1
Round 1: hybrid shortlists candidates
Fast, separate encoding pulls the top few likely documents out of all 96. Gold document is in there, just not on top: it lands at #2.
→
2
Round 2: cross-encoder re-reads each one
Only these few candidates get the slow, careful "query and document together" treatment. It directly judges each one against this exact question.
→
3
New order comes out
The gold document moves from #2 to #1. Nothing new was retrieved, the same shortlist just got sorted more carefully.
step 1 result, before rerank
gold @ #2
top 1 was: "Batch multiple requests together to amortize fixed per-call..." — plausible-sounding, but not the answer.
step 3 result, after rerank
gold @ #1
top 1 is now: "Raise the log level of the HTTP client library..." — the document that actually answers the question.

Before the details

Five vendors for the same job: round 2

These next five slides aren't five different ideas. They're five different tools that all do the exact same job, the careful "read query and document together" re-read, just with different tradeoffs of speed, cost, and quality.
cheapest, fastestmost expensive, slowest
1
FlashRank
free, CPU
2
Cross-encoder
self-hosted default
3
Cohere
managed API
4
ColBERT
token-level
5
LLM listwise
whole-list reasoning

Notice the order here is roughly speed-first, it doesn't match the order we'll walk through next, the notebook introduces them by mechanism instead (pointwise, then lightweight, then listwise, then managed, then late-interaction). Keep this picture in mind as the throughline.

Strategy 1 of 5

1

Cross-encoder, pointwise

Scores each (query, document) pair independently, then sorts by score. This is what just fixed the HTTP logs query. ms-marco-MiniLM is small and fast; BGE-reranker-v2-m3 is the larger, higher-quality open self-hosted option.

the production default self-hosted pointwise

Strategy 2 of 5

2

FlashRank, lightweight and CPU-only

A quantized cross-encoder with no torch dependency. Same pointwise idea as strategy 1, tuned to be tiny and run anywhere, including serverless functions and edge devices with no GPU.

free CPU-only slightly lower ceiling than a full cross-encoder

Strategy 3 of 5

3

LLM listwise, RankGPT-style

Instead of scoring pairs one at a time, hand the LLM the whole candidate list at once and ask it to reorder them. It can reason about nuance and relative context across the whole set, not just one pair in isolation.

highest quality, most flexible slowest priciest, one LLM call per query

Strategy 4 of 5

4

Managed API, Cohere Rerank

No model to host, no infra to run, a per-call API instead. The notebook actually called it live:

query: "How do I fix a 502 Bad Gateway error from our API?"
Cohere top-1 → "A 502 Bad Gateway from the API gateway means the upstream se..." gold ✓
zero-ops strong quality per-call cost + vendor dependency

Strategy 5 of 5

5

Late interaction, ColBERT / RAGatouille

Covered conceptually, not run live, it needs heavier infrastructure. Instead of one vector per document, ColBERT keeps one vector per token and scores with MaxSim across all of them, finer-grained than a single-vector bi-encoder. It can even serve as a first-stage retriever, not just a reranker. Reach for it when single-vector recall plateaus on token-sensitive corpora, code search, part numbers, that kind of thing.

token-level precision needs a multi-vector index GPU for scale

The proof, across all 16 queries

Reranking lifts the fused baseline

MethodMRRR@1R@3nDCG@10
Hybrid, no rerank0.9690.941.000.977
Hybrid + FlashRank1.0001.001.001.000
Hybrid + Cross-encoder1.0001.001.001.000
Hybrid + LLM listwise1.0001.001.001.000
Hybrid + Cohere Rerank0.9690.941.000.977

Three of the four rerankers lift the fused baseline to a perfect score. Cohere Rerank matched the baseline exactly, it didn't fix the one query the others did, and it didn't break anything either.

Before drawing conclusions

Reading the numbers honestly

With only 16 queries, a one-query difference is a 6.25 percentage-point swing in every metric. This is not a leaderboard ranking the rerankers against each other, it's proof that reranking, as a category, works.

The two takeaways the notebook actually supports:

reranking integrates cleanly into a hybrid pipeline, including a managed API
all five approaches are production-strong, benchmark on your own labeled set before choosing

The tradeoff nobody skips

Quality has to be paid for in latency, somewhere

Measured on the same 20 candidates per query.

Cross-encoder
47 ms
FlashRank
125 ms
LLM listwise
1135 ms

The surprise: the full cross-encoder actually ran faster than FlashRank here, small local models don't always sort the way their category label suggests, and this is exactly why the notebook keeps saying "benchmark on your own set." LLM listwise is over 20× slower than either, that's a real network round trip plus generation on top of an already-slow strategy.

Three shapes of "rerank"

Pointwise, pairwise, listwise

Pointwise
Score each pair independently, then sort. Simple, fast, parallelizable.
cross-encoders, FlashRank, Cohere
Pairwise
Compare two documents directly: "is A more relevant than B?" Higher-quality signal, needs more comparisons.
less common in production
Listwise
Order the whole candidate list at once, using full context across every candidate together.
LLM / RankGPT-style
RerankerQualityLatencyUse when
FlashRankgoodvery low, CPUedge / serverless / cost-sensitive
Cross-encoderhighmediumdefault self-hosted choice
Cohere / Jinahighlow–medzero-ops, no model to run
LLM listwisehighesthighsmall candidate sets, nuanced intent
ColBERThighmediumtoken-sensitive corpora at scale
Rule of thumb for candidate-set size: retrieve top 20–100 from fusion, rerank down to top 3–10 for the generator. Bigger candidate sets improve recall into the reranker but cost latency, tune it against your own eval set.

Notebook 08 · summary

The retrieval half of RAG, complete

06
BM25
06
Dense
07
Fuse
08
Rerank
01
Answer
Reranking is a precision second stage over fused candidates: retrieve@k → rerank → top-5.
Cross-encoders are the default. FlashRank for CPU-only. LLM listwise for the hardest, smallest sets. Cohere/Jina for zero-ops. ColBERT for token-level at scale.
The robust, compounding gain is the pipeline itself: sparse + dense, then fuse, then rerank, each stage fixing what the last one couldn't.
Next, notebook 09: evaluating the pipeline with RAGAS →