Capsule 04: Reciprocal Rank Fusion — the simple formula that combines rankings from any source

Capsule description

You have two rankings: one from BM25 (capsule 03) and one from semantic search. How do you combine them into a single unified ranking? The intuitive answer is "average the scores", but that intuition breaks in practice — the scores have completely different ranges (BM25 runs from 0 to 50+, cosine similarity from 0 to 2). Adding apples to oranges gives meaningless results.

Reciprocal Rank Fusion (RRF) solves the problem by changing the frame: instead of combining scores, it combines positions in the ranking. A document in position #1 contributes 1/(60+1) = 0.0164 to its final score, no matter which algorithm produced that ranking. A document in position #5 contributes 1/(60+5) = 0.0154. The fusion is robust to any difference between the source rankings.

This capsule teaches you the formula, its trivial implementation (~10 lines), why the parameter k=60 is the reasonable default, when it's worth tuning, and the typical failure modes.

By the end of this capsule you'll be able to:

• ✅ Implement RRF in under 15 lines with correct typing
• ✅ Explain why RRF works where "adding scores" breaks
• ✅ Tune the k parameter for your case (default: 60, useful range: 20-100)
• ✅ Decide when RRF is enough vs when weighted fusion is worth it (capsule 05)
• ✅ Anticipate the failure modes: rankings that are too short, score sign, deduplication
• ✅ Combine more than two rankings (3+ signals) without rewriting the logic

Estimated time: 25-30 minutes

The insight: combine positions, not scores

To understand why RRF wins, look at what happens with the intuitive "add the scores" option:

# Ranking 1: BM25
bm25 = [
    ("doc_A", 12.5),  # rank 1
    ("doc_B", 8.2),   # rank 2
    ("doc_C", 5.1),   # rank 3
]

# Ranking 2: Semantic (cosine distance, lower = better)
semantic = [
    ("doc_C", 0.21),  # rank 1
    ("doc_A", 0.34),  # rank 2
    ("doc_D", 0.42),  # rank 3
]

# Naively adding scores:
# doc_A: 12.5 + 0.34 = 12.84  ← BM25 dominates by magnitude
# doc_B: 8.2 + ?              ← doesn't appear in semantic
# doc_C: 5.1 + 0.21 = 5.31    ← cosine dominates if we invert it
# doc_D: ? + 0.42             ← doesn't appear in BM25

The problems:

1. Incomparable magnitudes: BM25 runs from 0 to 50+, cosine from 0 to 2. The sum is always dominated by BM25, regardless of real quality.
2. Handling "doesn't appear": what score do you give a doc that's in one but not the other? Put 0, and you penalize it too much. Ignore it, and you lose information.
3. Cosine is "lower = better", BM25 is "higher = better": opposite signs. You have to invert one.

RRF changes the frame: instead of scores, it uses positions. Each document contributes 1 / (k + rank) for every ranking where it appears. Positions are on the same scale (1, 2, 3, ...) no matter what algorithm produced them.

# With RRF (k=60):
# doc_A: 1/(60+1) + 1/(60+2) = 0.0164 + 0.0161 = 0.0325  ← high rank in both
# doc_C: 1/(60+3) + 1/(60+1) = 0.0159 + 0.0164 = 0.0323  ← high rank in both
# doc_B: 1/(60+2)            = 0.0161                    ← only in BM25
# doc_D: 1/(60+3)            = 0.0159                    ← only in semantic

# Final ranking: doc_A, doc_C, doc_B, doc_D

The reading: docs that show up high in both rankings win (doc_A, doc_C). Docs that show up in only one with a decent rank come after. The fusion reflects the "combined confidence" of the two signals.

The complete formula

RRF_score(doc) = Σ_i  1 / (k + rank_i(doc))

where:
  - i iterates over every ranking you're fusing
  - rank_i(doc) = the doc's position in ranking i (1-based: 1, 2, 3, ...)
  - k = a constant (default 60)
  - If the doc does NOT appear in ranking i, that term is 0

Why k=60 is the default: it comes from the original paper (Cormack et al., 2009). Empirically it works well in most cases. An important detail: k controls how "decisive" high vs low rankings are.

# With k=60 (default)
1/(60+1)  = 0.0164  # rank 1
1/(60+5)  = 0.0154  # rank 5
1/(60+10) = 0.0143  # rank 10
1/(60+50) = 0.0091  # rank 50

# Gap between rank 1 and rank 50: 0.0164 / 0.0091 = 1.8x

# With k=10 (more decisive toward high ranks)
1/(10+1)  = 0.0909  # rank 1
1/(10+5)  = 0.0667  # rank 5
1/(10+50) = 0.0167  # rank 50

# Gap between rank 1 and rank 50: 0.0909 / 0.0167 = 5.4x

A small k makes high rankings dominate. A large k flattens the differences.

The implementation

# rrf.py
from typing import List
from collections import defaultdict


def reciprocal_rank_fusion(
    rankings: List[List[str]],
    k: int = 60,
) -> List[tuple[str, float]]:
    """
    Fuses N rankings using Reciprocal Rank Fusion.

    Args:
        rankings: a list of rankings. Each ranking is a list of doc_ids sorted
                  by descending relevance (rank 1 = best).
        k: the RRF constant. Default 60 (from the original paper).

    Returns:
        A list of (doc_id, rrf_score) sorted by descending score.
    """
    rrf_scores: dict[str, float] = defaultdict(float)

    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            rrf_scores[doc_id] += 1.0 / (k + rank)

    sorted_results = sorted(rrf_scores.items(), key=lambda x: -x[1])
    return sorted_results


# Try it
bm25_ranking = ["doc_A", "doc_B", "doc_C", "doc_D", "doc_E"]
semantic_ranking = ["doc_C", "doc_A", "doc_F", "doc_B", "doc_G"]

fused = reciprocal_rank_fusion([bm25_ranking, semantic_ranking])
print("Fused ranking:")
for doc_id, score in fused:
    print(f"  {doc_id}: {score:.4f}")

Output:

Fused ranking:
  doc_A: 0.0325   ← top in BM25 (#1), high in semantic (#2)
  doc_C: 0.0323   ← middling in BM25 (#3), top in semantic (#1)
  doc_B: 0.0318   ← #2 in BM25, #4 in semantic
  doc_F: 0.0159   ← only in semantic (#3)
  doc_D: 0.0156   ← only in BM25 (#4)
  doc_E: 0.0154   ← only in BM25 (#5)
  doc_G: 0.0154   ← only in semantic (#5)

Note:

• Docs that appear in both rankings (A, B, C) dominate the top.
• Docs that appear in only one (D, E, F, G) fall behind.
• Among the ones that appear in only one, a high rank matters: D (#4 in BM25) beats G (#5 in semantic).

The complete hybrid pipeline with RRF

# hybrid_search.py
import chromadb
from chromadb.utils import embedding_functions
from rank_bm25 import BM25Okapi
import os


# Setup
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key=os.getenv("OPENAI_API_KEY"),
    model_name="text-embedding-3-small",
)
client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_or_create_collection("docs", embedding_function=openai_ef)


# Load every doc and build the BM25 index (once, at startup)
all_docs = collection.get()
all_doc_ids = all_docs['ids']
all_doc_texts = all_docs['documents']

tokenized_corpus = [doc.lower().split() for doc in all_doc_texts]
bm25_index = BM25Okapi(tokenized_corpus)


def hybrid_search(query: str, top_k: int = 5, n_per_method: int = 30):
    """
    The hybrid pipeline: BM25 + semantic + RRF.

    Args:
        query: the user's query
        top_k: the number of final results
        n_per_method: how many candidates each method retrieves

    Returns:
        A list of doc_ids sorted by hybrid relevance.
    """
    # 1. Semantic search
    semantic_results = collection.query(query_texts=[query], n_results=n_per_method)
    semantic_ranking = semantic_results['ids'][0]

    # 2. BM25 search
    query_tokens = query.lower().split()
    bm25_scores = bm25_index.get_scores(query_tokens)
    bm25_top_indices = sorted(range(len(bm25_scores)), key=lambda i: -bm25_scores[i])[:n_per_method]
    bm25_ranking = [all_doc_ids[i] for i in bm25_top_indices]

    # 3. RRF
    fused = reciprocal_rank_fusion([semantic_ranking, bm25_ranking])
    final_ids = [doc_id for doc_id, score in fused[:top_k]]

    # 4. Retrieve the content
    return collection.get(ids=final_ids)


# Try it
results = hybrid_search("OAuth2PasswordBearer scopes", top_k=5)
print(f"Hybrid top 5:")
for i, (doc_id, doc) in enumerate(zip(results['ids'], results['documents']), 1):
    print(f"\n#{i} [{doc_id}]")
    print(f"   {doc[:120]}...")

Tuning the k parameter

k=60 is the reasonable default, but some cases justify adjusting it:

How to find the optimal k empirically:

def find_optimal_k(eval_set, k_values=[10, 30, 60, 100, 150]):
    """Finds the k that maximizes recall over the eval set."""
    best_k = 60
    best_recall = 0

    for k in k_values:
        recalls = []
        for item in eval_set:
            # Run hybrid with this k
            semantic_ids = get_semantic_ranking(item.query, top_k=30)
            bm25_ids = get_bm25_ranking(item.query, top_k=30)
            fused = reciprocal_rank_fusion([semantic_ids, bm25_ids], k=k)
            top_5_ids = set(doc_id for doc_id, _ in fused[:5])

            relevant_in_top_5 = top_5_ids & set(item.expected_doc_ids)
            recalls.append(len(relevant_in_top_5) / len(item.expected_doc_ids))

        avg_recall = sum(recalls) / len(recalls)
        print(f"k={k}: recall={avg_recall:.2%}")
        if avg_recall > best_recall:
            best_recall = avg_recall
            best_k = k

    return best_k


optimal_k = find_optimal_k(eval_set)
print(f"\nOptimal k: {optimal_k}")

In most cases, the optimal k is 30-80. If your eval set suggests k <20 or k >150, there's probably another problem (poor source rankings).

Combining more than two rankings

RRF scales trivially to N rankings. You just add more entries to the loop:

# Hybrid with 3 signals: semantic + BM25 + cross-encoder reranking
semantic_ranking = collection.query(...)['ids'][0]
bm25_ranking = bm25_search(...)
cross_encoder_ranking = cross_encoder_rerank(...)  # top-N only

fused = reciprocal_rank_fusion(
    [semantic_ranking, bm25_ranking, cross_encoder_ranking],
    k=60,
)

The typical "state of the art" pattern in production RAG:

Query
  ↓
  ├─→ Semantic search (top-30)        ──┐
  ├─→ BM25 search (top-30)             ──┼─→ RRF ─→ top-10 ─→ Cross-encoder rerank ─→ top-5
  └─→ HyDE search (top-30, optional)   ──┘

Traps and common mistakes

Trap 1: unbalanced ranking sizes

The mistake: semantic returns the top-50, BM25 returns the top-5.

The symptom: docs that appear in BM25 contribute far less to the RRF score than docs only in semantic, simply because BM25 has fewer positions.

How to prevent it: rankings of equal size. A consistent n_per_method across sources.

Trap 2: running the BM25 ranking ONLY over the docs already retrieved by semantic

The mistake: first semantic search top-30, then BM25 reranking over those 30.

The symptom: BM25 is already limited to what semantic found. It loses its advantage of discovering relevant docs that semantic never found.

How to prevent it: BM25 must run over the full corpus, not over semantic's subset. Each method searches independently, then you fuse.

Trap 3: the score's sign (cosine is "lower = better")

The mistake: ChromaDB returns cosine distance (lower = more similar). You pass that straight through as a ranking — but inverted.

The symptom: RRF produces strange rankings because the order is reversed in one of the sources.

How to prevent it: always verify that your rankings are sorted with rank 1 = most relevant. For ChromaDB:

# The results already come sorted by relevance (lowest distance first)
# The first ID is rank 1 (most relevant)
semantic_ranking = results['ids'][0]  # OK

Trap 4: forgetting to deduplicate before RRF

The mistake: two different chunks of the same document show up in the ranking. RRF ranks them as two different docs.

The symptom: the top-5 contains 2-3 chunks of the same doc, and you lose diversity.

How to prevent it: deduplicate by doc_id before the fusion, or add logic that filters afterwards:

def dedupe_by_parent_doc(fused_ids: list[str]) -> list[str]:
    """Keeps only the first chunk of each parent doc."""
    seen_parents = set()
    deduped = []
    for chunk_id in fused_ids:
        parent = chunk_id.split("_")[0]  # assumes the format parent_chunk_N
        if parent not in seen_parents:
            seen_parents.add(parent)
            deduped.append(chunk_id)
    return deduped

Trap 5: RRF with k=0

The mistake: some tutorial suggests k=0 and you take it at face value.

The symptom: division by zero, or by a very small number. The scores explode.

How to prevent it: k must always be >= 1. The default of 60 is safe and sensible.

Trap 6: comparing RRF against weighted fusion without measuring

The mistake: you assume RRF always beats weighted fusion (the next capsule).

Reality: weighted fusion wins when one of the signals is clearly better than the other for your domain (e.g. BM25 far better than semantic, or the other way around). RRF assumes "both are equally trustworthy".

How to prevent it: measure it empirically over the eval set. If one signal is 20%+ better than the other, weighted fusion with an adaptive weight can improve the final ranking.

Applied exercise

Scenario: you're an AI Engineer at an e-commerce company. Your RAG system answers questions about products.

The data:

• 200K products with descriptions, specs, reviews
• Queries from the log:
  • 40% are exact SKUs and product codes: "NIKE-AM2024-RED-43"
  • 25% are partial names with identifiers: "Air Max 2024 size 43"
  • 25% are conceptual: "running shoes for marathons"
  • 10% are comparative: "Air Max vs Pegasus"

Current system: semantic search only. Recall@5 = 58%.

Your job:

1. Implement hybrid search with BM25 + RRF.
2. Decide the appropriate k and justify it.
3. Estimate the expected impact on recall.

# ecommerce_hybrid.py
from rank_bm25 import BM25Okapi
import re


def ecommerce_tokenizer(text: str) -> list[str]:
    """A tokenizer that preserves SKUs and product codes."""
    text_lower = text.lower()

    # Ordinary tokens
    tokens = re.findall(r'\b\w+\b', text_lower)

    # SKUs and codes (NIKE-AM2024-RED-43)
    sku_tokens = re.findall(r'[A-Z]+-[A-Z0-9-]+', text)  # preserve the uppercase
    tokens.extend([t.lower() for t in sku_tokens])

    # Product versions (AM2024)
    version_tokens = re.findall(r'[A-Z]{2,}\d+', text)
    tokens.extend([t.lower() for t in version_tokens])

    # Sizes (size 43, 43 EU)
    size_tokens = re.findall(r'\b\d{2,3}\b', text)
    tokens.extend(size_tokens)

    return tokens


def setup_hybrid(products: list[dict]):
    """Builds the BM25 index + assumes semantic is already indexed in ChromaDB."""
    bm25_corpus = []
    product_ids = []
    for p in products:
        # Concatenate SKU + name + description
        text = f"{p['sku']} {p['name']} {p['description']}"
        tokenized = ecommerce_tokenizer(text)
        bm25_corpus.append(tokenized)
        product_ids.append(p['sku'])

    bm25_index = BM25Okapi(bm25_corpus)
    return bm25_index, product_ids


def hybrid_search(query: str, bm25_index, product_ids, semantic_collection, top_k=5):
    # Semantic
    sem_results = semantic_collection.query(query_texts=[query], n_results=30)
    semantic_ranking = sem_results['ids'][0]

    # BM25
    query_tokens = ecommerce_tokenizer(query)
    bm25_scores = bm25_index.get_scores(query_tokens)
    bm25_top = sorted(range(len(bm25_scores)), key=lambda i: -bm25_scores[i])[:30]
    bm25_ranking = [product_ids[i] for i in bm25_top]

    # RRF
    fused = reciprocal_rank_fusion([semantic_ranking, bm25_ranking], k=30)
    return [doc_id for doc_id, _ in fused[:top_k]]

for k in [10, 30, 60, 100]:
    recall = evaluate_hybrid_with_k(eval_set, k=k)
    print(f"k={k}: recall@5={recall:.2%}")

# Expected output:
# k=10: recall@5=82%
# k=30: recall@5=85%   ← the best
# k=60: recall@5=82%
# k=100: recall@5=78%

Category                     %       Current recall   Hybrid recall
────────────────────────────────────────────────────────────────────
Exact SKUs                  40%      35%              92% (BM25 nails it)
Partial names               25%      62%              82%
Conceptual                  25%      78%              80% (semantic already wins)
Comparative                 10%      55%              78%

Weighted global recall:
  Current: 0.40(0.35) + 0.25(0.62) + 0.25(0.78) + 0.10(0.55)
         = 0.140 + 0.155 + 0.195 + 0.055 = 0.545 ≈ 54%

  Hybrid: 0.40(0.92) + 0.25(0.82) + 0.25(0.80) + 0.10(0.78)
        = 0.368 + 0.205 + 0.200 + 0.078 = 0.851 ≈ 85%

Gain: +31 points of global recall

Summary and next step

What you learned:

• RRF combines rankings from multiple sources while ignoring absolute scores. Only the position matters.
• The formula: RRF_score(doc) = Σ 1/(k + rank_i(doc)) for each ranking i.
• k=60 is the reasonable default. Tune it to 20-100 depending on the domain.
• A small k amplifies the gap between high and low rankings. Useful when one signal should dominate.
• A trivial implementation: ~10 lines. Robust to any score difference between sources.
• It scales to N rankings: you add more entries to the loop, no rewrite required.
• The typical pipeline: semantic + BM25 → RRF → cross-encoder rerank → top-K.
• The traps: unbalanced rankings, the score's sign, forgotten deduplication.

Checkpoint: before moving on, you should be able to:

• Implement RRF in under 15 lines with correct typing.
• Justify your choice of k for your case with eval set data.
• Design a hybrid pipeline with RRF + dedupe + rerank in a cascade.

Next capsule: 05 — Weighted hybrid blending.

RRF is robust, but it assumes every source is equally trustworthy. Capsule 05 covers weighted fusion: when you know BM25 is better for specific queries (the ones with identifiers) and semantic is better for conceptual ones, you can weight each signal according to the query type. More tuning, more complexity, but sometimes the marginal gain is worth it.

Resources

1. RRF Paper (Cormack, Clarke, Buettcher 2009) — The original paper with empirical analysis
2. Pinecone — Hybrid Search with RRF — A visual tutorial
3. Elasticsearch — RRF Implementation — The native version in ES
4. LangChain — EnsembleRetriever — An implementation with RRF
5. LlamaIndex — QueryFusionRetriever — The pattern in LlamaIndex
6. BEIR Benchmark — Empirical comparison of RRF vs the alternatives

Estimated time: 25-30 minutes Next: 05-weighted-hybrid-blending.md