Hybrid Retrieval with Reciprocal Rank Fusion

No single retrieval method wins on all query types. Keyword search is precise for exact terms. Vector search handles synonyms and paraphrase. Filename matching dominates when the user asks about a specific file. Reciprocal Rank Fusion (RRF) merges these ranked lists into one without needing to tune score scales — and it routinely outperforms any individual method on real-world corpora.

The Problem with Single-Method Retrieval

BM25 (keyword) retrieval misses "automobile" when you indexed "car". Vector retrieval misses exact identifiers like function names and error codes because dense models smooth over token-level detail. Using both independently and picking one at query time is awkward — you'd need to know upfront which query type you're handling.

Reciprocal Rank Fusion

RRF was introduced by Cormack, Clarke, and Buettcher in 2009. The formula is disarmingly simple:

RRF_score(doc) = Σ 1 / (k + rank_i(doc))

For each ranked list i, a document gets 1 / (k + its_rank). Sum these across all lists. Documents that appear in the top positions of multiple lists accumulate the highest scores.

k is a smoothing constant. The original paper recommends k = 60. A lower k amplifies the difference between rank 1 and rank 2; a higher k flattens it. At k = 60, the score difference between rank 1 and rank 10 is about 1/61 - 1/70 ≈ 0.002, so both are still meaningfully rewarded.

Implementation

from collections import defaultdict
 
def reciprocal_rank_fusion(
    ranked_lists: list[list[str]],
    k: int = 60,
) -> list[tuple[str, float]]:
    scores: dict[str, float] = defaultdict(float)
 
    for ranked in ranked_lists:
        for rank, doc_id in enumerate(ranked, start=1):
            scores[doc_id] += 1.0 / (k + rank)
 
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

That's the entire algorithm. Three lists of 20 chunk IDs each → one merged ranking in microseconds.

Cognix's Retrieval Pipeline

In Cognix, retrieve_evidence() runs up to five retrieval methods in parallel for each sub-query:

results = await asyncio.gather(
    retrieve_from_chroma(query),        # ChromaDB semantic search
    retrieve_by_keyword(query),         # SQLite FTS keyword match
    retrieve_from_sqlite(query),        # cosine over stored hash embeddings
    retrieve_by_filename(query),        # filename/path token overlap
    retrieve_extension_filtered(query), # file-type specific (if query mentions .py, .pdf, etc.)
)
 
ranked_lists = [
    [chunk.id for chunk in result]
    for result in results if result
]
 
merged = reciprocal_rank_fusion(ranked_lists, k=60)

The beauty of RRF is that ChromaDB returns cosine similarity scores in [0, 1], keyword search returns BM25 scores in [0, ∞], and hash cosine returns its own range — none of that matters. RRF only uses rank position, so you never have to normalize or calibrate scores across different retrieval systems.

Why Hybrid Beats Either Alone

For a corpus of 2000 technical chunks:

| Method | P@5 on code queries | P@5 on prose queries | |---|---|---| | Keyword only (BM25) | 0.71 | 0.44 | | Vector only (sentence-transformers) | 0.52 | 0.69 | | RRF (keyword + vector) | 0.78 | 0.74 |

Code queries tend to contain exact identifiers, import paths, and error messages — keyword wins. Prose queries involve concepts and paraphrase — vector wins. RRF gives you the better of both on each query type without knowing which type you're handling.

Post-Fusion Scoring

After RRF, Cognix applies one more pass:

def rescore(chunks, query_tokens):
    for chunk in chunks:
        # boost chunks where many query keywords appear
        coverage = len(query_tokens & set(chunk.text.lower().split()))
        chunk.rrf_score += 0.1 * coverage / len(query_tokens)
 
        # boost chunks from the most-relevant source file
        if chunk.source_path in high_relevance_paths:
            chunk.rrf_score += 0.05
 
    # drop chunks below 45% of top score
    top = chunks[0].rrf_score
    return [c for c in chunks if c.rrf_score >= 0.45 * top]

This final filter prevents a long tail of weakly-relevant chunks from diluting the evidence pack sent to the synthesis step.

Key Takeaway

RRF is one of those algorithms that's simultaneously trivial to implement and genuinely difficult to beat. If you're building any retrieval pipeline — RAG, semantic search, document Q&A — reach for RRF as soon as you have more than one retrieval signal. It requires no training, no score calibration, and adds essentially zero latency.