Hybrid Retrieval
Most production RAG systems use hybrid retrieval — combining vector (semantic) search with keyword (BM25) search and merging the results. This is not a complex choice: it is the default architecture for any RAG system that serves a real user population.
Why hybrid is necessary
Vector search fails on exact entities. A user asking about "GPT-4o-mini rate limits" or "ORD-58291 status" expects the document that contains those exact strings. Embedding similarity does not optimize for exact match — it finds semantically similar text. A document that contains "GPT-4o" and discusses rate limits will score highly even if it never mentions "GPT-4o-mini."
Keyword search fails on paraphrases. A user asking "how do I undo a payment" will not match a document titled "reversing a transaction" unless the document also contains the word "undo." Embedding similarity handles synonyms and paraphrases naturally; BM25 does not.
Combined, they cover each other's failure modes. For any realistic query distribution — factual lookups, explanatory questions, entity-specific queries, paraphrase-heavy natural language — hybrid retrieval consistently outperforms either method alone.
Reciprocal Rank Fusion
The standard merging strategy is Reciprocal Rank Fusion (RRF). Each method independently ranks candidates. For each document, the RRF score is the sum of 1/(k + rank) across all methods (k=60 is the standard default).
RRF has one important property: it uses only rank positions, not absolute scores. This means you do not need to normalize cosine similarity scores (0 to 1) against BM25 scores (unbounded). Each method just needs to rank its candidates. This makes RRF robust to differences in score distribution between methods.
When to weight differently
The default k=60 treats both methods roughly equally. Adjust when:
- Your corpus is heavy on exact entity queries (product names, IDs, error codes): lower k to reward top-ranked keyword results more aggressively.
- Your users ask mostly natural language questions: lower k to reward top-ranked vector results more.
- You have a third retrieval method (e.g., a graph-based retriever for structured data): add it to the RRF sum with the same formula.
Common mistakes
Only using vector search because it is easier to set up. This consistently degrades quality for entity-specific queries. Add BM25 before your first production launch, not after users complain.
Treating both methods as interchangeable. Vector search and keyword search fail on different query types. When you see a quality regression, always check which method was responsible for the failure before tuning.
Not logging which method contributed each result. When debugging retrieval failures, you need to know whether the failing result came from the vector search, the keyword search, or only appeared after merging. Log source_methods per result.
Practical takeaway
Start with BM25 + vector search merged with RRF k=60. Run your Recall@K eval harness to establish a baseline. Then experiment with k values and method weights if you have a labeled test set large enough to detect differences. Most systems do not need to go beyond the default.