Hierarchical Retrieval

A mainstream paradigm for large-scale vector retrieval. In the first stage, lightweight methods such as a hybrid of BM25 inverted indexes and ANN approximate nearest neighbor search quickly filter tens to hundreds of candidates from the full corpus, trading precision for speed. In the second stage, a heavy model such as a cross-encoder rescores and reranks the candidate set. By restricting expensive precise computation to the candidate set, recall is preserved while P95/P99 latency is kept within an interactively acceptable range.