Search Engine Latency Optimization: Block-WAND, MaxScore & Tiered Caching

Evaluating thousands of candidate documents for a multi-term query can exceed interactive latency budgets. Modern search engines rely on early termination algorithms to evaluate only top-scoring documents.

1. Block-WAND (Weak AND) Dynamic Pruning

Block-WAND precomputes the maximum score achievable within contiguous blocks of posting lists. During query evaluation, if the upper bound score of a posting block plus other query terms cannot exceed the current top-K result threshold, the entire posting block is skipped without decompressing individual DocIDs.

2. MaxScore Evaluation

MaxScore partitions query terms into essential and non-essential lists based on maximum possible score contributions, evaluating low-IDF terms only when documents already satisfy minimum threshold criteria.

3. Tiered Caching Architecture

Enterprise search architectures utilize three-layer cache hierarchies: Result Cache (caching fully rendered JSON responses for frequent queries), Posting Cache (caching uncompressed posting lists in RAM), and Filter Cache (caching bitsets for facet filters).