Tail-Aware Adaptive-k makes RAG choose its context size

Most RAG systems hard-code top-k: send the same number of passages to the generator for every query. Tail-Aware Adaptive-k, or TAA-k, treats the ranked similarity curve as a per-query signal. It first finds the curve's knee, then tests a small neighborhood for a stable noise tail and returns the earliest cutoff that passes. The method is training-free. 1

A fixed cutoff can waste context on low-relevance passages or remove evidence that matters to a multi-hop answer. TAA-k is designed to favor recall near the relevance-to-noise boundary instead of trusting an absolute score gap. In the paper's 64-dimensional Bailian embedding setup, it reports 94.25% recall and 65.81 F1 on 2WikiMultiHopQA. 1

Put the selector between retrieval and generation. Run it beside your current fixed-k path on the same candidate pools, then log k_hat, recall, answer accuracy, token count, and p95 latency. The paper reports 4.03 ms with knee localization versus 40.59 ms for its exhaustive variant, plus 61.63% average answer accuracy across WebQuestions, 2WikiMultiHopQA, and MuSiQue. Those are paper results, not a deployment guarantee. 1

Keep a rollback path. The authors note that score overlap, very small candidate pools, and finite-sample effects can blur the relevance-to-noise boundary. In practice, the useful question is not "Can adaptive k save tokens?" It is "Which queries need more context, and does that extra context improve the answer?" 1