Cohere Rerank 5 — Context Focus calibrated threshold — What does it mean?

The news. On October 21, 2026, Cohere released Rerank 5 in Pro and Fast tiers. Both add Context Focus, and the release describes calibrated relevance scores. Cohere reports that Pro ranked first on 39 of 46 composite datasets, and that Fast reranks about 187 documents of 1,000 tokens per second on one NVIDIA A100. Read the release →

Think of an assistant sent to the library for your question. The old way is to say "bring me the five best books" and carry all five to your desk, even if the fifth is barely related. A top-k cutoff fixes the count of books and leaves their quality to chance. If only two books are really useful, you still pay to read three bad ones, and an agent that searches again and again repeats that cost on every search.

The new way gives every book a sticky note with a grade. Rerank 5 makes that grade honest: the returned relevance score is calibrated, so 0.7 means roughly a 70% chance the document is useful. Cohere reports that earlier rerankers gave scores that were mostly good for sorting only, so teams had to tune a cutoff for each model and tune it again on every upgrade. Cohere says calibrated scores let teams set one threshold across queries and domains, and that Pro and Fast agree that "more likely useful than not" is about 0.50 for Pro and 0.45 for Fast. In the retrieve-then-generate pipeline this is the step between search and the model, and it is where context stays scarce.

Then comes the highlighter. Often only a few sentences inside a long document answer the question, so sending the whole page still wastes tokens and adds noise. With Context Focus, a single Rerank call returns the document-level relevance score and scored text spans, and your application passes on only the spans above the threshold. It adds no extra step to the pipeline. This is a different lever from chunking: chunking decides how the corpus is cut before indexing, while Context Focus decides how much of a retrieved document to keep at query time.

The table is Cohere's own test: an agent with search as its only tool, 5 results per search, averaged over 17 datasets. Read it as vendor-reported, not independently reproduced. The useful part is the shape: raising the threshold from 0.1 to 0.4 cut more tokens and accuracy did not fall, it rose from 57% to 58%. Less text did not mean less evidence, because the removed text was mostly noise. Cohere also describes the setting as a dial: across 30 question-answering datasets, removing half the retrieved text kept 96% of answer quality, and removing three quarters kept 87%.

Here is a worked example with numbers fixed (illustrative, not from the source). An agent runs 12 searches per task, and each search returns 5 documents of 1,000 tokens, so the model reads 5 x 1,000 = 5,000 tokens per search and 12 x 5,000 = 60,000 tokens per task. Suppose the threshold keeps only 40% of each document's text. Then it reads 2,000 tokens per search and 12 x 2,000 = 24,000 tokens per task. That is 36,000 fewer tokens per task, a 60% cut. Cohere's measured figure is smaller (35% fewer tokens than BM25 alone), and it is a measured average over real tasks, not a calculation like this one.

One limit to keep in mind: the post shows calibration evidence for the document-level relevance score and does not report a matching check for span scores, so treat a span threshold as a setting to tune on your own data. The architecture of the model is also not described in the post, and the benchmark and throughput numbers are the vendor's own.

Goes deeper in: AI Agents → Retrieval & RAG → Retrieve-Then-Generate