A one-line-per-article map turns LLM query rewriting into routing: 85.2% passage recall on paraphrased questions, reading 2,000 tokens of a 3-million-token book per query.
Abukar A. Abukar·November 15, 2026·8 min read
Abstract
Sending a whole book with every question is costly and capped by the context window. Thumbindex (working name) keeps documents as written, indexes them with cheap content-derived addresses, and re-reads the source before returning evidence. On a 3,256-page encyclopedia (≈3M tokens) it places the gold passage inside a 2,000-token budget for 77.9% of 308 test questions (BM25: 48.1%). The main result: giving the query-rewriting model a map of the corpus, one generated line per article, adds 14.8 points on paraphrased questions with no vectors (95% CI 10.0 to 19.3) and 5.2 points on top of hybrid retrieval with reranking (CI 1.7 to 8.6), reaching 85.2%. Content-hashed spans keep every pointer valid after an edit.
Pointers valid after a paragraph insert (207 of 207).
1. System
Documents are the source of truth. Retrieval runs over small derived structures; every candidate is re-read from the store and hash-checked before it is returned, so a stale index entry can cost recall but never return text that is not in the source.
Blocks. Paragraphs named by content hash. A version is the ordered list of its block hashes.
Spans. ~200-word runs of blocks with content-defined boundaries; an edit moves boundaries only near itself.
Addresses. Lexical fields, heading path, position, source and named entities per span, scored by BM25 and floor-bounded soft cue scorers.
Map. One model-written line per article, built at ingest and placed in the rewrite prompt, where it is cacheable.
Levels. L0: BM25 + soft cues, standard library only. L1: + an LLM rewrite fused with the original query. L2: + a local dense embedder and cross-encoder reranker.
Figure 1. Architecture. Each stored structure (dark) feeds the query step directly below it. TESTED: measured in experiments, not yet in the library. NEXT: planned. Full diagram (PNG).
How a question is answered
Figure 1. The query path and the stored structure each step reads. Full diagram (PNG).
2. Setup
3,256
PDF pages, two columns per page.
≈3M
Tokens (2.0M words) in one technical encyclopedia.
555
Articles, one document each.
13,756
Spans of about 200 words.
308 + 420
Test questions: whole book, and paraphrased over a 185-article core.
2,000
Tokens read per question, at most.
Primary metric
Evidence hit rate @2k: did the system actually read the passage that supports the answer, within 2,000 tokens? Answer fluency is not scored.
Pre-registered contrasts and minimum effects, paired bootstrap 95% CIs, Holm correction. Test questions never reach an indexing step.
Models
DeepSeek chat API for rewrites and map lines. Local embedder and cross-encoder reranker at level 2.
3. Results
3.1 Paraphrased questions are where retrieval breaks
Every method solves lexical-overlap questions. Paraphrases are the gap: BM25 finds 13.0% of them, and hybrid retrieval with a cross-encoder reranker still misses 39%.
Evidence hit rate @2k by question type
308 test questions over the whole book (≈3M tokens).
Best configuration: hybrid + cues (RRF)Other configurations
All questions308 questions
BM25
48.1%
BM25 + LLM rewrite
67.9%
Hybrid + reranker
76.6%
Hybrid + LLM rewrite
75.0%
Hybrid + cues (RRF)
80.8%
Word overlap57 questions
BM25
98.2%
BM25 + LLM rewrite
94.7%
Hybrid + reranker
96.5%
Hybrid + LLM rewrite
86.0%
Hybrid + cues (RRF)
98.2%
Paraphrased92 questions
BM25
13.0%
BM25 + LLM rewrite
32.6%
Hybrid + reranker
60.9%
Hybrid + LLM rewrite
52.2%
Hybrid + cues (RRF)
62.0%
Vague159 questions
BM25
50.3%
BM25 + LLM rewrite
78.6%
Hybrid + reranker
78.6%
Hybrid + LLM rewrite
84.3%
Hybrid + cues (RRF)
85.5%
Figure 2. Hybrid: BM25 and dense retrieval fused, cross-encoder reranking, heading-path prefix. Rewrite: the LLM's rewrite replaces the query. Cues (RRF): cue rankings merged by reciprocal rank fusion.
Method
All
Word overlap
Paraphrased
Vague
BM25
48.1
98.2
13.0
50.3
BM25 + LLM rewrite
67.9
94.7
32.6
78.6
Hybrid + reranker
76.6
96.5
60.9
78.6
Hybrid + LLM rewrite
75.0
86.0
52.2
84.3
Hybrid + cues (RRF)
80.8
98.2
62.0
85.5
Table 1. Evidence hit rate @2k (%), whole book. Best per column in bold.
Why the remaining misses happened
50 failures from the dev split, read and sorted by cause.
Paraphrased with no cueOther causes
Paraphrased, no cue
27
Paraphrased, no cue, loosely worded
10
Built-in wrong cue
5
Only position or topic cues
4
Cue corrupted by noise
3
Word-overlap question
1
Figure 3. 37 of 50 failures (74%) were paraphrased questions with no cue to go on. That is the case the map targets.
3.2 A one-line map turns rewriting into routing
A plain LLM rewrite lifts paraphrase recall only from 41.9% to 51.0%, because the model knows neither the corpus vocabulary nor where a topic lives. With the map in its prompt, the rewrite names the right article and uses its terms.
Evidence hit rate @2k, paraphrased questions
420 test questions, 185-article core (≈1M tokens).
Map-conditioned rewriteWithout the map
BM25
41.9%
BM25 + LLM rewrite
51.0%
BM25 + hypothetical passage
59.5%
BM25 + map-conditioned rewrite
65.7%
Dense retrieval alone
66.0%
Hybrid + reranker + rewrite
80.0%
Hybrid + reranker + map-conditioned rewrite
85.2%
Figure 4. The map adds 14.8 points to the plain rewrite (95% CI 10.0 to 19.3) and 5.2 points to the full stack (CI 1.7 to 8.6, Holm-corrected p = 0.015).
Vector-level recall without vectors. BM25 with a map-conditioned rewrite (65.7%) matches dense retrieval alone (66.0%).
Additive with vectors. On top of hybrid retrieval, reranking and a rewrite, the map adds 5.2 points, to 85.2%.
The mechanism is routing. All of the gain comes from the 250 of 420 questions where the rewrite names the right article.
Ahead of an agent. On 150 questions, an agent that searched with BM25 and read pages (1.6 tool calls per question) reached 54.0%. Level 2 reached 82.7%, and level 2 with the map 88.0%.
3.3 The map at full scale
Over all 555 articles the flat map is about 24,500 prompt tokens per query, nearly all served from the prompt cache.
Evidence hit rate @2k, paraphrased questions, whole book
555 articles (≈3M tokens). Cost as billed with prompt caching.
With a mapWithout
BM25
32.1%
LLM rewrite, no map
43.3%
Two-step map4,530 prompt tokens · $0.00041 · 2.8 s
53.8%
Flat map24,532 prompt tokens · $0.00034 · 1.9 s
58.6%
Show the numbers as a table
Method
Hit @2k
Prompt tokens
Billed
Uncached
Latency
BM25
32.1%
—
—
—
—
LLM rewrite, no map
43.3%
—
—
—
—
Two-step map
53.8%
4,530
$0.00041
$0.0014
2.8 s
Flat map
58.6%
24,532
$0.00034
$0.0074
1.9 s
Figure 5. The flat map adds 15.3 points over the plain rewrite. The two-step map misroutes 39% of questions and trails the flat map by 4.8 points (95% CI 0.7 to 9.0).
3.4 Three levels behind one interface
Measured through the library's own interface on the 308 whole-book questions, with names, authors and sections corrupted 30% of the time:
Evidence hit rate @2k by level and question type
Whole book (≈3M tokens).
Version 0 levelsBM25 baseline
All questions308 questions
BM25
48.1%
Level 0 · no model
53.6%
Level 1 · + rewrite
71.1%
Level 2 · + dense, reranker
77.9%
Word overlap57 questions
BM25
98.2%
Level 0 · no model
98.2%
Level 1 · + rewrite
98.2%
Level 2 · + dense, reranker
94.7%
Paraphrased92 questions
BM25
13.0%
Level 0 · no model
14.1%
Level 1 · + rewrite
31.5%
Level 2 · + dense, reranker
63.0%
Vague159 questions
BM25
50.3%
Level 0 · no model
60.4%
Level 1 · + rewrite
84.3%
Level 2 · + dense, reranker
80.5%
Figure 6. Level 2 lifts paraphrased questions from 13.0% to 63.0%. Level 0 needs nothing beyond the Python standard library.
Level
Adds
All
Latency
0
BM25 + soft cues, no model
53.6%
17 ms
1
Fused LLM rewrite
71.1%
23 ms + call
2
Dense model + reranker
77.9%
3.0 s
Table 2. Hit rate on all 308 questions and median latency per question.
3.5 Content-addressed spans survive edits
A pointer names what a passage says; its position is only a hint.
One paragraph inserted at the top of a page
Figure 7. Illustration with made-up hashes and offsets. Matching by paragraph hash keeps every old pointer; character ranges all shift.
Addresses still resolving to the same text after an insert
One paragraph added at the top of each page. War and Peace, 365 chapters as pages.
Used by the systemAlternatives
Character offsets
0%
Ordinal span number
53%
Span hash, greedy chunking
67%
Span hash, content-defined chunking
84%
Paragraph hash
100%
Figure 8. Ordinal pointers fail silently: a stale number resolves to different text with no error.
Real articles. 207 of 207 pointers resolved to the same text, with no model call.
Incremental re-indexing. 10 edited articles cost 80 model requests. All 4,778 entries on unchanged text were kept; stale ones were retired.
3.6 Wrong cues have a bounded cost
Cues (names, author, section, rough position) are scored softly with a floor: a wrong cue lowers a score but never excludes a passage.
Points gained by adding cues, as more of them are wrong
159 vague test questions, whole book. Bars are 95% intervals where computed.
Cue arm − baselineBoth sides with a fused rewrite
Cues corrupted
Cue arm − baseline
Both with fused rewrite
0% (exact)
+8.8 (4.4 to 13.8)
+6.3
15%
+8.2 (3.1 to 13.8)
+5.0 (0.6 to 9.4)
30% (pre-registered)
−0.6 (−5.7 to 4.4)
+0.6 (−4.4 to 5.7)
60%
0.0
−1.3
Figure 9 and Table 3. Gains hold up to 15% corruption and vanish by 30%. No setting falls measurably below its baseline.
Primary result
Cues pay off only when they are accurate, and soft scoring makes wrong ones cheap. The robust gain comes from routing, not from remembered details.
3.7 What did not help
Idea
Result
Surprise-weighted landmark index
50.2%, against 51.7% for random landmarks
Twelve zero-call index-key types, including the book's own back-of-book index
Best +2.6 over BM25; the pre-registered bar was +5
Replacing the query with its rewrite
−1.6 on the hybrid; fusing it with the original gives +3.6
Map-conditioned hypothetical passage
65.7%, no gain over the map-conditioned rewrite
Table 4. Ideas tested and closed for this corpus.
4. Scope
One corpus; questions generated by the same model family that writes the map. On the 205 of 420 paraphrased questions that pass a strict single-passage check, the vector-free map arm comes within 8.3 points of the hybrid (95% CI 2.4 to 14.1). Total model cost of all experiments: $8.15 at list prices.
Takeaway
On paraphrased questions, retrieval is largely a routing problem. A cacheable map with one line per article lets any chat model route before it searches: it matches a vector index on its own, stacks with one to 85.2%, and every answer is still read from the source.