THE LADDER

200 saved questions; 184 answerable. Processing, rewriting, retries and generation off. Overall Hit@5 differences are not statistically distinguishable in this dataset.

Naive

Page text, overlapping chunks and dense retrieval. A baseline you can follow.

Single-turn; previous questions are not passed to the model.

Explore Naive

THE SAVED MEASUREMENTS

The saved measurements.

Frozen runtime processing-off retrieval. Quality averages exclude the 16 unanswerable questions.
MetricNaiveIntermediateAdvanced
hit@146.2%45.1%59.8%
hit@368.5%71.2%77.7%
hit@577.7%79.3%82.6%
hit@1088.0%86.4%Not measured
mrr@50.72060.75380.8656
recall@583.8%88.4%91.9%
location_hit@579.9%87.0%90.8%
Evidence coverage@583.1%85.4%88.8%

Hit@5 requires the evaluator’s complete evidence rule. Recall@5 is keyword coverage; evidence coverage credits complete required entries. MRR@5 is the saved evaluator’s reciprocal-rank metric. Advanced Hit@10 is not measured because the returned output is capped at five parents.

TRADE-OFFS YOU CAN INSPECT

Trade-offs you can inspect.

StageNaiveIntermediateAdvanced
InputText PDFsPDFs with text recognitionPDFs with structured parsing
PassagesOverlapping windowsHeading-aware passagesSmaller passages with surrounding context
SearchMeaning-based searchMeaning and keywordsComplementary search and reranking
ConversationIndependent questionsRecent conversationNo hosted conversation
AvailabilityHosted sessionHosted sessionPrivate local setup