This preview includes your current edits. Unsaved changes are included in the download, too.
Retrieval or a longer context?
Research question
For question answering over our own documents, should we start with retrieval or a longer context window?
Evidence
Your notes and interpretations. Saving a paper does not verify a claim.
A retrieval baseline is worth testing
RAG combines a generator with retrieved documents and evaluates it on knowledge-intensive tasks. That motivates a baseline, not a guarantee for our application. Source: Lewis et al., 2020
DPR studies a dual-encoder approach to passage retrieval. It is a useful reference when designing our retrieval evaluation. Source: Karpukhin et al., 2020
More context is not the whole answer
Lost in the Middle finds sensitivity to where relevant information appears in the tasks it evaluates. This does not establish the behavior of every current model. Source: Liu et al., 2023
Decisions
Proposed experiment — not yet run
- Compare a retrieval baseline and a long-context baseline on the same questions and documents.
- Record source support, latency, and cost separately.
- Choose an approach only after reviewing those results.
This is an illustrative research plan, not a finding or recommendation from the paper authors.
Open questions
- Which question types fail on our own documents?
- Does document position affect the model we actually plan to use?
- How will we assess whether a citation supports the answer?
- What latency and per-question cost can we accept?