Illustrative example
When the right page leads to a wrong answer
This fictional purchase order illustrates a failure category from the study.
| Item | Qty | Unit price | Line total |
|---|---|---|---|
| Notebook | 4 | $12.00 | $48.00 |
Ask for the unit price
The question asks for the price of one notebook.
What is the unit price of one notebook?
In the study’s paired sample of 1,000 questions, the model scored 0.3952 ANLS even when given the correct page. ANLS measures answer similarity on a scale from 0 to 1.
View the measured resultsStudy details
This study compares OCR-text, dense-text, visual, and reciprocal-rank-fusion retrieval under document-disjoint evaluation. It then measures how retrieved evidence affects a fixed vision-language answer generator, so retrieval failures are not confused with generation failures.