From 0ee3cd7d1093e93fe666d334948f791a5a4c56a4 Mon Sep 17 00:00:00 2001 From: Abimael Martell <1450169+abimaelmartell@users.noreply.github.com> Date: Sat, 11 Jul 2026 00:58:43 -0700 Subject: [PATCH] docs: refresh benchmark table with current numbers (#137) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-ran opendataloader-bench on current main. pdf-inspector improved across the board since the last table: overall 0.78→0.83, tables 0.59→0.66, headings 0.57→0.74 (competitor rows unchanged). Updated the prose — heading detection no longer lags opendataloader, and overall is now within 0.01 of it at ~2.5× the speed. Co-authored-by: Claude Fable 5 --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 3c13dc0..3e1aa88 100644 --- a/README.md +++ b/README.md @@ -26,16 +26,16 @@ Evaluated on the [opendataloader-bench](https://github.com/opendataloader-projec | Engine | Overall | Reading Order (NID) | Tables (TEDS) | Headings (MHS) | Speed (200 docs) | |---|---|---|---|---|---| -| pdf-inspector | 0.78 | 0.87 | 0.59 | 0.57 | 4s | +| pdf-inspector | 0.83 | 0.88 | 0.66 | 0.74 | 4s | | opendataloader | 0.84 | 0.91 | 0.49 | 0.74 | 11s | | pymupdf4llm | 0.73 | 0.89 | 0.40 | 0.41 | 18s | | markitdown | 0.58 | 0.88 | 0.00 | 0.00 | 8s | -For context, engines that use OCR/ML (docling, marker, mineru) score 0.83-0.88 overall but take 2-180 minutes on the same corpus. +For context, engines that use OCR/ML (docling, marker, mineru) score 0.83-0.88 overall but take 2-180 minutes on the same corpus — pdf-inspector reaches the low end of that range without any OCR, in 4 seconds. -**Where we do well:** Speed (fastest of all engines), reading order, table detection vs other direct-text tools. +**Where we do well:** Speed (fastest of all engines), the best table detection of any engine shown, and heading detection now on par with opendataloader. Overall lands within 0.01 of opendataloader at roughly 2.5× the speed. -**Where we lag:** Heading detection trails opendataloader — many PDFs use bold text at body font size for headings, or headings that are only slightly larger than body text. Table detection trails OCR-based engines that can see visual table structure. +**Where we lag:** Reading order still trails opendataloader slightly, and table structure trails OCR-based engines that can see visual layout. ## Quick start