mu-ref/pdf-inspector
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(benchmark): refresh parser comparison (#178)

Browse Source 
* docs: refresh benchmark comparison

* docs(site): refresh benchmark section

* docs: reframe benchmark positioning

* docs: focus benchmark positioning on best fit
This commit is contained in:
Abimael Martell
2026-07-16 17:43:53 -07:00
committed by GitHub
parent 15c0b22093
commit a910b7df1d
6 changed files with 47 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
site/index.html
+9 -10
View File
@@ -147,8 +147,7 @@
tbody tr.us td:first-child::before { content: "▸ "; color: var(--accent); }
.bench-foot { padding: 15px 18px; font-size: 13.5px; color: var(--ink-soft); background: var(--paper-2); border-top: 1px solid var(--line); }
.bench-wrap { overflow-x: auto; }
.callouts { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; margin-top: 22px; }
@media (max-width: 640px) { .callouts { grid-template-columns: 1fr; } }
.callouts { display: grid; grid-template-columns: 1fr; gap: 16px; margin-top: 22px; }
.callout { border-left: 3px solid var(--accent); padding: 4px 0 4px 16px; }
.callout .ct { font-family: var(--mono); font-size: 11px; letter-spacing: 0.1em; text-transform: uppercase; color: var(--accent-deep); margin-bottom: 5px; }
.callout p { font-size: 14.5px; color: var(--ink-soft); }
@@ -301,7 +300,7 @@
<div class="sec-head">
<span class="sec-num">02</span>
<h2 class="sec-title">Fastest of the direct-text engines</h2>
<p class="sec-sub">Evaluated on the <a href="https://github.com/opendataloader-project/opendataloader-bench" style="color:var(--accent-deep);border-bottom:1px solid var(--line)">opendataloader-bench</a> corpus (200 PDFs). Direct text-extraction engines only — no OCR, no ML. Higher is better.</p>
<p class="sec-sub">Evaluated on the <a href="https://github.com/opendataloader-project/opendataloader-bench" style="color:var(--accent-deep);border-bottom:1px solid var(--line)">opendataloader-bench</a> corpus (200 PDFs). Local engines without model-based PDF parsing; OCR disabled. Higher is better.</p>
</div>
<div class="bench">
<div class="bench-wrap">
@@ -310,18 +309,18 @@
<tr><th>Engine</th><th>Overall</th><th>Reading order</th><th>Tables</th><th>Headings</th><th>200 docs</th></tr>
</thead>
<tbody>
<tr class="us"><td>pdf-inspector</td><td>0.83</td><td>0.89</td><td>0.66</td><td>0.74</td><td>4s</td></tr>
<tr><td>opendataloader</td><td>0.84</td><td>0.91</td><td>0.49</td><td>0.74</td><td>11s</td></tr>
<tr><td>pymupdf4llm</td><td>0.73</td><td>0.89</td><td>0.40</td><td>0.41</td><td>18s</td></tr>
<tr><td>markitdown</td><td>0.58</td><td>0.88</td><td>0.00</td><td>0.00</td><td>8s</td></tr>
<tr class="us"><td>pdf-inspector</td><td>0.875</td><td>0.915</td><td>0.814</td><td>0.788</td><td>2.8s</td></tr>
<tr><td>liteparse</td><td>0.870</td><td>0.908</td><td>0.693</td><td>0.811</td><td>13.9s</td></tr>
<tr><td>opendataloader</td><td>0.843</td><td>0.912</td><td>0.489</td><td>0.760</td><td>9.8s</td></tr>
<tr><td>pymupdf4llm</td><td>0.735</td><td>0.886</td><td>0.401</td><td>0.424</td><td>15.5s</td></tr>
<tr><td>markitdown</td><td>0.583</td><td>0.879</td><td>0.000</td><td>0.000</td><td>6.7s</td></tr>
</tbody>
</table>
</div>
<div class="bench-foot">OCR/ML engines (docling, marker, mineru) score 0.83–0.88 overall — but take 2–180 minutes on the same corpus.</div>
<div class="bench-foot">Refreshed July 16, 2026, on Apple M4 Pro. Speed is the median of three complete corpus runs.</div>
</div>
<div class="callouts">
<div class="callout"><div class="ct">Where we win</div><p>Fastest engine measured, the best table detection of any engine here, and heading quality now on par with opendataloader — at ~2.5× its speed.</p></div>
<div class="callout"><div class="ct">Where we're working</div><p>Reading order still trails opendataloader slightly, and tables that need the visual structure only an OCR engine can see.</p></div>
<div class="callout"><div class="ct">Best fit</div><p>Native-text PDFs where speed, reading order, and table structure matter. pdf-inspector delivered the highest overall, reading-order, and table scores, along with the fastest complete run in this benchmark. That makes it a strong local default for reports, research papers, financial documents, invoices, and legal PDFs that need clean, structured Markdown without adding OCR latency or infrastructure.</p></div>
</div>
</div>
</section>