mu-ref/pdf-inspector
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(readme): refresh local parser benchmark results (#192)

Browse Source 
* docs(readme): refresh parser benchmark results

* docs(readme): include stored parser results

* docs(readme): refresh remaining parser results

* docs: sync benchmark references
This commit is contained in:
Abimael Martell
2026-07-31 22:58:46 -06:00
committed by GitHub
parent 7b7960ee73
commit a15ec2d68d
5 changed files with 34 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/benchmarking.md
+8 -5
View File
@@ -30,11 +30,14 @@ overwrite one another.
## Published comparison protocol
The public benchmark table was refreshed on July 16, 2026, on an Apple M4 Pro
using pdf-inspector 0.1.6, LiteParse 2.6.0, OpenDataLoader 2.1.1,
PyMuPDF4LLM 0.2.0, and MarkItDown 0.1.4. Every engine processed the same 200
PDFs with OCR disabled. Reported speed is the median of three complete corpus
runs; quality scores come from the benchmark evaluator over all 200 outputs.
The public benchmark table was refreshed on July 31, 2026, on an Apple M4 Pro
using pdf-inspector 0.2.6, LiteParse 2.10.1, OpenDataLoader 2.2.1,
PyMuPDF4LLM 0.2.0, and MarkItDown 0.1.5. Every engine processed the same 200
PDFs sequentially in a single process with OCR disabled. Reported speed is the
median of five alternating or rotating complete corpus runs after an excluded
warm-up run; quality scores come from the benchmark evaluator over all 200
outputs. Raw timings, predictions, evaluations, and charts are available in the
[results branch](https://github.com/firecrawl/opendataloader-bench/tree/abi/pdf-parser-benchmark-results).
## Optional backend evidence probe
docs/python.md
+6 -6
View File
@@ -18,13 +18,13 @@ Built by [Firecrawl](https://firecrawl.dev) to handle text-based PDFs locally in
| Engine | Overall | Reading order | Tables (TEDS) | Headings | Speed |
|---|---|---|---|---|---|
| **pdf-inspector** | **0.875** | **0.915** | **0.814** | 0.788 | **2.8s** |
| liteparse | 0.870 | 0.908 | 0.693 | **0.811** | 13.9s |
| opendataloader | 0.843 | 0.912 | 0.489 | 0.760 | 9.8s |
| pymupdf4llm | 0.735 | 0.886 | 0.401 | 0.424 | 15.5s |
| markitdown | 0.583 | 0.879 | 0.000 | 0.000 | 6.7s |
| **pdf-inspector** | **0.875** | **0.915** | **0.814** | 0.788 | **0.470s** |
| liteparse | 0.873 | 0.913 | 0.693 | **0.811** | 0.750s |
| opendataloader | 0.831 | 0.902 | 0.489 | 0.739 | 2.569s |
| pymupdf4llm | 0.735 | 0.886 | 0.401 | 0.424 | 17.117s |
| markitdown | 0.589 | 0.844 | 0.273 | 0.000 | 16.165s |
Refreshed July 16, 2026, on Apple M4 Pro; speed is the median of three complete corpus runs. Full methodology and versions are in the [repo README](https://github.com/firecrawl/pdf-inspector#benchmark).
Refreshed July 31, 2026, on Apple M4 Pro; speed is the median of five complete corpus runs after an excluded warm-up. Full methodology and versions are in the [repo README](https://github.com/firecrawl/pdf-inspector#benchmark), with raw timings and artifacts in the [results branch](https://github.com/firecrawl/opendataloader-bench/tree/abi/pdf-parser-benchmark-results).
## Install
docs/rust-api.md
+6 -6
View File
@@ -18,13 +18,13 @@ Built by [Firecrawl](https://firecrawl.dev) to handle text-based PDFs locally in
| Engine | Overall | Reading order | Tables (TEDS) | Headings | Speed |
|---|---|---|---|---|---|
| **pdf-inspector** | **0.875** | **0.915** | **0.814** | 0.788 | **2.8s** |
| liteparse | 0.870 | 0.908 | 0.693 | **0.811** | 13.9s |
| opendataloader | 0.843 | 0.912 | 0.489 | 0.760 | 9.8s |
| pymupdf4llm | 0.735 | 0.886 | 0.401 | 0.424 | 15.5s |
| markitdown | 0.583 | 0.879 | 0.000 | 0.000 | 6.7s |
| **pdf-inspector** | **0.875** | **0.915** | **0.814** | 0.788 | **0.470s** |
| liteparse | 0.873 | 0.913 | 0.693 | **0.811** | 0.750s |
| opendataloader | 0.831 | 0.902 | 0.489 | 0.739 | 2.569s |
| pymupdf4llm | 0.735 | 0.886 | 0.401 | 0.424 | 17.117s |
| markitdown | 0.589 | 0.844 | 0.273 | 0.000 | 16.165s |
Refreshed July 16, 2026, on Apple M4 Pro; speed is the median of three complete corpus runs. Full methodology and versions are in the [repo README](https://github.com/firecrawl/pdf-inspector#benchmark).
Refreshed July 31, 2026, on Apple M4 Pro; speed is the median of five complete corpus runs after an excluded warm-up. Full methodology and versions are in the [repo README](https://github.com/firecrawl/pdf-inspector#benchmark), with raw timings and artifacts in the [results branch](https://github.com/firecrawl/opendataloader-bench/tree/abi/pdf-parser-benchmark-results).
## Install