Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c19bb56c66 | ||
|
|
8552efba5e | ||
|
|
40672d6752 |
@@ -5,7 +5,7 @@
|
||||
[](https://pypi.org/project/pdf-inspector/)
|
||||
[](LICENSE)
|
||||
|
||||
Fast Rust library for PDF classification and text extraction. Detects whether a PDF is text-based or scanned, extracts text with position awareness, and converts to clean Markdown — all without OCR. Includes bindings for [Python](docs/python.md), [Node.js](napi/README.md), and [browser WebAssembly](wasm/README.md).
|
||||
Fast Rust library for PDF classification and text extraction. By default it detects whether a PDF is text-based or scanned, extracts text with position awareness, and converts to clean Markdown without OCR. Native Rust and CLI consumers can opt into selective OCR. Includes bindings for [Python](docs/python.md), [Node.js](napi/README.md), and [browser WebAssembly](wasm/README.md).
|
||||
|
||||
Built by [Firecrawl](https://firecrawl.dev) to handle text-based PDFs locally in under 200ms, skipping expensive OCR services for the ~54% of PDFs that don't need them.
|
||||
|
||||
@@ -18,9 +18,10 @@ Built by [Firecrawl](https://firecrawl.dev) to handle text-based PDFs locally in
|
||||
- **CID font support** — ToUnicode CMap decoding for Type0/Identity-H fonts, UTF-16BE, UTF-8, and Latin-1 encodings.
|
||||
- **Multi-column layout** — Automatic detection of newspaper-style columns, sequential reading order, and RTL text support.
|
||||
- **Encoding issue detection** — Automatically flags broken font encodings so callers can fall back to OCR.
|
||||
- **Optional OCR** — An opt-in Rust and CLI feature selectively renders only pages that need OCR, runs PP-OCRv6 Small locally, and preserves per-page provenance and hosted-fallback recommendations.
|
||||
- **Single document load** — The document is parsed once and shared between detection and extraction, avoiding redundant I/O.
|
||||
- **Browser WebAssembly** — Run the same Rust parser locally in browsers and Web Workers, with embedded CMaps and no server round trip.
|
||||
- **Lightweight** — Pure Rust, no ML models, no external services. Single dependency on `lopdf` for PDF parsing.
|
||||
- **Lightweight by default** — The default build is pure Rust with no ML models or external services. PDFium, ONNX Runtime, and OCR models are added only when the native `ocr` feature is selected and remain external runtime artifacts.
|
||||
|
||||
## Benchmark
|
||||
|
||||
@@ -160,6 +161,19 @@ detect-pdf document.pdf --json
|
||||
detect-pdf document.pdf --analyze --json
|
||||
```
|
||||
|
||||
OCR is a separate native CLI build and does not change the default package:
|
||||
|
||||
```bash
|
||||
cargo install pdf-inspector --features ocr --bin pdf2md
|
||||
PDFIUM_LIB_PATH=/path/to/libpdfium ORT_DYLIB_PATH=/path/to/libonnxruntime \
|
||||
pdf2md scan.pdf --ocr auto --json
|
||||
```
|
||||
|
||||
The OCR JSON envelope is versioned and reports routed pages, per-page source
|
||||
and confidence, warnings, and pages recommended for the hosted document
|
||||
pipeline. See the [Rust API guide](docs/rust-api.md#complete-ocr-api) for model
|
||||
cache and offline configuration.
|
||||
|
||||
From a source checkout, use `cargo run --bin pdf2md -- document.pdf` or `cargo run --bin detect-pdf -- document.pdf` instead.
|
||||
|
||||
## Architecture
|
||||
|
||||
+36
-27
@@ -1,6 +1,6 @@
|
||||
# pdf-inspector
|
||||
|
||||
Fast PDF classification and text extraction. Detects whether a PDF is text-based or scanned, extracts text with position awareness, and converts to clean Markdown — all without OCR. The default build is pure Rust, has no ML models or external services, and uses [lopdf](https://crates.io/crates/lopdf) for PDF parsing. Also available for [Python](https://pypi.org/project/pdf-inspector/) and [Node.js](https://www.npmjs.com/package/@firecrawl/pdf-inspector/).
|
||||
Fast PDF classification and text extraction. The default build detects whether a PDF is text-based or scanned, extracts text with position awareness, and converts to clean Markdown without OCR. It is pure Rust, has no ML models or external services, and uses [lopdf](https://crates.io/crates/lopdf) for PDF parsing. Native Rust and CLI consumers can opt into selective OCR. Also available for [Python](https://pypi.org/project/pdf-inspector/) and [Node.js](https://www.npmjs.com/package/@firecrawl/pdf-inspector/).
|
||||
|
||||
Built by [Firecrawl](https://firecrawl.dev) to handle text-based PDFs locally in under 200ms, skipping expensive OCR services for the ~54% of PDFs that don't need them.
|
||||
|
||||
@@ -123,10 +123,10 @@ The native-only `vision` feature exposes the stable seam used by OCR
|
||||
integrations without selecting or embedding an inference runtime. The
|
||||
separate `model-cache` feature adds pinned artifact management:
|
||||
|
||||
- `PageRenderer`, `OcrEngine`, and `LayoutEngine` traits;
|
||||
- `PageRenderer` and `OcrEngine` traits;
|
||||
- renderer-neutral owned page buffers and affine pixel↔PDF transforms;
|
||||
- `OcrOptions` and opt-in `Off`/`Auto`/`Force` routing modes;
|
||||
- positioned OCR/layout results and per-page provenance types; and
|
||||
- positioned OCR results and per-page provenance types; and
|
||||
- a versioned PP-OCRv6 Small manifest with checksum-verified, locked, atomic
|
||||
model-cache installation and explicit offline-directory overrides.
|
||||
|
||||
@@ -135,9 +135,9 @@ separate `model-cache` feature adds pinned artifact management:
|
||||
pdf-inspector = { version = "1", features = ["vision", "model-cache"] }
|
||||
```
|
||||
|
||||
The OCR contracts preserve existing behavior by default: OCR is `Off`, learned
|
||||
layout is disabled, and model resolution is never reached. `ModelStore` itself
|
||||
does not access the network. The optional `model-download` feature provides an
|
||||
The OCR contracts preserve existing behavior by default: OCR is `Off` and
|
||||
model resolution is never reached. `ModelStore` itself does not access the
|
||||
network. The optional `model-download` feature provides an
|
||||
HTTPS downloader that streams pinned artifacts into the checksum-verified
|
||||
cache only after routing has selected OCR work. Offline consumers set an
|
||||
explicit model directory and `ModelDownloadPolicy::Offline`. Renderer-only
|
||||
@@ -171,9 +171,10 @@ and `PdfiumRenderer` implements the renderer-neutral `PageRenderer` trait.
|
||||
pdf-inspector = { version = "1", features = ["render-pdfium"] }
|
||||
```
|
||||
|
||||
PDFium is loaded at runtime. Set `PDFIUM_LIB_PATH`, place its shared library
|
||||
next to the executable, or use another discovery route supported by
|
||||
`firecrawl-pdfium`.
|
||||
PDFium is loaded at runtime and is not bundled into the crate. Set
|
||||
`PDFIUM_LIB_PATH` to the platform shared library, place that library next to
|
||||
the executable, or use another discovery route supported by
|
||||
`firecrawl-pdfium`. A load failure reports this prerequisite directly.
|
||||
|
||||
```rust
|
||||
use pdf_inspector::vision::{PdfiumRenderer, RenderOptions};
|
||||
@@ -205,9 +206,11 @@ The native-only `ocr-oar` feature adds a CPU PP-OCRv6 Small implementation of
|
||||
not enable model auto-download, ONNX Runtime download, or PDF rendering. Model
|
||||
files remain external, must match the pinned manifest, and are opened only
|
||||
after `ModelStore` verifies their exact size and SHA-256 digest. Install an
|
||||
ONNX Runtime shared library separately and set `ORT_DYLIB_PATH` when it is not
|
||||
available through the platform library search path. The feature currently
|
||||
requires Rust 1.95 or newer, matching OAR 0.9.1's MSRV.
|
||||
ONNX Runtime shared library separately and set `ORT_DYLIB_PATH` to its full
|
||||
path when it is not available through the platform library search path. The
|
||||
runtime is resolved only when an OCR engine is first constructed; clean
|
||||
`Auto` requests do not require it. The feature currently requires Rust 1.95
|
||||
or newer, matching OAR 0.9.1's MSRV.
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
@@ -329,7 +332,10 @@ let fused = fuse_ocr_pages(
|
||||
for page in &fused.pages {
|
||||
println!("{}", page.markdown);
|
||||
if page.provenance.hosted_recommended {
|
||||
eprintln!("page {} needs the hosted document pipeline", page.page + 1);
|
||||
eprintln!(
|
||||
"page {} needs the hosted document pipeline",
|
||||
page.page_number,
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -354,15 +360,11 @@ pdf-inspector = { version = "1", features = ["ocr"] }
|
||||
```
|
||||
|
||||
```rust
|
||||
use pdf_inspector::vision::{
|
||||
process_pdf_with_ocr, OcrMode, OcrPdfOptions,
|
||||
};
|
||||
use pdf_inspector::vision::{process_pdf_with_ocr, OcrPdfOptions};
|
||||
|
||||
let result = process_pdf_with_ocr(
|
||||
"document.pdf",
|
||||
OcrPdfOptions::new()
|
||||
.mode(OcrMode::Auto)
|
||||
.pages([1, 2, 3]),
|
||||
OcrPdfOptions::auto().page_numbers([1, 2, 3]),
|
||||
)?;
|
||||
|
||||
println!("{}", result.markdown);
|
||||
@@ -377,17 +379,21 @@ Native extraction always runs first. In `Auto`, a clean PDF returns before
|
||||
PDFium loading, model-cache access, HTTP, or OAR initialization. Model files
|
||||
remain external and the default crate feature set remains unchanged. `Off`
|
||||
provides the same native-only behavior through the OCR result/provenance
|
||||
shape; `Force` renders every selected page. Learned layout intentionally
|
||||
returns an explicit unsupported error in this lightweight pipeline.
|
||||
shape; `Force` renders every selected page. OCR uses the existing deterministic
|
||||
table, column, reading-order, and Markdown assembly path; no learned layout
|
||||
model is included.
|
||||
|
||||
For ambiguous mixed pages, `Auto` privately retains clean native fragments
|
||||
instead of discarding them when OCR is selected. After recognition it compares
|
||||
script-agnostic text quality, OCR confidence, character overlap, and material
|
||||
new coverage. Exact native text wins over a duplicate or weak OCR hypothesis;
|
||||
complementary image-backed text is fused; and pages where both candidates are
|
||||
weak recommend the hosted document pipeline. Public native-only extraction
|
||||
continues to suppress pages marked unreliable, and clean text documents pay no
|
||||
renderer or model-initialization cost.
|
||||
weak recommend the hosted document pipeline. A page routed because native
|
||||
coverage appeared incomplete also recommends hosted processing when confident
|
||||
OCR only duplicates the retained fragment: the agreement preserves trustworthy
|
||||
text, but neither hypothesis proves full-page coverage. Public native-only
|
||||
extraction continues to suppress pages marked unreliable, and clean text
|
||||
documents pay no renderer or model-initialization cost.
|
||||
|
||||
In `Auto`, pages routed only for suspicious font encoding or vectorized text
|
||||
first get a bounded positioned-text probe through PDFium. A credible recovered
|
||||
@@ -407,11 +413,13 @@ not hot-reload a running process; restart the process when intentionally
|
||||
replacing files at the same paths. CPU inference uses at most four intra-op
|
||||
threads per ONNX session so a single small page does not oversubscribe larger
|
||||
hosts, and recognizes variable-width line crops individually to avoid
|
||||
padding-heavy CPU batches.
|
||||
padding-heavy CPU batches. The high-level pipeline renders and fuses at most
|
||||
four routed pages at a time, bounding bitmap memory on long documents.
|
||||
|
||||
Build the CLI with the same opt-in feature:
|
||||
|
||||
```bash
|
||||
cargo install pdf-inspector --features ocr --bin pdf2md
|
||||
cargo build --release --features ocr --bin pdf2md
|
||||
pdf2md document.pdf --ocr auto --raw
|
||||
pdf2md document.pdf --ocr auto --json
|
||||
@@ -420,10 +428,11 @@ pdf2md document.pdf --ocr auto --ocr-offline --ocr-model-dir /opt/models/pp-ocrv
|
||||
|
||||
CLI controls include `--ocr-dpi`, `--ocr-min-confidence`,
|
||||
`--ocr-hosted-threshold`, `--select-pages`, and the existing encrypted-PDF
|
||||
`--password` option. JSON output includes per-page Markdown, source/model
|
||||
`--password` option. JSON output has `schema_version: 1` and includes per-page Markdown, source/model
|
||||
provenance, confidence, timings, warnings, routed pages, and hosted-fallback
|
||||
recommendations. Page numbers in `OcrPdfResult` and its per-page provenance
|
||||
are 1-indexed, matching the PDF page numbers accepted by `OcrPdfOptions::pages`.
|
||||
are 1-indexed, matching the PDF page numbers accepted by
|
||||
`OcrPdfOptions::page_numbers`.
|
||||
|
||||
Extract per-page Markdown (one string per page, plus document-wide layout
|
||||
metadata):
|
||||
|
||||
+61
-29
@@ -165,8 +165,8 @@ fn format_ocr_json(result: &OcrPdfResult) -> String {
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
format!(
|
||||
r#"{{"page":{},"source":"{}","markdown":"{}","ocr_model":{},"render_dpi":{},"ocr_confidence":{},"hosted_recommended":{},"timings":{{"render_ms":{},"ocr_ms":{},"layout_ms":{},"assembly_ms":{}}},"warnings":[{}]}}"#,
|
||||
provenance.page,
|
||||
r#"{{"page":{},"source":"{}","markdown":"{}","ocr_model":{},"render_dpi":{},"ocr_confidence":{},"hosted_recommended":{},"timings":{{"render_ms":{},"ocr_ms":{},"assembly_ms":{}}},"warnings":[{}]}}"#,
|
||||
provenance.page_number,
|
||||
source,
|
||||
json_escape(&page.markdown),
|
||||
model,
|
||||
@@ -175,7 +175,6 @@ fn format_ocr_json(result: &OcrPdfResult) -> String {
|
||||
provenance.hosted_recommended,
|
||||
provenance.timings.render_ms,
|
||||
provenance.timings.ocr_ms,
|
||||
provenance.timings.layout_ms,
|
||||
provenance.timings.assembly_ms,
|
||||
warnings,
|
||||
)
|
||||
@@ -196,7 +195,7 @@ fn format_ocr_json(result: &OcrPdfResult) -> String {
|
||||
.join(",");
|
||||
let ocr_reasons = format_ocr_reasons_by_page(&result.ocr_reasons_by_page);
|
||||
format!(
|
||||
r#"{{"page_count":{},"processing_time_ms":{},"render_time_ms":{},"ocr_time_ms":{},"pages_recommended_for_ocr":[{}],"pages_routed_to_ocr":[{}],"pages_recommending_hosted":[{}],"ocr_reasons_by_page":[{}],"is_complex":{},"pages_with_tables":[{}],"pages_with_columns":[{}],"pages":[{}],"markdown":"{}"}}"#,
|
||||
r#"{{"schema_version":1,"page_count":{},"processing_time_ms":{},"render_time_ms":{},"ocr_time_ms":{},"pages_recommended_for_ocr":[{}],"pages_routed_to_ocr":[{}],"pages_recommending_hosted":[{}],"ocr_reasons_by_page":[{}],"is_complex":{},"pages_with_tables":[{}],"pages_with_columns":[{}],"pages":[{}],"markdown":"{}"}}"#,
|
||||
result.page_count,
|
||||
result.processing_time_ms,
|
||||
result.render_time_ms,
|
||||
@@ -224,6 +223,19 @@ fn argument_value<'a>(args: &'a [String], name: &str) -> Result<Option<&'a str>,
|
||||
.transpose()
|
||||
}
|
||||
|
||||
fn format_ocr_error_json(error: &str) -> String {
|
||||
format!(r#"{{"schema_version":1,"error":"{}"}}"#, json_escape(error))
|
||||
}
|
||||
|
||||
fn exit_ocr_error(error: &str, json_output: bool) -> ! {
|
||||
if json_output {
|
||||
println!("{}", format_ocr_error_json(error));
|
||||
} else {
|
||||
eprintln!("Error: {error}");
|
||||
}
|
||||
process::exit(1);
|
||||
}
|
||||
|
||||
#[cfg(all(feature = "ocr", not(target_arch = "wasm32")))]
|
||||
fn float_argument(args: &[String], name: &str, default: f32) -> Result<f32, String> {
|
||||
argument_value(args, name)?
|
||||
@@ -247,7 +259,9 @@ fn extract_items_json(
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{extract_items_json, format_items_json};
|
||||
use super::{extract_items_json, format_items_json, format_ocr_error_json};
|
||||
#[cfg(all(feature = "ocr", not(target_arch = "wasm32")))]
|
||||
use super::{format_ocr_json, process_pdf_with_ocr, OcrPdfOptions};
|
||||
use pdf_inspector::extractor::ItemType;
|
||||
use pdf_inspector::TextItem;
|
||||
|
||||
@@ -297,6 +311,27 @@ mod tests {
|
||||
"decrypted item JSON should contain fixture text, got {json}"
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(all(feature = "ocr", not(target_arch = "wasm32")))]
|
||||
#[test]
|
||||
fn ocr_json_has_a_versioned_stable_envelope() {
|
||||
let result =
|
||||
process_pdf_with_ocr("tests/fixtures/thermo-freon12.pdf", OcrPdfOptions::new())
|
||||
.unwrap();
|
||||
let json = format_ocr_json(&result);
|
||||
|
||||
assert!(json.starts_with(r#"{"schema_version":1,"page_count":3,"#));
|
||||
assert!(json.contains(r#""page":1,"source":"native""#));
|
||||
assert!(!json.contains("layout_ms"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ocr_json_errors_use_the_same_versioned_envelope() {
|
||||
assert_eq!(
|
||||
format_ocr_error_json("bad \"value\""),
|
||||
r#"{"schema_version":1,"error":"bad \"value\""}"#
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a page specification like "1,3,5-10,20" into a HashSet of page numbers.
|
||||
@@ -441,23 +476,27 @@ fn main() {
|
||||
.iter()
|
||||
.any(|option| args.iter().any(|argument| argument == option));
|
||||
if ocr_mode_argument.is_none() && has_ocr_only_option {
|
||||
eprintln!("Error: OCR options require --ocr off, --ocr auto, or --ocr force");
|
||||
process::exit(1);
|
||||
exit_ocr_error(
|
||||
"OCR options require --ocr off, --ocr auto, or --ocr force",
|
||||
json_output,
|
||||
);
|
||||
}
|
||||
|
||||
if let Some(mode) = ocr_mode_argument {
|
||||
if items_json_output || detect_only || analyze {
|
||||
eprintln!(
|
||||
"Error: --ocr cannot be combined with --items-json, --detect-only, or --analyze"
|
||||
exit_ocr_error(
|
||||
"--ocr cannot be combined with --items-json, --detect-only, or --analyze",
|
||||
json_output,
|
||||
);
|
||||
process::exit(1);
|
||||
}
|
||||
|
||||
#[cfg(not(all(feature = "ocr", not(target_arch = "wasm32"))))]
|
||||
{
|
||||
let _ = mode;
|
||||
eprintln!("Error: this pdf2md build does not include OCR; rebuild with --features ocr");
|
||||
process::exit(1);
|
||||
exit_ocr_error(
|
||||
"this pdf2md build does not include OCR; rebuild with --features ocr",
|
||||
json_output,
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(all(feature = "ocr", not(target_arch = "wasm32")))]
|
||||
@@ -467,28 +506,26 @@ fn main() {
|
||||
"auto" => OcrMode::Auto,
|
||||
"force" => OcrMode::Force,
|
||||
value => {
|
||||
eprintln!("Error: invalid --ocr mode {value:?}; expected off, auto, or force");
|
||||
process::exit(1);
|
||||
exit_ocr_error(
|
||||
&format!("invalid --ocr mode {value:?}; expected off, auto, or force"),
|
||||
json_output,
|
||||
);
|
||||
}
|
||||
};
|
||||
let dpi = float_argument(&args, "--ocr-dpi", 150.0).unwrap_or_else(|error| {
|
||||
eprintln!("Error: {error}");
|
||||
process::exit(1);
|
||||
exit_ocr_error(&error, json_output);
|
||||
});
|
||||
let minimum_confidence = float_argument(&args, "--ocr-min-confidence", 0.0)
|
||||
.unwrap_or_else(|error| {
|
||||
eprintln!("Error: {error}");
|
||||
process::exit(1);
|
||||
exit_ocr_error(&error, json_output);
|
||||
});
|
||||
let hosted_threshold = float_argument(&args, "--ocr-hosted-threshold", 0.5)
|
||||
.unwrap_or_else(|error| {
|
||||
eprintln!("Error: {error}");
|
||||
process::exit(1);
|
||||
exit_ocr_error(&error, json_output);
|
||||
});
|
||||
let model_directory =
|
||||
argument_value(&args, "--ocr-model-dir").unwrap_or_else(|error| {
|
||||
eprintln!("Error: {error}");
|
||||
process::exit(1);
|
||||
exit_ocr_error(&error, json_output);
|
||||
});
|
||||
|
||||
let mut ocr = OcrOptions::new()
|
||||
@@ -511,7 +548,7 @@ fn main() {
|
||||
.markdown(markdown)
|
||||
.hosted_recommendation_confidence(hosted_threshold);
|
||||
if let Some(pages) = page_filter.clone() {
|
||||
pdf_options = pdf_options.pages(pages);
|
||||
pdf_options = pdf_options.page_numbers(pages);
|
||||
}
|
||||
if let Some(password) = password.clone() {
|
||||
pdf_options = pdf_options.password(password);
|
||||
@@ -549,12 +586,7 @@ fn main() {
|
||||
}
|
||||
}
|
||||
Err(error) => {
|
||||
if json_output {
|
||||
println!(r#"{{"error":"{}"}}"#, json_escape(&error.to_string()));
|
||||
} else {
|
||||
eprintln!("Error: {error}");
|
||||
}
|
||||
process::exit(1);
|
||||
exit_ocr_error(&error.to_string(), json_output);
|
||||
}
|
||||
}
|
||||
return;
|
||||
|
||||
+3
-152
@@ -1,4 +1,4 @@
|
||||
//! Public contracts between rendering, OCR, layout, and orchestration.
|
||||
//! Public contracts between rendering, OCR, and orchestration.
|
||||
|
||||
use std::error::Error;
|
||||
use std::path::PathBuf;
|
||||
@@ -18,19 +18,6 @@ pub enum OcrMode {
|
||||
Force,
|
||||
}
|
||||
|
||||
/// Resource/quality profile for the OCR engine.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
#[non_exhaustive]
|
||||
pub enum OcrProfile {
|
||||
/// Lowest latency and memory footprint.
|
||||
Edge,
|
||||
/// OCR-oriented balance of quality and CPU cost.
|
||||
#[default]
|
||||
Balanced,
|
||||
/// Highest quality within the lightweight model family.
|
||||
Quality,
|
||||
}
|
||||
|
||||
/// Controls whether missing model artifacts may be fetched.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
#[non_exhaustive]
|
||||
@@ -47,12 +34,8 @@ pub enum ModelDownloadPolicy {
|
||||
pub struct OcrOptions {
|
||||
/// Page-level routing behavior.
|
||||
pub mode: OcrMode,
|
||||
/// Local quality/resource profile.
|
||||
pub profile: OcrProfile,
|
||||
/// Drop recognition spans below this confidence threshold.
|
||||
pub minimum_confidence: f32,
|
||||
/// Optional language hints understood by the selected engine.
|
||||
pub languages: Vec<String>,
|
||||
/// Optional directory containing an offline model set.
|
||||
pub model_directory: Option<PathBuf>,
|
||||
/// Whether a missing pinned artifact may be downloaded.
|
||||
@@ -63,9 +46,7 @@ impl Default for OcrOptions {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
mode: OcrMode::Off,
|
||||
profile: OcrProfile::Balanced,
|
||||
minimum_confidence: 0.0,
|
||||
languages: Vec::new(),
|
||||
model_directory: None,
|
||||
model_downloads: ModelDownloadPolicy::IfMissing,
|
||||
}
|
||||
@@ -84,24 +65,12 @@ impl OcrOptions {
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the local resource/quality profile.
|
||||
pub fn profile(mut self, profile: OcrProfile) -> Self {
|
||||
self.profile = profile;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the minimum accepted recognition confidence.
|
||||
pub fn minimum_confidence(mut self, minimum_confidence: f32) -> Self {
|
||||
self.minimum_confidence = minimum_confidence;
|
||||
self
|
||||
}
|
||||
|
||||
/// Replaces the language hints passed to the OCR engine.
|
||||
pub fn languages(mut self, languages: impl IntoIterator<Item = impl Into<String>>) -> Self {
|
||||
self.languages = languages.into_iter().map(Into::into).collect();
|
||||
self
|
||||
}
|
||||
|
||||
/// Uses an explicit model directory, suitable for offline packaging.
|
||||
pub fn model_directory(mut self, directory: impl Into<PathBuf>) -> Self {
|
||||
self.model_directory = Some(directory.into());
|
||||
@@ -115,55 +84,6 @@ impl OcrOptions {
|
||||
}
|
||||
}
|
||||
|
||||
/// Configuration for an optional learned layout engine.
|
||||
///
|
||||
/// Layout inference is disabled by default. Existing deterministic layout,
|
||||
/// table, and Markdown logic remains the assembly path when this is disabled.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct LayoutOptions {
|
||||
/// Whether the learned layout extension may run.
|
||||
pub enabled: bool,
|
||||
/// Drop layout regions below this confidence threshold.
|
||||
pub minimum_confidence: f32,
|
||||
/// Optional directory containing an offline layout model set.
|
||||
pub model_directory: Option<PathBuf>,
|
||||
}
|
||||
|
||||
impl Default for LayoutOptions {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
enabled: false,
|
||||
minimum_confidence: 0.0,
|
||||
model_directory: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl LayoutOptions {
|
||||
/// Creates layout options with learned layout disabled.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Enables or disables learned layout inference.
|
||||
pub fn enabled(mut self, enabled: bool) -> Self {
|
||||
self.enabled = enabled;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the minimum accepted region confidence.
|
||||
pub fn minimum_confidence(mut self, minimum_confidence: f32) -> Self {
|
||||
self.minimum_confidence = minimum_confidence;
|
||||
self
|
||||
}
|
||||
|
||||
/// Uses an explicit layout model directory.
|
||||
pub fn model_directory(mut self, directory: impl Into<PathBuf>) -> Self {
|
||||
self.model_directory = Some(directory.into());
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
/// A point in bitmap space, measured from the top-left in pixels.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq)]
|
||||
pub struct ImagePoint {
|
||||
@@ -230,7 +150,7 @@ pub struct OcrSpan {
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct OcrPage {
|
||||
/// 1-indexed PDF page number.
|
||||
pub page: u32,
|
||||
pub page_number: u32,
|
||||
/// Positioned recognition spans.
|
||||
pub spans: Vec<OcrSpan>,
|
||||
/// Mean confidence across accepted spans, when available.
|
||||
@@ -243,54 +163,6 @@ pub struct OcrPage {
|
||||
pub warnings: Vec<String>,
|
||||
}
|
||||
|
||||
/// Normalized semantic class emitted by a learned layout engine.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
#[non_exhaustive]
|
||||
pub enum LayoutRegionKind {
|
||||
/// Body or other prose text.
|
||||
Text,
|
||||
/// Document heading or title.
|
||||
Heading,
|
||||
/// Table region.
|
||||
Table,
|
||||
/// Figure/image region.
|
||||
Figure,
|
||||
/// Figure or table caption.
|
||||
Caption,
|
||||
/// Header/footer/page furniture.
|
||||
Furniture,
|
||||
/// Model-specific class retained without changing the common taxonomy.
|
||||
Other(String),
|
||||
}
|
||||
|
||||
/// One learned layout region in bitmap coordinates.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct LayoutRegion {
|
||||
/// Normalized semantic class.
|
||||
pub kind: LayoutRegionKind,
|
||||
/// Region polygon in the original rendered page's pixel space.
|
||||
pub polygon: ImageQuad,
|
||||
/// Model confidence in the inclusive range 0–1.
|
||||
pub confidence: f32,
|
||||
/// Optional model-provided reading-order position.
|
||||
pub reading_order: Option<u32>,
|
||||
}
|
||||
|
||||
/// Learned layout output for one 1-indexed page.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct LayoutPage {
|
||||
/// 1-indexed PDF page number.
|
||||
pub page: u32,
|
||||
/// Semantic regions.
|
||||
pub regions: Vec<LayoutRegion>,
|
||||
/// Exact model identity used for this result.
|
||||
pub model: ModelIdentity,
|
||||
/// Layout inference wall time for this page.
|
||||
pub processing_time_ms: u64,
|
||||
/// Non-fatal engine warnings.
|
||||
pub warnings: Vec<String>,
|
||||
}
|
||||
|
||||
/// How final page content was sourced.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[non_exhaustive]
|
||||
@@ -310,8 +182,6 @@ pub struct VisionTimings {
|
||||
pub render_ms: u64,
|
||||
/// OCR wall time.
|
||||
pub ocr_ms: u64,
|
||||
/// Optional learned layout wall time.
|
||||
pub layout_ms: u64,
|
||||
/// Native/OCR fusion and assembly wall time.
|
||||
pub assembly_ms: u64,
|
||||
}
|
||||
@@ -320,13 +190,11 @@ pub struct VisionTimings {
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct PageProvenance {
|
||||
/// 1-indexed PDF page number.
|
||||
pub page: u32,
|
||||
pub page_number: u32,
|
||||
/// Final page-content source.
|
||||
pub source: PageContentSource,
|
||||
/// OCR model, when OCR ran.
|
||||
pub ocr_model: Option<ModelIdentity>,
|
||||
/// Learned layout model, when layout inference ran.
|
||||
pub layout_model: Option<ModelIdentity>,
|
||||
/// Render resolution used for local vision.
|
||||
pub render_dpi: Option<f32>,
|
||||
/// Mean accepted OCR confidence, when available.
|
||||
@@ -371,23 +239,6 @@ pub trait OcrEngine: Send + Sync {
|
||||
) -> Result<Vec<OcrPage>, Self::Error>;
|
||||
}
|
||||
|
||||
/// Optional learned semantic layout extension.
|
||||
pub trait LayoutEngine: Send + Sync {
|
||||
/// Engine-specific failure type.
|
||||
type Error: Error + Send + Sync + 'static;
|
||||
|
||||
/// Exact model identity used by this engine instance.
|
||||
fn model(&self) -> &ModelIdentity;
|
||||
|
||||
/// Analyzes rendered pages, optionally using their OCR spans.
|
||||
fn analyze(
|
||||
&self,
|
||||
pages: &[RenderedPage],
|
||||
ocr: &[OcrPage],
|
||||
options: &LayoutOptions,
|
||||
) -> Result<Vec<LayoutPage>, Self::Error>;
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
+18
-12
@@ -74,7 +74,7 @@ impl OcrFusionOptions {
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct FusedPageMarkdown {
|
||||
/// 1-indexed document page number, matching OCR and provenance fields.
|
||||
pub page: u32,
|
||||
pub page_number: u32,
|
||||
/// Final page Markdown.
|
||||
pub markdown: String,
|
||||
/// Native/OCR source, model, timing, and fallback metadata.
|
||||
@@ -184,8 +184,9 @@ pub fn ocr_page_to_markdown(
|
||||
/// OCR replaces pages whose native extraction was already rejected. On clean
|
||||
/// native pages (for example in `Force` mode), normalized duplicate OCR blocks
|
||||
/// are removed and only genuinely additional blocks are appended. Pages that
|
||||
/// needed OCR but still have no credible OCR result recommend the hosted
|
||||
/// document pipeline instead of silently presenting an empty result as final.
|
||||
/// needed OCR but still have no credible OCR result, or whose confident OCR
|
||||
/// only repeats an incomplete native fragment, recommend the hosted document
|
||||
/// pipeline instead of silently presenting partial content as final.
|
||||
pub fn fuse_ocr_pages(
|
||||
native_pages: &[PageMarkdown],
|
||||
ocr_run: &OcrRun,
|
||||
@@ -330,13 +331,12 @@ fn fuse_ocr_pages_impl(
|
||||
};
|
||||
|
||||
pages.push(FusedPageMarkdown {
|
||||
page: page_number,
|
||||
page_number,
|
||||
markdown,
|
||||
provenance: PageProvenance {
|
||||
page: page_number,
|
||||
page_number,
|
||||
source,
|
||||
ocr_model,
|
||||
layout_model: None,
|
||||
render_dpi: ocr_by_page
|
||||
.contains_key(&page_number)
|
||||
.then_some(options.render_dpi),
|
||||
@@ -344,7 +344,6 @@ fn fuse_ocr_pages_impl(
|
||||
timings: VisionTimings {
|
||||
render_ms: render_by_page.get(&page_number).copied().unwrap_or(0),
|
||||
ocr_ms,
|
||||
layout_ms: 0,
|
||||
assembly_ms: elapsed_ms(assembly_started),
|
||||
},
|
||||
warnings,
|
||||
@@ -405,7 +404,11 @@ fn choose_adaptive_content(
|
||||
"kept trustworthy {} because OCR added no material coverage",
|
||||
native.origin.description()
|
||||
),
|
||||
recommend_hosted: false,
|
||||
// The candidate exists only because this page was routed with
|
||||
// incomplete native coverage. Agreement between two partial
|
||||
// hypotheses preserves trustworthy text, but does not prove that
|
||||
// the rest of the page was recovered.
|
||||
recommend_hosted: true,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1011,7 +1014,7 @@ mod tests {
|
||||
RoutedOcrPage {
|
||||
rendered: rendered_page(page),
|
||||
ocr: OcrPage {
|
||||
page,
|
||||
page_number: page,
|
||||
spans,
|
||||
mean_confidence: confidence,
|
||||
model: ModelIdentity::new("test-ocr", "v1"),
|
||||
@@ -1066,8 +1069,11 @@ mod tests {
|
||||
< result.pages[0].markdown.find("Second").unwrap()
|
||||
);
|
||||
assert_eq!(result.pages[0].provenance.source, PageContentSource::Ocr);
|
||||
assert_eq!(result.pages[0].page, 1);
|
||||
assert_eq!(result.pages[0].page, result.pages[0].provenance.page);
|
||||
assert_eq!(result.pages[0].page_number, 1);
|
||||
assert_eq!(
|
||||
result.pages[0].page_number,
|
||||
result.pages[0].provenance.page_number
|
||||
);
|
||||
assert_eq!(
|
||||
result.pages[0].provenance.ocr_model.as_ref().unwrap().name,
|
||||
"test-ocr"
|
||||
@@ -1243,7 +1249,7 @@ mod tests {
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(result.pages[0].provenance.source, PageContentSource::Native);
|
||||
assert!(!result.pages[0].provenance.hosted_recommended);
|
||||
assert!(result.pages[0].provenance.hosted_recommended);
|
||||
assert_eq!(result.pages[0].markdown.matches("Invoice").count(), 1);
|
||||
assert!(result.pages[0].provenance.warnings[0].contains("no material coverage"));
|
||||
}
|
||||
|
||||
+2
-3
@@ -30,9 +30,8 @@ mod pdfium;
|
||||
|
||||
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
|
||||
pub use contracts::{
|
||||
ImagePoint, ImageQuad, LayoutEngine, LayoutOptions, LayoutPage, LayoutRegion, LayoutRegionKind,
|
||||
ModelDownloadPolicy, ModelIdentity, OcrEngine, OcrMode, OcrOptions, OcrPage, OcrProfile,
|
||||
OcrSpan, PageContentSource, PageProvenance, PageRenderer, VisionTimings,
|
||||
ImagePoint, ImageQuad, ModelDownloadPolicy, ModelIdentity, OcrEngine, OcrMode, OcrOptions,
|
||||
OcrPage, OcrSpan, PageContentSource, PageProvenance, PageRenderer, VisionTimings,
|
||||
};
|
||||
#[cfg(all(feature = "model-download", not(target_arch = "wasm32")))]
|
||||
pub use download::{HttpModelDownloadError, HttpModelDownloader, DEFAULT_MODEL_DOWNLOAD_TIMEOUT};
|
||||
|
||||
+4
-6
@@ -49,7 +49,9 @@ pub enum OarOcrError {
|
||||
page: u32,
|
||||
},
|
||||
/// The external ONNX Runtime shared library could not be loaded.
|
||||
#[error("failed to load ONNX Runtime from {path}: {source}")]
|
||||
#[error(
|
||||
"failed to load ONNX Runtime from {path}; install a compatible ONNX Runtime shared library or set ORT_DYLIB_PATH to its path: {source}"
|
||||
)]
|
||||
OnnxRuntimeLoad {
|
||||
/// Requested shared-library path or platform library name.
|
||||
path: PathBuf,
|
||||
@@ -143,10 +145,6 @@ impl OarOcrEngine {
|
||||
}
|
||||
|
||||
let mut warnings = Vec::new();
|
||||
if !options.languages.is_empty() {
|
||||
warnings
|
||||
.push("language hints are not used by the PP-OCRv6 Small OAR backend".to_string());
|
||||
}
|
||||
if missing_recognition > 0 {
|
||||
warnings.push(format!(
|
||||
"discarded {missing_recognition} regions without usable recognition output"
|
||||
@@ -166,7 +164,7 @@ impl OarOcrEngine {
|
||||
let processing_time_ms = u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX);
|
||||
|
||||
Ok(OcrPage {
|
||||
page: page.page(),
|
||||
page_number: page.page(),
|
||||
spans,
|
||||
mean_confidence,
|
||||
model: self.model.clone(),
|
||||
|
||||
+12
-2
@@ -49,6 +49,15 @@ pub enum RenderError {
|
||||
/// Number of pages in the document.
|
||||
page_count: usize,
|
||||
},
|
||||
/// The PDFium shared library could not be discovered or loaded.
|
||||
#[error(
|
||||
"failed to load PDFium; install a compatible PDFium shared library or set PDFIUM_LIB_PATH to its path"
|
||||
)]
|
||||
PdfiumLoad {
|
||||
/// Dynamic loading failure.
|
||||
#[source]
|
||||
source: firecrawl_pdfium::Error,
|
||||
},
|
||||
/// PDFium loading, document parsing, form setup, or rendering failed.
|
||||
#[error(transparent)]
|
||||
Pdfium(#[from] firecrawl_pdfium::Error),
|
||||
@@ -80,14 +89,15 @@ impl PdfiumRenderer {
|
||||
/// Loads PDFium using `firecrawl-pdfium`'s documented discovery chain.
|
||||
pub fn load() -> Result<Self, RenderError> {
|
||||
Ok(Self {
|
||||
pdfium: Pdfium::load()?,
|
||||
pdfium: Pdfium::load().map_err(|source| RenderError::PdfiumLoad { source })?,
|
||||
})
|
||||
}
|
||||
|
||||
/// Loads PDFium from an explicit native library path.
|
||||
pub fn load_from_path(path: impl AsRef<Path>) -> Result<Self, RenderError> {
|
||||
Ok(Self {
|
||||
pdfium: Pdfium::load_from_path(path)?,
|
||||
pdfium: Pdfium::load_from_path(path)
|
||||
.map_err(|source| RenderError::PdfiumLoad { source })?,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
+337
-45
@@ -11,7 +11,7 @@ use crate::text_quality::{
|
||||
analyze_text_quality, detect_encoding_issues, is_cid_garbage, is_garbage_text,
|
||||
};
|
||||
use crate::{
|
||||
MarkdownOptions, PageOcrReasons, PdfError, OCR_REASON_SUSPECTED_GARBLED_TEXT,
|
||||
MarkdownOptions, PageMarkdown, PageOcrReasons, PdfError, OCR_REASON_SUSPECTED_GARBLED_TEXT,
|
||||
OCR_REASON_VECTOR_TEXT,
|
||||
};
|
||||
|
||||
@@ -22,12 +22,15 @@ use super::fusion::{
|
||||
use super::oar::onnx_runtime_library_path;
|
||||
use super::pdfium::PdfiumTextPage;
|
||||
use super::{
|
||||
route_ocr_pages, run_ocr_pages, FusedPageMarkdown, HttpModelDownloadError, HttpModelDownloader,
|
||||
ModelAcquireError, ModelStore, ModelStoreError, OarOcrEngine, OarOcrError, OcrFusionError,
|
||||
OcrFusionOptions, OcrMode, OcrOptions, OcrRoutingError, OcrRun, OcrRunError, PdfiumRenderer,
|
||||
RenderError, RenderOptions, PP_OCR_V6_SMALL,
|
||||
route_ocr_pages, run_ocr_pages, FusedPageMarkdown, FusedPages, HttpModelDownloadError,
|
||||
HttpModelDownloader, ModelAcquireError, ModelStore, ModelStoreError, OarOcrEngine, OarOcrError,
|
||||
OcrEngine, OcrFusionError, OcrFusionOptions, OcrMode, OcrOptions, OcrRoutingError, OcrRun,
|
||||
OcrRunError, PageRenderer, PdfiumRenderer, RenderError, RenderOptions, PP_OCR_V6_SMALL,
|
||||
};
|
||||
|
||||
/// Bounds live rendered-page memory while preserving small OCR batches.
|
||||
const OCR_PAGE_CHUNK_SIZE: usize = 4;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
struct OcrEngineCacheKey {
|
||||
model_root: PathBuf,
|
||||
@@ -56,10 +59,13 @@ pub struct OcrPdfOptions {
|
||||
/// Markdown formatting shared by native and OCR assembly.
|
||||
pub markdown: MarkdownOptions,
|
||||
/// Optional 1-indexed page selection. `None` processes the full document.
|
||||
pub page_filter: Option<BTreeSet<u32>>,
|
||||
pub page_numbers: Option<BTreeSet<u32>>,
|
||||
/// Password for an encrypted PDF.
|
||||
pub password: Option<String>,
|
||||
/// Weak OCR threshold for recommending the hosted pipeline.
|
||||
///
|
||||
/// Pages with incomplete native coverage may also recommend the hosted
|
||||
/// pipeline when confident OCR only duplicates the retained fragment.
|
||||
pub hosted_recommendation_confidence: f32,
|
||||
}
|
||||
|
||||
@@ -69,7 +75,7 @@ impl Default for OcrPdfOptions {
|
||||
render: RenderOptions::default(),
|
||||
ocr: OcrOptions::default(),
|
||||
markdown: MarkdownOptions::default(),
|
||||
page_filter: None,
|
||||
page_numbers: None,
|
||||
password: None,
|
||||
hosted_recommendation_confidence: 0.5,
|
||||
}
|
||||
@@ -83,7 +89,7 @@ impl std::fmt::Debug for OcrPdfOptions {
|
||||
.field("render", &self.render)
|
||||
.field("ocr", &self.ocr)
|
||||
.field("markdown", &self.markdown)
|
||||
.field("page_filter", &self.page_filter)
|
||||
.field("page_numbers", &self.page_numbers)
|
||||
.field("password", &self.password.as_ref().map(|_| "[REDACTED]"))
|
||||
.field(
|
||||
"hosted_recommendation_confidence",
|
||||
@@ -99,6 +105,11 @@ impl OcrPdfOptions {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Creates options with selective OCR enabled for recommended pages.
|
||||
pub fn auto() -> Self {
|
||||
Self::default().mode(OcrMode::Auto)
|
||||
}
|
||||
|
||||
/// Replaces page rasterization settings.
|
||||
pub fn render(mut self, render: RenderOptions) -> Self {
|
||||
self.render = render;
|
||||
@@ -124,8 +135,8 @@ impl OcrPdfOptions {
|
||||
}
|
||||
|
||||
/// Restricts processing to 1-indexed pages in ascending order.
|
||||
pub fn pages(mut self, pages: impl IntoIterator<Item = u32>) -> Self {
|
||||
self.page_filter = Some(pages.into_iter().collect());
|
||||
pub fn page_numbers(mut self, pages: impl IntoIterator<Item = u32>) -> Self {
|
||||
self.page_numbers = Some(pages.into_iter().collect());
|
||||
self
|
||||
}
|
||||
|
||||
@@ -203,7 +214,7 @@ pub fn process_pdf_with_ocr_mem(
|
||||
});
|
||||
}
|
||||
if options
|
||||
.page_filter
|
||||
.page_numbers
|
||||
.as_ref()
|
||||
.is_some_and(|pages| pages.contains(&0))
|
||||
{
|
||||
@@ -212,7 +223,7 @@ pub fn process_pdf_with_ocr_mem(
|
||||
|
||||
let started = Instant::now();
|
||||
let selected_pages: Option<Vec<u32>> = options
|
||||
.page_filter
|
||||
.page_numbers
|
||||
.as_ref()
|
||||
.map(|pages| pages.iter().copied().collect());
|
||||
let selected_pages_zero_indexed: Option<Vec<u32>> = selected_pages
|
||||
@@ -327,12 +338,22 @@ pub fn process_pdf_with_ocr_mem(
|
||||
}
|
||||
}
|
||||
|
||||
let ocr_run = if routed.is_empty() {
|
||||
OcrRun {
|
||||
pages: Vec::new(),
|
||||
render_time_ms: 0,
|
||||
ocr_time_ms: 0,
|
||||
}
|
||||
let fusion_options = OcrFusionOptions::new()
|
||||
.markdown(page_markdown_options)
|
||||
.render_dpi(options.render.dpi)
|
||||
.hosted_recommendation_confidence(options.hosted_recommendation_confidence);
|
||||
let mut fused = if routed.is_empty() {
|
||||
fuse_ocr_pages_adaptive(
|
||||
&native.pages,
|
||||
&OcrRun {
|
||||
pages: Vec::new(),
|
||||
render_time_ms: 0,
|
||||
ocr_time_ms: 0,
|
||||
},
|
||||
page_count,
|
||||
&fusion_options,
|
||||
&native_candidates,
|
||||
)?
|
||||
} else {
|
||||
// Resolve the native renderer before any network request so a missing
|
||||
// PDFium installation cannot trigger a model download it cannot use.
|
||||
@@ -341,7 +362,7 @@ pub fn process_pdf_with_ocr_mem(
|
||||
None => PdfiumRenderer::load()?,
|
||||
};
|
||||
let engine = cached_ocr_engine(&options.ocr)?;
|
||||
run_ocr_pages(
|
||||
run_and_fuse_ocr_chunks(
|
||||
&renderer,
|
||||
engine.as_ref(),
|
||||
buffer,
|
||||
@@ -349,22 +370,14 @@ pub fn process_pdf_with_ocr_mem(
|
||||
options.password.as_deref(),
|
||||
&options.render,
|
||||
&options.ocr,
|
||||
&native.pages,
|
||||
page_count,
|
||||
&fusion_options,
|
||||
&native_candidates,
|
||||
)?
|
||||
};
|
||||
|
||||
let fusion_options = OcrFusionOptions::new()
|
||||
.markdown(page_markdown_options)
|
||||
.render_dpi(options.render.dpi)
|
||||
.hosted_recommendation_confidence(options.hosted_recommendation_confidence);
|
||||
let mut fused = fuse_ocr_pages_adaptive(
|
||||
&native.pages,
|
||||
&ocr_run,
|
||||
page_count,
|
||||
&fusion_options,
|
||||
&native_candidates,
|
||||
)?;
|
||||
for page in &mut fused.pages {
|
||||
if recovered_natively.contains(&page.page) {
|
||||
if recovered_natively.contains(&page.page_number) {
|
||||
page.provenance
|
||||
.warnings
|
||||
.push("recovered a credible positioned native text layer before OCR".to_string());
|
||||
@@ -374,14 +387,14 @@ pub fn process_pdf_with_ocr_mem(
|
||||
.pages
|
||||
.iter()
|
||||
.filter(|page| page.provenance.hosted_recommended)
|
||||
.map(|page| page.provenance.page)
|
||||
.map(|page| page.provenance.page_number)
|
||||
.collect();
|
||||
let markdown = assemble_document_markdown(&fused.pages, options.markdown.include_page_numbers);
|
||||
|
||||
let mut pages_with_tables = native.pages_with_tables;
|
||||
for page in &fused.pages {
|
||||
if markdown_has_table(&page.markdown) && !pages_with_tables.contains(&page.page) {
|
||||
pages_with_tables.push(page.page);
|
||||
if markdown_has_table(&page.markdown) && !pages_with_tables.contains(&page.page_number) {
|
||||
pages_with_tables.push(page.page_number);
|
||||
}
|
||||
}
|
||||
pages_with_tables.sort_unstable();
|
||||
@@ -403,6 +416,122 @@ pub fn process_pdf_with_ocr_mem(
|
||||
})
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn run_and_fuse_ocr_chunks<R, O>(
|
||||
renderer: &R,
|
||||
engine: &O,
|
||||
pdf_bytes: &[u8],
|
||||
routed_pages: &[u32],
|
||||
password: Option<&str>,
|
||||
render_options: &RenderOptions,
|
||||
ocr_options: &OcrOptions,
|
||||
native_pages: &[PageMarkdown],
|
||||
document_page_count: u32,
|
||||
fusion_options: &OcrFusionOptions,
|
||||
native_candidates: &BTreeMap<u32, NativeFallbackCandidate>,
|
||||
) -> Result<FusedPages, OcrPipelineError>
|
||||
where
|
||||
R: PageRenderer,
|
||||
O: OcrEngine,
|
||||
{
|
||||
let routed: BTreeSet<u32> = routed_pages.iter().copied().collect();
|
||||
let native_by_number: BTreeMap<u32, &PageMarkdown> = native_pages
|
||||
.iter()
|
||||
.map(|page| (page.page + 1, page))
|
||||
.collect();
|
||||
let mut pages_by_number = BTreeMap::new();
|
||||
let mut render_time_ms = 0u64;
|
||||
let mut ocr_time_ms = 0u64;
|
||||
|
||||
for chunk in routed_pages.chunks(OCR_PAGE_CHUNK_SIZE) {
|
||||
let native_chunk = chunk
|
||||
.iter()
|
||||
.map(|page_number| {
|
||||
clone_native_page(
|
||||
native_by_number
|
||||
.get(page_number)
|
||||
.copied()
|
||||
.expect("every routed page has a native page"),
|
||||
)
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
let run = run_ocr_pages(
|
||||
renderer,
|
||||
engine,
|
||||
pdf_bytes,
|
||||
chunk,
|
||||
password,
|
||||
render_options,
|
||||
ocr_options,
|
||||
)?;
|
||||
let fused = fuse_ocr_pages_adaptive(
|
||||
&native_chunk,
|
||||
&run,
|
||||
document_page_count,
|
||||
fusion_options,
|
||||
native_candidates,
|
||||
)?;
|
||||
render_time_ms = render_time_ms.saturating_add(fused.render_time_ms);
|
||||
ocr_time_ms = ocr_time_ms.saturating_add(fused.ocr_time_ms);
|
||||
for page in fused.pages {
|
||||
pages_by_number.insert(page.page_number, page);
|
||||
}
|
||||
// `run` and its rendered bitmaps are released before the next chunk.
|
||||
}
|
||||
|
||||
let native_only = select_native_pages(native_pages, |page| !routed.contains(&page));
|
||||
if !native_only.is_empty() {
|
||||
let fused = fuse_ocr_pages_adaptive(
|
||||
&native_only,
|
||||
&OcrRun {
|
||||
pages: Vec::new(),
|
||||
render_time_ms: 0,
|
||||
ocr_time_ms: 0,
|
||||
},
|
||||
document_page_count,
|
||||
fusion_options,
|
||||
native_candidates,
|
||||
)?;
|
||||
for page in fused.pages {
|
||||
pages_by_number.insert(page.page_number, page);
|
||||
}
|
||||
}
|
||||
|
||||
let pages = native_pages
|
||||
.iter()
|
||||
.map(|native| {
|
||||
pages_by_number
|
||||
.remove(&(native.page + 1))
|
||||
.expect("every native page is fused exactly once")
|
||||
})
|
||||
.collect();
|
||||
Ok(FusedPages {
|
||||
pages,
|
||||
render_time_ms,
|
||||
ocr_time_ms,
|
||||
})
|
||||
}
|
||||
|
||||
fn select_native_pages(
|
||||
native_pages: &[PageMarkdown],
|
||||
include: impl Fn(u32) -> bool,
|
||||
) -> Vec<PageMarkdown> {
|
||||
native_pages
|
||||
.iter()
|
||||
.filter(|page| include(page.page + 1))
|
||||
.map(clone_native_page)
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn clone_native_page(page: &PageMarkdown) -> PageMarkdown {
|
||||
PageMarkdown {
|
||||
page: page.page,
|
||||
markdown: page.markdown.clone(),
|
||||
needs_ocr: page.needs_ocr,
|
||||
ocr_reason: page.ocr_reason.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
fn cached_ocr_engine(options: &OcrOptions) -> Result<Arc<OarOcrEngine>, OcrPipelineError> {
|
||||
let store = ModelStore::from_options(options)?;
|
||||
let key = OcrEngineCacheKey {
|
||||
@@ -677,6 +806,7 @@ fn markdown_has_table(markdown: &str) -> bool {
|
||||
fn remove_duplicate_table_lines(markdown: &str) -> String {
|
||||
let mut output = String::new();
|
||||
let mut adjacent_table_row = None;
|
||||
let mut wide_table_rows = BTreeSet::new();
|
||||
for line in markdown.lines() {
|
||||
let trimmed = line.trim();
|
||||
let is_table_line = trimmed.starts_with('|') && trimmed.ends_with('|');
|
||||
@@ -684,19 +814,28 @@ fn remove_duplicate_table_lines(markdown: &str) -> String {
|
||||
if !trimmed.contains("|---") && trimmed.matches('|').count() >= 4 {
|
||||
let canonical = canonical_table_text(trimmed);
|
||||
if !canonical.is_empty() {
|
||||
if table_cell_count(trimmed) >= 8 {
|
||||
wide_table_rows.insert(canonical.clone());
|
||||
}
|
||||
adjacent_table_row = Some(canonical);
|
||||
}
|
||||
}
|
||||
} else if trimmed.is_empty() {
|
||||
// Keep adjacency across the blank line emitted after a table.
|
||||
} else {
|
||||
let duplicate = adjacent_table_row
|
||||
.as_ref()
|
||||
.is_some_and(|table_row| *table_row == canonical_table_text(trimmed));
|
||||
let canonical = canonical_table_text(trimmed);
|
||||
let duplicate = wide_table_rows.contains(&canonical)
|
||||
|| adjacent_table_row
|
||||
.as_ref()
|
||||
.is_some_and(|table_row| *table_row == canonical);
|
||||
adjacent_table_row = None;
|
||||
if duplicate {
|
||||
continue;
|
||||
}
|
||||
// Wide rows are only candidates inside the duplicate block that
|
||||
// immediately follows a table. Once unrelated prose begins, the
|
||||
// same text may be a legitimate later reference.
|
||||
wide_table_rows.clear();
|
||||
}
|
||||
output.push_str(line);
|
||||
output.push('\n');
|
||||
@@ -707,6 +846,13 @@ fn remove_duplicate_table_lines(markdown: &str) -> String {
|
||||
output
|
||||
}
|
||||
|
||||
fn table_cell_count(row: &str) -> usize {
|
||||
row.trim_matches('|')
|
||||
.split('|')
|
||||
.filter(|cell| !cell.trim().is_empty())
|
||||
.count()
|
||||
}
|
||||
|
||||
fn canonical_table_text(text: &str) -> String {
|
||||
text.replace('|', " ")
|
||||
.split_whitespace()
|
||||
@@ -721,7 +867,7 @@ fn assemble_document_markdown(pages: &[FusedPageMarkdown], include_page_numbers:
|
||||
document.push_str("\n\n");
|
||||
}
|
||||
if include_page_numbers {
|
||||
document.push_str(&format!("<!-- Page {} -->\n\n", page.page));
|
||||
document.push_str(&format!("<!-- Page {} -->\n\n", page.page_number));
|
||||
}
|
||||
document.push_str(page.markdown.trim());
|
||||
}
|
||||
@@ -781,6 +927,84 @@ pub enum OcrPipelineError {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
struct TrackingRenderer {
|
||||
batches: Mutex<Vec<Vec<u32>>>,
|
||||
}
|
||||
|
||||
impl PageRenderer for TrackingRenderer {
|
||||
type Error = std::convert::Infallible;
|
||||
|
||||
fn render_pages(
|
||||
&self,
|
||||
_pdf_bytes: &[u8],
|
||||
pages: &[u32],
|
||||
_password: Option<&str>,
|
||||
_options: &RenderOptions,
|
||||
) -> Result<Vec<super::super::RenderedPage>, Self::Error> {
|
||||
self.batches.lock().unwrap().push(pages.to_vec());
|
||||
Ok(pages
|
||||
.iter()
|
||||
.map(|page| {
|
||||
let transform = super::super::PageTransform::from_corners(
|
||||
1,
|
||||
1,
|
||||
(0.0, 1.0),
|
||||
(1.0, 1.0),
|
||||
(0.0, 0.0),
|
||||
)
|
||||
.unwrap();
|
||||
super::super::RenderedPage::new(
|
||||
*page,
|
||||
1.0,
|
||||
1.0,
|
||||
1,
|
||||
1,
|
||||
3,
|
||||
super::super::RenderPixelFormat::Rgb8,
|
||||
vec![255; 3],
|
||||
transform,
|
||||
)
|
||||
.unwrap()
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
}
|
||||
|
||||
struct TrackingEngine {
|
||||
batches: Mutex<Vec<Vec<u32>>>,
|
||||
model: super::super::ModelIdentity,
|
||||
}
|
||||
|
||||
impl OcrEngine for TrackingEngine {
|
||||
type Error = std::convert::Infallible;
|
||||
|
||||
fn model(&self) -> &super::super::ModelIdentity {
|
||||
&self.model
|
||||
}
|
||||
|
||||
fn recognize(
|
||||
&self,
|
||||
pages: &[super::super::RenderedPage],
|
||||
_options: &OcrOptions,
|
||||
) -> Result<Vec<super::super::OcrPage>, Self::Error> {
|
||||
self.batches
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push(pages.iter().map(super::super::RenderedPage::page).collect());
|
||||
Ok(pages
|
||||
.iter()
|
||||
.map(|page| super::super::OcrPage {
|
||||
page_number: page.page(),
|
||||
spans: Vec::new(),
|
||||
mean_confidence: None,
|
||||
model: self.model.clone(),
|
||||
processing_time_ms: 0,
|
||||
warnings: Vec::new(),
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
}
|
||||
|
||||
fn recovery_item(text: &str, x: f32, y: f32, width: f32, height: f32) -> crate::TextItem {
|
||||
crate::TextItem {
|
||||
text: text.to_string(),
|
||||
@@ -815,6 +1039,51 @@ mod tests {
|
||||
assert_eq!(before, after);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn high_level_ocr_bounds_rendering_and_inference_batches() {
|
||||
let renderer = TrackingRenderer {
|
||||
batches: Mutex::new(Vec::new()),
|
||||
};
|
||||
let engine = TrackingEngine {
|
||||
batches: Mutex::new(Vec::new()),
|
||||
model: super::super::ModelIdentity::new("test-ocr", "v1"),
|
||||
};
|
||||
let native_pages = (0..10)
|
||||
.map(|page| PageMarkdown {
|
||||
page,
|
||||
markdown: String::new(),
|
||||
needs_ocr: true,
|
||||
ocr_reason: Some(crate::OCR_REASON_SCANNED.to_string()),
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
let routed_pages = (1..=10).collect::<Vec<_>>();
|
||||
let ocr_options = OcrOptions::new().mode(OcrMode::Force);
|
||||
|
||||
let fused = run_and_fuse_ocr_chunks(
|
||||
&renderer,
|
||||
&engine,
|
||||
b"test",
|
||||
&routed_pages,
|
||||
None,
|
||||
&RenderOptions::new(),
|
||||
&ocr_options,
|
||||
&native_pages,
|
||||
10,
|
||||
&OcrFusionOptions::new(),
|
||||
&BTreeMap::new(),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let expected = vec![vec![1, 2, 3, 4], vec![5, 6, 7, 8], vec![9, 10]];
|
||||
assert_eq!(*renderer.batches.lock().unwrap(), expected);
|
||||
assert_eq!(*engine.batches.lock().unwrap(), expected);
|
||||
assert_eq!(fused.pages.len(), 10);
|
||||
assert!(fused
|
||||
.pages
|
||||
.iter()
|
||||
.all(|page| page.provenance.hosted_recommended));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn native_recovery_is_limited_to_recoverable_routing_reasons() {
|
||||
let routed = [1, 2, 3, 4];
|
||||
@@ -908,6 +1177,23 @@ mod tests {
|
||||
assert_eq!(remove_duplicate_table_lines(markdown), markdown);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recovered_markdown_keeps_wide_table_text_after_unrelated_prose() {
|
||||
let markdown = "|Date|A|B|C|D|E|F|G|\n|---|---|---|---|---|---|---|---|\n|April 1|1|2|3|4|5|6|7|\n\nSummary follows.\n\nApril 1 1 2 3 4 5 6 7\n";
|
||||
|
||||
assert_eq!(remove_duplicate_table_lines(markdown), markdown);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recovered_markdown_drops_contiguous_duplicate_block_of_wide_table_rows() {
|
||||
let markdown = "|Date|A|B|C|D|E|F|G|\n|---|---|---|---|---|---|---|---|\n|April 1|1|2|3|4|5|6|7|\n|April 2|8|9|10|11|12|13|14|\n\nApril 1 1 2 3 4 5 6 7\nApril 2 8 9 10 11 12 13 14\n\nSummary follows.\n";
|
||||
|
||||
assert_eq!(
|
||||
remove_duplicate_table_lines(markdown),
|
||||
"|Date|A|B|C|D|E|F|G|\n|---|---|---|---|---|---|---|---|\n|April 1|1|2|3|4|5|6|7|\n|April 2|8|9|10|11|12|13|14|\n\n\nSummary follows.\n"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extractor_candidates_require_clean_partial_content_reasons() {
|
||||
let reasons = [
|
||||
@@ -969,13 +1255,18 @@ mod tests {
|
||||
let bytes = std::fs::read("tests/fixtures/thermo-freon12.pdf").unwrap();
|
||||
let mut markdown = MarkdownOptions::default();
|
||||
markdown.include_page_numbers = true;
|
||||
let result =
|
||||
process_pdf_with_ocr_mem(&bytes, OcrPdfOptions::new().pages([2]).markdown(markdown))
|
||||
.unwrap();
|
||||
let result = process_pdf_with_ocr_mem(
|
||||
&bytes,
|
||||
OcrPdfOptions::new().page_numbers([2]).markdown(markdown),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(result.pages.len(), 1);
|
||||
assert_eq!(result.pages[0].page, 2);
|
||||
assert_eq!(result.pages[0].page, result.pages[0].provenance.page);
|
||||
assert_eq!(result.pages[0].page_number, 2);
|
||||
assert_eq!(
|
||||
result.pages[0].page_number,
|
||||
result.pages[0].provenance.page_number
|
||||
);
|
||||
assert!(result.markdown.starts_with("<!-- Page 2 -->"));
|
||||
}
|
||||
|
||||
@@ -1022,7 +1313,8 @@ mod tests {
|
||||
#[test]
|
||||
fn rejects_out_of_range_selection_even_with_ocr_off() {
|
||||
let bytes = std::fs::read("tests/fixtures/thermo-freon12.pdf").unwrap();
|
||||
let error = process_pdf_with_ocr_mem(&bytes, OcrPdfOptions::new().pages([4])).unwrap_err();
|
||||
let error =
|
||||
process_pdf_with_ocr_mem(&bytes, OcrPdfOptions::new().page_numbers([4])).unwrap_err();
|
||||
assert!(matches!(
|
||||
error,
|
||||
OcrPipelineError::InvalidSelectedPage { page: 4 }
|
||||
|
||||
@@ -106,7 +106,11 @@ where
|
||||
source: Box::new(source),
|
||||
})?;
|
||||
let ocr_time_ms = elapsed_ms(ocr_started);
|
||||
validate_page_order("OCR engine", pages, recognized.iter().map(|page| page.page))?;
|
||||
validate_page_order(
|
||||
"OCR engine",
|
||||
pages,
|
||||
recognized.iter().map(|page| page.page_number),
|
||||
)?;
|
||||
|
||||
Ok(OcrRun {
|
||||
pages: rendered
|
||||
@@ -268,7 +272,7 @@ mod tests {
|
||||
Ok(pages
|
||||
.iter()
|
||||
.map(|page| OcrPage {
|
||||
page: page.page(),
|
||||
page_number: page.page(),
|
||||
spans: vec![OcrSpan {
|
||||
text: format!("page {}", page.page()),
|
||||
polygon: ImageQuad::new([
|
||||
|
||||
@@ -5,9 +5,7 @@ use pdf_inspector::vision::{PdfiumRenderer, RenderError, RenderOptions, RenderPi
|
||||
fn load_renderer() -> Option<PdfiumRenderer> {
|
||||
match PdfiumRenderer::load() {
|
||||
Ok(renderer) => Some(renderer),
|
||||
Err(RenderError::Pdfium(firecrawl_pdfium::Error::Load(
|
||||
firecrawl_pdfium::LoadError::LibraryNotFound { .. },
|
||||
))) => {
|
||||
Err(RenderError::PdfiumLoad { .. }) => {
|
||||
eprintln!("skipping PDFium runtime test because no native library is installed");
|
||||
None
|
||||
}
|
||||
|
||||
+2
-4
@@ -20,9 +20,7 @@ const EXPECTED_TEXT_ENV: &str = "PDF_INSPECTOR_OCR_TEST_EXPECTED";
|
||||
fn load_renderer() -> Option<PdfiumRenderer> {
|
||||
match PdfiumRenderer::load() {
|
||||
Ok(renderer) => Some(renderer),
|
||||
Err(RenderError::Pdfium(firecrawl_pdfium::Error::Load(
|
||||
firecrawl_pdfium::LoadError::LibraryNotFound { .. },
|
||||
))) => {
|
||||
Err(RenderError::PdfiumLoad { .. }) => {
|
||||
eprintln!("skipping OCR runtime test because no native PDFium library is installed");
|
||||
None
|
||||
}
|
||||
@@ -208,7 +206,7 @@ fn recognize(
|
||||
|
||||
fn assert_usable_result(results: &[pdf_inspector::vision::OcrPage]) {
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0].page, 1);
|
||||
assert_eq!(results[0].page_number, 1);
|
||||
assert_eq!(results[0].model.name, PP_OCR_V6_SMALL.id);
|
||||
assert_eq!(results[0].model.revision, PP_OCR_V6_SMALL.revision);
|
||||
assert!(!results[0].spans.is_empty());
|
||||
|
||||
Reference in New Issue
Block a user