Compare commits

...
15 changed files with 4464 additions and 3 deletions
+24
View File
@@ -49,6 +49,25 @@ ttf-parser = "0.25"
lopdf = { version = "0.42.0", features = ["rayon"] }
rayon = "1.10"
env_logger = "0.11"
# Optional native page rendering for OCR pipelines. PDFium is loaded at
# runtime, so enabling this feature does not link or download a native library.
firecrawl-pdfium = { version = "0.1.0", optional = true }
# Small support crates used only by the opt-in model cache. Model files remain
# external and are never embedded in pdf-inspector artifacts.
dirs = { version = "6.0", optional = true }
fs2 = { version = "0.4", optional = true }
sha2 = { version = "0.11", optional = true }
# Optional CPU OCR backend. Models and ONNX Runtime stay external: the latter
# is loaded dynamically from ORT_DYLIB_PATH or the platform library search path.
image = { version = "0.25.6", default-features = false, optional = true }
oar-ocr = { version = "0.9.1", default-features = false, features = ["simd"], optional = true }
ort = { version = "=2.0.0-rc.13", default-features = false, features = ["load-dynamic"], optional = true }
# HTTPS-only streaming downloader for pinned model artifacts. Kept separate
# from model-cache so offline and package-managed deployments avoid HTTP/TLS.
ureq = { version = "3.4", default-features = false, features = ["rustls", "platform-verifier"], optional = true }
[target.'cfg(all(windows, not(target_arch = "wasm32")))'.dependencies]
windows-sys = { version = "0.61", features = ["Win32_Storage_FileSystem"], optional = true }
# Browser builds use JavaScript randomness for encrypted PDFs and embed the
# bundled CMaps because there is no filesystem at runtime.
@@ -62,6 +81,11 @@ tempfile = "3.3"
[features]
default = []
python = ["pyo3"]
vision = []
model-cache = ["vision", "dep:dirs", "dep:fs2", "dep:sha2", "dep:windows-sys"]
model-download = ["model-cache", "dep:ureq"]
ocr-oar = ["model-cache", "dep:image", "dep:oar-ocr", "dep:ort"]
render-pdfium = ["vision", "dep:firecrawl-pdfium"]
[[bin]]
name = "pdf2md"
+224 -1
View File
@@ -1,6 +1,6 @@
# pdf-inspector
Fast PDF classification and text extraction. Detects whether a PDF is text-based or scanned, extracts text with position awareness, and converts to clean Markdown — all without OCR. Pure Rust, no ML models, no external services; the only PDF dependency is [lopdf](https://crates.io/crates/lopdf). Also available for [Python](https://pypi.org/project/pdf-inspector/) and [Node.js](https://www.npmjs.com/package/@firecrawl/pdf-inspector).
Fast PDF classification and text extraction. Detects whether a PDF is text-based or scanned, extracts text with position awareness, and converts to clean Markdown — all without OCR. The default build is pure Rust, has no ML models or external services, and uses [lopdf](https://crates.io/crates/lopdf) for PDF parsing. Also available for [Python](https://pypi.org/project/pdf-inspector/) and [Node.js](https://www.npmjs.com/package/@firecrawl/pdf-inspector/).
Built by [Firecrawl](https://firecrawl.dev) to handle text-based PDFs locally in under 200ms, skipping expensive OCR services for the ~54% of PDFs that don't need them.
@@ -117,6 +117,229 @@ let bytes = std::fs::read("document.pdf")?;
let result = process_pdf_mem(&bytes)?;
```
### Vision extension contracts
The native-only `vision` feature exposes the stable seam used by OCR
integrations without selecting or embedding an inference runtime. The
separate `model-cache` feature adds pinned artifact management:
- `PageRenderer`, `OcrEngine`, and `LayoutEngine` traits;
- renderer-neutral owned page buffers and affine pixel↔PDF transforms;
- `OcrOptions` and opt-in `Off`/`Auto`/`Force` routing modes;
- positioned OCR/layout results and per-page provenance types; and
- a versioned PP-OCRv6 Small manifest with checksum-verified, locked, atomic
model-cache installation and explicit offline-directory overrides.
```toml
[dependencies]
pdf-inspector = { version = "1", features = ["vision", "model-cache"] }
```
The OCR contracts preserve existing behavior by default: OCR is `Off`, learned
layout is disabled, and model resolution is never reached. `ModelStore` itself
does not access the network. The optional `model-download` feature provides an
HTTPS downloader that streams pinned artifacts into the checksum-verified
cache only after routing has selected OCR work. Offline consumers set an
explicit model directory and `ModelDownloadPolicy::Offline`. Renderer-only
consumers do not enable `model-cache` or `model-download` and therefore do not
compile their filesystem, hashing, or HTTP dependencies.
```rust
use pdf_inspector::vision::{
ModelDownloadPolicy, ModelStore, OcrMode, OcrOptions, PP_OCR_V6_SMALL,
};
let ocr = OcrOptions::new()
.mode(OcrMode::Auto)
.model_directory("/opt/firecrawl/models/pp-ocrv6-small")
.model_downloads(ModelDownloadPolicy::Offline);
// Verifies exact sizes and SHA-256 digests before an engine opens the files.
let models = ModelStore::from_options(&ocr)?.resolve(&PP_OCR_V6_SMALL)?;
println!("using {} at {}", models.manifest_id(), models.revision());
```
### Optional native page rendering
The `render-pdfium` feature adds a native-only page renderer backed by
[`firecrawl-pdfium`](https://crates.io/crates/firecrawl-pdfium). It is the
rendering boundary for OCR pipelines; enabling it does not include an OCR
model or change the existing extraction functions. It implies `vision`,
and `PdfiumRenderer` implements the renderer-neutral `PageRenderer` trait.
```toml
[dependencies]
pdf-inspector = { version = "1", features = ["render-pdfium"] }
```
PDFium is loaded at runtime. Set `PDFIUM_LIB_PATH`, place its shared library
next to the executable, or use another discovery route supported by
`firecrawl-pdfium`.
```rust
use pdf_inspector::vision::{PdfiumRenderer, RenderOptions};
let renderer = PdfiumRenderer::load()?;
let bytes = std::fs::read("document.pdf")?;
let pages = renderer.render_pages(
&bytes,
&[1, 3], // 1-indexed, matching pages_needing_ocr
None, // optional PDF password
&RenderOptions::new().dpi(150.0),
)?;
for page in pages {
// Owned RGB pixels can leave the PDFium critical section and be sent to
// an OCR worker. OCR pixel boxes can be mapped back to PDF coordinates.
let rect = page.pixel_rect_to_pdf_rect(20.0, 30.0, 100.0, 24.0);
println!("page {}: {}x{}, rect={rect:?}", page.page(), page.width(), page.height());
}
```
Browser WASM remains on the default text-only path and does not expose native
PDFium rendering.
### Optional OCR engine
The native-only `ocr-oar` feature adds a CPU PP-OCRv6 Small implementation of
`OcrEngine` backed by OAR and ONNX Runtime. It implies `model-cache`, but does
not enable model auto-download, ONNX Runtime download, or PDF rendering. Model
files remain external, must match the pinned manifest, and are opened only
after `ModelStore` verifies their exact size and SHA-256 digest. Install an
ONNX Runtime shared library separately and set `ORT_DYLIB_PATH` when it is not
available through the platform library search path. The feature currently
requires Rust 1.95 or newer, matching OAR 0.9.1's MSRV.
```toml
[dependencies]
pdf-inspector = { version = "1", features = ["ocr-oar", "render-pdfium"] }
```
Direct engine invocation is intentionally separate from extraction routing and
native/OCR fusion:
```rust
use pdf_inspector::vision::{
ModelDownloadPolicy, ModelStore, OarOcrEngine, OcrEngine, OcrMode,
OcrOptions, PdfiumRenderer, RenderOptions, PP_OCR_V6_SMALL,
};
let options = OcrOptions::new()
.mode(OcrMode::Force)
.minimum_confidence(0.45)
.model_directory("/opt/firecrawl/models/pp-ocrv6-small")
.model_downloads(ModelDownloadPolicy::Offline);
let models = ModelStore::from_options(&options)?.resolve(&PP_OCR_V6_SMALL)?;
let engine = OarOcrEngine::from_models(&models)?;
let renderer = PdfiumRenderer::load()?;
let bytes = std::fs::read("scan.pdf")?;
let pages = renderer.render_pages(&bytes, &[1], None, &RenderOptions::new())?;
let ocr_pages = engine.recognize(&pages, &options)?;
for span in &ocr_pages[0].spans {
println!("{:.3}: {}", span.confidence, span.text);
}
```
The engine accepts renderer-neutral RGB, RGBA, and grayscale pages, preserves
OAR's positioned quadrilaterals in bitmap coordinates, filters spans using
`minimum_confidence`, and records the pinned model revision in every `OcrPage`.
`OcrMode::Off` is rejected at the engine boundary so default options cannot run
inference accidentally.
### Selective routing and lazy model acquisition
`route_ocr_pages` applies the existing detector/text-quality recommendations to
the configured mode. `Auto` processes only recommended pages, `Force` processes
all pages (or an explicit page selection), and `Off` always returns an empty
route. `run_ocr_pages` renders only that route, checks that both dependencies
preserve its order, and retains each bitmap's PDF transform for fusion.
```toml
[dependencies]
pdf-inspector = { version = "1", features = [
"render-pdfium",
"ocr-oar",
"model-download",
] }
```
```rust
use pdf_inspector::vision::{
route_ocr_pages, run_ocr_pages, HttpModelDownloader, ModelStore,
OarOcrEngine, OcrMode, OcrOptions, PdfiumRenderer, RenderOptions,
PP_OCR_V6_SMALL,
};
let bytes = std::fs::read("scan.pdf")?;
let extraction = pdf_inspector::extract_pages_markdown_mem(&bytes, None)?;
let options = OcrOptions::new().mode(OcrMode::Auto);
let routed = route_ocr_pages(
options.mode,
extraction.pages.len() as u32,
&extraction.pages_needing_ocr,
None,
)?;
if !routed.is_empty() {
// No HTTP request or model initialization occurs before this point.
let store = ModelStore::from_options(&options)?;
let models = store.resolve_or_download(
&PP_OCR_V6_SMALL,
options.model_downloads,
&HttpModelDownloader::default(),
)?;
let run = run_ocr_pages(
&PdfiumRenderer::load()?,
&OarOcrEngine::from_models(&models)?,
&bytes,
&routed,
None,
&RenderOptions::new(),
&options,
)?;
println!("OCR processed {} pages", run.pages.len());
}
```
The downloader accepts HTTPS only, checks a declared content length, caps the
response stream to the pinned size plus one byte, and delegates final size and
SHA-256 verification to `ModelStore`. The store serializes installation across
processes and publishes completed artifacts atomically. Warm caches make no
network calls; offline mode and explicit model directories never download.
### OCR Markdown assembly and native fusion
`fuse_ocr_pages` maps OCR polygons back into PDF coordinates and sends the
result through pdf-inspector's existing deterministic reading-order, table,
and Markdown pipeline. Pages whose native extraction was rejected use OCR
output. When `Force` runs on a clean native page, normalized duplicate OCR
blocks are removed and only additional image-backed text is retained.
```rust
use pdf_inspector::vision::{fuse_ocr_pages, OcrFusionOptions};
let fused = fuse_ocr_pages(
&extraction.pages,
&run,
extraction.pages.len() as u32,
&OcrFusionOptions::new().render_dpi(150.0),
)?;
for page in &fused.pages {
println!("{}", page.markdown);
if page.provenance.hosted_recommended {
eprintln!("page {} needs the hosted document pipeline", page.page + 1);
}
}
```
Each page carries `Native`, `Ocr`, or `Fused` provenance, the exact OCR model
revision, accepted-page confidence, local stage timings, and non-fatal
warnings. A page that required OCR recommends the hosted pipeline when local
OCR is missing, empty, or below the configurable page-confidence threshold.
This keeps the lightweight path explicit about cases it cannot finish well.
Extract per-page Markdown (one string per page, plus document-wide layout
metadata):
+1
View File
@@ -43,6 +43,7 @@ mod text_quality;
pub mod text_utils;
pub mod tounicode;
pub mod types;
pub mod vision;
pub use detector::{
detect_pdf_type, detect_pdf_type_mem, detect_pdf_type_mem_with_config,
+2 -2
View File
@@ -604,7 +604,7 @@ mod tests {
let items: Vec<(usize, &TextItem)> = vec![];
assert_eq!(
find_column_boundaries(&items, TableDetectionMode::SmallFont),
vec![]
Vec::<f32>::new()
);
}
@@ -661,7 +661,7 @@ mod tests {
#[test]
fn test_find_row_boundaries_empty() {
let items: Vec<(usize, &TextItem)> = vec![];
assert_eq!(find_row_boundaries(&items), vec![]);
assert_eq!(find_row_boundaries(&items), Vec::<f32>::new());
}
#[test]
+414
View File
@@ -0,0 +1,414 @@
//! Public contracts between rendering, OCR, layout, and orchestration.
use std::error::Error;
use std::path::PathBuf;
use super::{RenderOptions, RenderedPage};
/// Selects when OCR may run.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
#[non_exhaustive]
pub enum OcrMode {
/// Never run OCR. This is the default and preserves existing behavior.
#[default]
Off,
/// Run OCR only on pages selected by pdf-inspector's OCR routing signals.
Auto,
/// Run OCR on every selected page, including pages with native text.
Force,
}
/// Resource/quality profile for the OCR engine.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
#[non_exhaustive]
pub enum OcrProfile {
/// Lowest latency and memory footprint.
Edge,
/// OCR-oriented balance of quality and CPU cost.
#[default]
Balanced,
/// Highest quality within the lightweight model family.
Quality,
}
/// Controls whether missing model artifacts may be fetched.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
#[non_exhaustive]
pub enum ModelDownloadPolicy {
/// Fetch a pinned artifact only after OCR has actually been selected.
#[default]
IfMissing,
/// Never access the network; require an override or a warm model cache.
Offline,
}
/// OCR engine configuration independent of a particular runtime.
#[derive(Debug, Clone, PartialEq)]
pub struct OcrOptions {
/// Page-level routing behavior.
pub mode: OcrMode,
/// Local quality/resource profile.
pub profile: OcrProfile,
/// Drop recognition spans below this confidence threshold.
pub minimum_confidence: f32,
/// Optional language hints understood by the selected engine.
pub languages: Vec<String>,
/// Optional directory containing an offline model set.
pub model_directory: Option<PathBuf>,
/// Whether a missing pinned artifact may be downloaded.
pub model_downloads: ModelDownloadPolicy,
}
impl Default for OcrOptions {
fn default() -> Self {
Self {
mode: OcrMode::Off,
profile: OcrProfile::Balanced,
minimum_confidence: 0.0,
languages: Vec::new(),
model_directory: None,
model_downloads: ModelDownloadPolicy::IfMissing,
}
}
}
impl OcrOptions {
/// Creates OCR options with OCR disabled.
pub fn new() -> Self {
Self::default()
}
/// Sets page-level OCR routing.
pub fn mode(mut self, mode: OcrMode) -> Self {
self.mode = mode;
self
}
/// Sets the local resource/quality profile.
pub fn profile(mut self, profile: OcrProfile) -> Self {
self.profile = profile;
self
}
/// Sets the minimum accepted recognition confidence.
pub fn minimum_confidence(mut self, minimum_confidence: f32) -> Self {
self.minimum_confidence = minimum_confidence;
self
}
/// Replaces the language hints passed to the OCR engine.
pub fn languages(mut self, languages: impl IntoIterator<Item = impl Into<String>>) -> Self {
self.languages = languages.into_iter().map(Into::into).collect();
self
}
/// Uses an explicit model directory, suitable for offline packaging.
pub fn model_directory(mut self, directory: impl Into<PathBuf>) -> Self {
self.model_directory = Some(directory.into());
self
}
/// Sets the missing-model download policy.
pub fn model_downloads(mut self, policy: ModelDownloadPolicy) -> Self {
self.model_downloads = policy;
self
}
}
/// Configuration for an optional learned layout engine.
///
/// Layout inference is disabled by default. Existing deterministic layout,
/// table, and Markdown logic remains the assembly path when this is disabled.
#[derive(Debug, Clone, PartialEq)]
pub struct LayoutOptions {
/// Whether the learned layout extension may run.
pub enabled: bool,
/// Drop layout regions below this confidence threshold.
pub minimum_confidence: f32,
/// Optional directory containing an offline layout model set.
pub model_directory: Option<PathBuf>,
}
impl Default for LayoutOptions {
fn default() -> Self {
Self {
enabled: false,
minimum_confidence: 0.0,
model_directory: None,
}
}
}
impl LayoutOptions {
/// Creates layout options with learned layout disabled.
pub fn new() -> Self {
Self::default()
}
/// Enables or disables learned layout inference.
pub fn enabled(mut self, enabled: bool) -> Self {
self.enabled = enabled;
self
}
/// Sets the minimum accepted region confidence.
pub fn minimum_confidence(mut self, minimum_confidence: f32) -> Self {
self.minimum_confidence = minimum_confidence;
self
}
/// Uses an explicit layout model directory.
pub fn model_directory(mut self, directory: impl Into<PathBuf>) -> Self {
self.model_directory = Some(directory.into());
self
}
}
/// A point in bitmap space, measured from the top-left in pixels.
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub struct ImagePoint {
/// Horizontal pixel coordinate.
pub x: f32,
/// Vertical pixel coordinate, increasing downward.
pub y: f32,
}
impl ImagePoint {
/// Creates a bitmap-space point.
pub fn new(x: f32, y: f32) -> Self {
Self { x, y }
}
}
/// Four-point polygon in bitmap coordinates.
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub struct ImageQuad {
/// Polygon points in engine-provided order.
pub points: [ImagePoint; 4],
}
impl ImageQuad {
/// Creates a four-point bitmap polygon.
pub fn new(points: [ImagePoint; 4]) -> Self {
Self { points }
}
}
/// Stable identity for an inference model used in output provenance.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ModelIdentity {
/// Model family/name, for example `pp-ocrv6-small`.
pub name: String,
/// Immutable model or artifact-set revision.
pub revision: String,
}
impl ModelIdentity {
/// Creates a model identity.
pub fn new(name: impl Into<String>, revision: impl Into<String>) -> Self {
Self {
name: name.into(),
revision: revision.into(),
}
}
}
/// One positioned OCR recognition result in bitmap coordinates.
#[derive(Debug, Clone, PartialEq)]
pub struct OcrSpan {
/// Recognized text.
pub text: String,
/// Detection polygon in the original rendered page's pixel space.
pub polygon: ImageQuad,
/// Recognition confidence in the inclusive range 01.
pub confidence: f32,
/// Optional text-line orientation in clockwise degrees.
pub orientation_degrees: Option<f32>,
}
/// OCR output for one 1-indexed page.
#[derive(Debug, Clone, PartialEq)]
pub struct OcrPage {
/// 1-indexed PDF page number.
pub page: u32,
/// Positioned recognition spans.
pub spans: Vec<OcrSpan>,
/// Mean confidence across accepted spans, when available.
pub mean_confidence: Option<f32>,
/// Exact model identity used for this result.
pub model: ModelIdentity,
/// OCR wall time for this page.
pub processing_time_ms: u64,
/// Non-fatal engine warnings.
pub warnings: Vec<String>,
}
/// Normalized semantic class emitted by a learned layout engine.
#[derive(Debug, Clone, PartialEq, Eq)]
#[non_exhaustive]
pub enum LayoutRegionKind {
/// Body or other prose text.
Text,
/// Document heading or title.
Heading,
/// Table region.
Table,
/// Figure/image region.
Figure,
/// Figure or table caption.
Caption,
/// Header/footer/page furniture.
Furniture,
/// Model-specific class retained without changing the common taxonomy.
Other(String),
}
/// One learned layout region in bitmap coordinates.
#[derive(Debug, Clone, PartialEq)]
pub struct LayoutRegion {
/// Normalized semantic class.
pub kind: LayoutRegionKind,
/// Region polygon in the original rendered page's pixel space.
pub polygon: ImageQuad,
/// Model confidence in the inclusive range 01.
pub confidence: f32,
/// Optional model-provided reading-order position.
pub reading_order: Option<u32>,
}
/// Learned layout output for one 1-indexed page.
#[derive(Debug, Clone, PartialEq)]
pub struct LayoutPage {
/// 1-indexed PDF page number.
pub page: u32,
/// Semantic regions.
pub regions: Vec<LayoutRegion>,
/// Exact model identity used for this result.
pub model: ModelIdentity,
/// Layout inference wall time for this page.
pub processing_time_ms: u64,
/// Non-fatal engine warnings.
pub warnings: Vec<String>,
}
/// How final page content was sourced.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[non_exhaustive]
pub enum PageContentSource {
/// Trusted native PDF text only.
Native,
/// OCR output only.
Ocr,
/// Native and OCR spans were fused.
Fused,
}
/// Per-page local processing timings.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct VisionTimings {
/// Rasterization wall time.
pub render_ms: u64,
/// OCR wall time.
pub ocr_ms: u64,
/// Optional learned layout wall time.
pub layout_ms: u64,
/// Native/OCR fusion and assembly wall time.
pub assembly_ms: u64,
}
/// Source and model metadata retained for one processed page.
#[derive(Debug, Clone, PartialEq)]
pub struct PageProvenance {
/// 1-indexed PDF page number.
pub page: u32,
/// Final page-content source.
pub source: PageContentSource,
/// OCR model, when OCR ran.
pub ocr_model: Option<ModelIdentity>,
/// Learned layout model, when layout inference ran.
pub layout_model: Option<ModelIdentity>,
/// Render resolution used for local vision.
pub render_dpi: Option<f32>,
/// Mean accepted OCR confidence, when available.
pub ocr_confidence: Option<f32>,
/// Stage timings.
pub timings: VisionTimings,
/// Non-fatal warnings surfaced to downstream users.
pub warnings: Vec<String>,
/// True when this lightweight local path detected a case better suited to
/// Firecrawl's hosted document pipeline.
pub hosted_recommended: bool,
}
/// Converts selected PDF pages into renderer-neutral owned bitmaps.
pub trait PageRenderer: Send + Sync {
/// Renderer-specific failure type.
type Error: Error + Send + Sync + 'static;
/// Renders selected 1-indexed pages in the same order as `pages`.
fn render_pages(
&self,
pdf_bytes: &[u8],
pages: &[u32],
password: Option<&str>,
options: &RenderOptions,
) -> Result<Vec<RenderedPage>, Self::Error>;
}
/// Recognizes positioned text from rendered pages.
pub trait OcrEngine: Send + Sync {
/// Engine-specific failure type.
type Error: Error + Send + Sync + 'static;
/// Exact model identity used by this engine instance.
fn model(&self) -> &ModelIdentity;
/// Recognizes pages in batch and returns results in input order.
fn recognize(
&self,
pages: &[RenderedPage],
options: &OcrOptions,
) -> Result<Vec<OcrPage>, Self::Error>;
}
/// Optional learned semantic layout extension.
pub trait LayoutEngine: Send + Sync {
/// Engine-specific failure type.
type Error: Error + Send + Sync + 'static;
/// Exact model identity used by this engine instance.
fn model(&self) -> &ModelIdentity;
/// Analyzes rendered pages, optionally using their OCR spans.
fn analyze(
&self,
pages: &[RenderedPage],
ocr: &[OcrPage],
options: &LayoutOptions,
) -> Result<Vec<LayoutPage>, Self::Error>;
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ocr_defaults_never_enable_recognition() {
let options = OcrOptions::default();
assert_eq!(options.mode, OcrMode::Off);
}
#[test]
fn offline_model_override_is_explicit() {
let options = OcrOptions::new()
.mode(OcrMode::Auto)
.model_directory("/models/pp-ocr")
.model_downloads(ModelDownloadPolicy::Offline);
assert_eq!(options.mode, OcrMode::Auto);
assert_eq!(options.model_downloads, ModelDownloadPolicy::Offline);
assert_eq!(
options.model_directory,
Some(PathBuf::from("/models/pp-ocr"))
);
}
}
+84
View File
@@ -0,0 +1,84 @@
//! HTTPS acquisition for pinned local model artifacts.
use std::fmt;
use std::io::Read;
use std::time::Duration;
use thiserror::Error;
use super::{ModelArtifact, ModelDownloader};
/// Default end-to-end timeout for one model artifact request.
pub const DEFAULT_MODEL_DOWNLOAD_TIMEOUT: Duration = Duration::from_secs(5 * 60);
/// Streaming HTTPS downloader used by lazy model resolution.
#[derive(Clone)]
pub struct HttpModelDownloader {
agent: ureq::Agent,
}
impl fmt::Debug for HttpModelDownloader {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("HttpModelDownloader")
.finish_non_exhaustive()
}
}
impl Default for HttpModelDownloader {
fn default() -> Self {
Self::new(DEFAULT_MODEL_DOWNLOAD_TIMEOUT)
}
}
impl HttpModelDownloader {
/// Creates an HTTPS-only downloader with an end-to-end request timeout.
pub fn new(timeout: Duration) -> Self {
let config = ureq::Agent::config_builder()
.https_only(true)
.timeout_global(Some(timeout))
.user_agent(concat!("pdf-inspector/", env!("CARGO_PKG_VERSION")))
.build();
Self {
agent: ureq::Agent::new_with_config(config),
}
}
}
impl ModelDownloader for HttpModelDownloader {
type Error = HttpModelDownloadError;
fn open(&self, artifact: &ModelArtifact) -> Result<Box<dyn Read + Send>, Self::Error> {
let response = self.agent.get(artifact.url).call()?;
if let Some(actual) = response.body().content_length() {
if actual != artifact.size {
return Err(HttpModelDownloadError::ContentLength {
expected: artifact.size,
actual,
});
}
}
// One extra byte lets ModelStore report an exact size mismatch while
// preventing a malicious or broken server from filling the disk.
let limit = artifact.size.saturating_add(1);
Ok(Box::new(response.into_body().into_reader().take(limit)))
}
}
/// Failures before a response stream reaches [`super::ModelStore`].
#[derive(Debug, Error)]
#[non_exhaustive]
pub enum HttpModelDownloadError {
/// DNS, TLS, redirect, HTTP status, or response-stream setup failed.
#[error(transparent)]
Request(#[from] ureq::Error),
/// The server declared a size that disagrees with the pinned manifest.
#[error("server declared {actual} bytes; manifest requires {expected}")]
ContentLength {
/// Pinned artifact size.
expected: u64,
/// Server-declared size.
actual: u64,
},
}
+792
View File
@@ -0,0 +1,792 @@
//! Geometry-aware OCR Markdown assembly and native-text fusion.
use std::collections::{BTreeMap, BTreeSet};
use std::time::Instant;
use thiserror::Error;
use crate::markdown::{to_markdown_from_items_with_rects_and_page_count, MarkdownOptions};
use crate::types::{ItemType, TextItem};
use crate::PageMarkdown;
use super::{
OcrRun, PageContentSource, PageProvenance, RoutedOcrPage, VisionTimings, DEFAULT_RENDER_DPI,
};
/// OCR assembly and hosted-fallback policy.
#[derive(Debug, Clone)]
pub struct OcrFusionOptions {
/// Markdown conversion options applied to positioned OCR spans.
pub markdown: MarkdownOptions,
/// Render resolution recorded in page provenance.
pub render_dpi: f32,
/// Recommend the hosted pipeline below this mean confidence when native
/// extraction already marked the page as requiring OCR.
pub hosted_recommendation_confidence: f32,
}
impl Default for OcrFusionOptions {
fn default() -> Self {
Self {
markdown: MarkdownOptions {
include_page_numbers: false,
strip_headers_footers: false,
..MarkdownOptions::default()
},
render_dpi: DEFAULT_RENDER_DPI,
hosted_recommendation_confidence: 0.5,
}
}
}
impl OcrFusionOptions {
/// Creates OCR fusion options with local defaults.
pub fn new() -> Self {
Self::default()
}
/// Replaces Markdown conversion options.
pub fn markdown(mut self, markdown: MarkdownOptions) -> Self {
self.markdown = markdown;
self
}
/// Records the renderer resolution in provenance.
pub fn render_dpi(mut self, render_dpi: f32) -> Self {
self.render_dpi = render_dpi;
self
}
/// Sets the weak-OCR confidence threshold for recommending hosted parsing.
pub fn hosted_recommendation_confidence(mut self, confidence: f32) -> Self {
self.hosted_recommendation_confidence = confidence;
self
}
}
/// Final Markdown and provenance for one page.
#[derive(Debug, Clone, PartialEq)]
pub struct FusedPageMarkdown {
/// 1-indexed document page number, matching OCR and provenance fields.
pub page: u32,
/// Final page Markdown.
pub markdown: String,
/// Native/OCR source, model, timing, and fallback metadata.
pub provenance: PageProvenance,
}
/// Output of fusing a selective OCR run into native page extraction.
#[derive(Debug, Clone, PartialEq)]
pub struct FusedPages {
/// Pages in the same order as the native input.
pub pages: Vec<FusedPageMarkdown>,
/// Batch rendering wall time from the OCR run.
pub render_time_ms: u64,
/// Batch OCR wall time from the OCR run.
pub ocr_time_ms: u64,
}
/// Converts positioned OCR spans to Markdown through pdf-inspector's existing
/// deterministic geometry, reading-order, table, and Markdown pipeline.
///
/// The page number is 1-indexed. `document_page_count` prevents a selected
/// page from being mistaken for a one-page document during page-number logic.
pub fn ocr_page_to_markdown(
page: &RoutedOcrPage,
document_page_count: u32,
options: &MarkdownOptions,
) -> String {
let (items, _) = ocr_text_items(page);
to_markdown_from_items_with_rects_and_page_count(
items,
options.clone(),
&[],
document_page_count,
)
}
/// Fuses a selective OCR run into per-page native Markdown.
///
/// OCR replaces pages whose native extraction was already rejected. On clean
/// native pages (for example in `Force` mode), normalized duplicate OCR blocks
/// are removed and only genuinely additional blocks are appended. Pages that
/// needed OCR but still have no credible local result recommend the hosted
/// document pipeline instead of silently presenting an empty result as final.
pub fn fuse_ocr_pages(
native_pages: &[PageMarkdown],
ocr_run: &OcrRun,
document_page_count: u32,
options: &OcrFusionOptions,
) -> Result<FusedPages, OcrFusionError> {
validate_options(options)?;
let mut native_numbers = BTreeSet::new();
for page in native_pages {
let page_number = page
.page
.checked_add(1)
.ok_or(OcrFusionError::PageOverflow)?;
if !native_numbers.insert(page_number) {
return Err(OcrFusionError::DuplicateNativePage { page: page_number });
}
}
let mut ocr_by_page = BTreeMap::new();
for page in &ocr_run.pages {
let page_number = page.rendered.page();
if !native_numbers.contains(&page_number) {
return Err(OcrFusionError::UnexpectedOcrPage { page: page_number });
}
if ocr_by_page.insert(page_number, page).is_some() {
return Err(OcrFusionError::DuplicateOcrPage { page: page_number });
}
}
let render_shares = equal_time_shares(ocr_run.render_time_ms, ocr_run.pages.len());
let render_by_page: BTreeMap<u32, u64> = ocr_run
.pages
.iter()
.zip(render_shares)
.map(|(page, share)| (page.rendered.page(), share))
.collect();
let mut pages = Vec::with_capacity(native_pages.len());
for native in native_pages {
let page_number = native.page + 1;
let assembly_started = Instant::now();
let mut warnings = Vec::new();
let (markdown, source, ocr_model, ocr_confidence, ocr_ms, hosted_recommended) =
if let Some(local) = ocr_by_page.get(&page_number) {
let (ocr_items, discarded_spans) = ocr_text_items(local);
if discarded_spans > 0 {
warnings.push(format!(
"discarded {discarded_spans} OCR spans with unusable text or geometry"
));
}
warnings.extend(local.ocr.warnings.iter().cloned());
let ocr_markdown = to_markdown_from_items_with_rects_and_page_count(
ocr_items,
options.markdown.clone(),
&[],
document_page_count,
);
let (markdown, source) = if native.markdown.trim().is_empty() || native.needs_ocr {
(ocr_markdown, PageContentSource::Ocr)
} else {
merge_native_and_ocr(&native.markdown, &ocr_markdown)
};
let weak_ocr = local
.ocr
.mean_confidence
.is_none_or(|confidence| confidence < options.hosted_recommendation_confidence);
let recommend_hosted = native.needs_ocr && (markdown.trim().is_empty() || weak_ocr);
if native.needs_ocr && markdown.trim().is_empty() {
warnings.push("OCR produced no usable text".to_string());
}
(
markdown,
source,
Some(local.ocr.model.clone()),
local.ocr.mean_confidence,
local.ocr.processing_time_ms,
recommend_hosted,
)
} else {
if native.needs_ocr {
warnings.push("page was recommended for OCR but was not processed".to_string());
}
(
native.markdown.clone(),
PageContentSource::Native,
None,
None,
0,
native.needs_ocr,
)
};
pages.push(FusedPageMarkdown {
page: page_number,
markdown,
provenance: PageProvenance {
page: page_number,
source,
ocr_model,
layout_model: None,
render_dpi: ocr_by_page
.contains_key(&page_number)
.then_some(options.render_dpi),
ocr_confidence,
timings: VisionTimings {
render_ms: render_by_page.get(&page_number).copied().unwrap_or(0),
ocr_ms,
layout_ms: 0,
assembly_ms: elapsed_ms(assembly_started),
},
warnings,
hosted_recommended,
},
});
}
Ok(FusedPages {
pages,
render_time_ms: ocr_run.render_time_ms,
ocr_time_ms: ocr_run.ocr_time_ms,
})
}
/// Converts recognized line polygons to ordinary PDF-space text items.
fn ocr_text_items(page: &RoutedOcrPage) -> (Vec<TextItem>, usize) {
let mut discarded = 0usize;
let mut items = Vec::with_capacity(page.ocr.spans.len());
for span in &page.ocr.spans {
if span.text.trim().is_empty() {
discarded += 1;
continue;
}
let Some((left, top, right, bottom)) = image_quad_bounds(
&span.polygon.points,
page.rendered.width(),
page.rendered.height(),
) else {
discarded += 1;
continue;
};
let rect = page.rendered.pixel_rect_to_pdf_rect(
f64::from(left),
f64::from(top),
f64::from(right - left),
f64::from(bottom - top),
);
items.push(TextItem {
text: span.text.trim().to_string(),
x: rect.x,
y: rect.y,
width: rect.width,
height: rect.height,
font: "OCR".to_string(),
font_size: rect.height.max(1.0),
page: page.rendered.page(),
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
});
}
// OCR engines do not share an ordering contract. Geometry gives the
// deterministic top-to-bottom seed expected by the existing layout
// pipeline, which can still replace it with column-aware reading order.
items.sort_by(|first, second| {
first
.page
.cmp(&second.page)
.then(second.y.total_cmp(&first.y))
.then(first.x.total_cmp(&second.x))
});
(items, discarded)
}
fn image_quad_bounds(
points: &[super::ImagePoint; 4],
width: u32,
height: u32,
) -> Option<(f32, f32, f32, f32)> {
if points
.iter()
.any(|point| !point.x.is_finite() || !point.y.is_finite())
{
return None;
}
let left = points
.iter()
.map(|point| point.x)
.fold(f32::INFINITY, f32::min)
.clamp(0.0, width as f32);
let right = points
.iter()
.map(|point| point.x)
.fold(f32::NEG_INFINITY, f32::max)
.clamp(0.0, width as f32);
let top = points
.iter()
.map(|point| point.y)
.fold(f32::INFINITY, f32::min)
.clamp(0.0, height as f32);
let bottom = points
.iter()
.map(|point| point.y)
.fold(f32::NEG_INFINITY, f32::max)
.clamp(0.0, height as f32);
(right > left && bottom > top).then_some((left, top, right, bottom))
}
fn merge_native_and_ocr(native: &str, ocr: &str) -> (String, PageContentSource) {
let native_keys = comparison_units(native);
let mut addition_keys = Vec::new();
let mut additions: Vec<String> = Vec::new();
for block in markdown_blocks(ocr) {
let key = ComparisonKey::new(block);
if key.is_empty()
|| native_keys.iter().any(|native| native.same_content(&key))
|| addition_keys
.iter()
.any(|existing: &ComparisonKey| existing.same_content(&key))
{
continue;
}
let addition = if let Some(native) = best_partial_overlap(&key, &native_keys) {
let Some(novel) = novel_fragments(block, native) else {
continue;
};
novel
} else {
block.trim_end().to_string()
};
let addition_key = ComparisonKey::new(&addition);
if addition_key.is_empty()
|| native_keys
.iter()
.any(|native| native.same_content(&addition_key))
|| addition_keys
.iter()
.any(|existing: &ComparisonKey| existing.same_content(&addition_key))
{
continue;
}
addition_keys.push(addition_key);
additions.push(addition);
}
if additions.is_empty() {
(ensure_trailing_newline(native), PageContentSource::Native)
} else {
let mut result = native.trim_end_matches('\n').to_string();
if !result.is_empty() {
result.push_str("\n\n");
}
result.push_str(&additions.join("\n\n"));
result.push('\n');
(result, PageContentSource::Fused)
}
}
fn markdown_blocks(markdown: &str) -> impl Iterator<Item = &str> {
markdown
.split("\n\n")
.map(str::trim_end)
.filter(|block| !block.trim().is_empty())
}
#[derive(Debug)]
struct ComparisonKey {
tokens: Vec<String>,
punctuation: String,
}
impl ComparisonKey {
fn new(text: &str) -> Self {
let mut tokens = Vec::new();
let mut token = String::new();
let mut punctuation = String::new();
for character in text.chars().flat_map(char::to_lowercase) {
if character.is_alphanumeric() {
token.push(character);
} else {
if !token.is_empty() {
tokens.push(std::mem::take(&mut token));
}
if !character.is_whitespace() {
punctuation.push(character);
}
}
}
if !token.is_empty() {
tokens.push(token);
}
Self {
tokens,
punctuation,
}
}
fn is_empty(&self) -> bool {
self.tokens.is_empty() && self.punctuation.is_empty()
}
fn same_content(&self, other: &Self) -> bool {
if self.tokens.is_empty() || other.tokens.is_empty() {
self.tokens.is_empty()
&& other.tokens.is_empty()
&& self.punctuation == other.punctuation
} else {
token_counts(&self.tokens) == token_counts(&other.tokens)
}
}
}
fn comparison_units(markdown: &str) -> Vec<ComparisonKey> {
let mut units: Vec<_> = markdown_blocks(markdown).map(ComparisonKey::new).collect();
units.extend(
markdown
.lines()
.filter(|line| !line.trim().is_empty())
.map(ComparisonKey::new),
);
units
}
fn token_counts(tokens: &[String]) -> BTreeMap<&str, usize> {
let mut counts = BTreeMap::new();
for token in tokens {
*counts.entry(token.as_str()).or_insert(0) += 1;
}
counts
}
fn best_partial_overlap<'a>(
ocr: &ComparisonKey,
native: &'a [ComparisonKey],
) -> Option<&'a ComparisonKey> {
if ocr.tokens.len() < 2 {
return None;
}
native
.iter()
.filter(|key| !key.tokens.is_empty())
.filter_map(|key| {
let overlap = token_overlap(&ocr.tokens, &key.tokens);
let substantial =
overlap >= 2 && overlap * 2 >= ocr.tokens.len() && overlap * 2 >= key.tokens.len();
substantial.then_some((overlap, key))
})
.max_by_key(|(overlap, _)| *overlap)
.map(|(_, key)| key)
}
fn token_overlap(first: &[String], second: &[String]) -> usize {
let first = token_counts(first);
let second = token_counts(second);
first
.iter()
.map(|(token, count)| (*count).min(second.get(token).copied().unwrap_or(0)))
.sum()
}
fn novel_fragments(block: &str, native: &ComparisonKey) -> Option<String> {
let mut available = token_counts(&native.tokens);
let mut novel = Vec::new();
for fragment in block.split_whitespace() {
let key = ComparisonKey::new(fragment);
let mut adds_content = false;
for token in &key.tokens {
match available.get_mut(token.as_str()) {
Some(count) if *count > 0 => *count -= 1,
_ => adds_content = true,
}
}
if adds_content {
novel.push(fragment);
}
}
(!novel.is_empty()).then(|| novel.join(" "))
}
fn ensure_trailing_newline(markdown: &str) -> String {
let mut result = markdown.trim_end_matches('\n').to_string();
result.push('\n');
result
}
fn equal_time_shares(total: u64, count: usize) -> Vec<u64> {
if count == 0 {
return Vec::new();
}
let count = count as u64;
let base = total / count;
let remainder = total % count;
(0..count)
.map(|index| base + u64::from(index < remainder))
.collect()
}
fn validate_options(options: &OcrFusionOptions) -> Result<(), OcrFusionError> {
if !options.render_dpi.is_finite() || options.render_dpi <= 0.0 {
return Err(OcrFusionError::InvalidRenderDpi {
value: options.render_dpi,
});
}
let confidence = options.hosted_recommendation_confidence;
if !confidence.is_finite() || !(0.0..=1.0).contains(&confidence) {
return Err(OcrFusionError::InvalidHostedConfidence { value: confidence });
}
Ok(())
}
fn elapsed_ms(started: Instant) -> u64 {
u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX)
}
/// Invalid page sets or fusion options.
#[derive(Debug, Error, PartialEq)]
#[non_exhaustive]
pub enum OcrFusionError {
/// A 0-indexed native page could not be converted to 1-indexed form.
#[error("native page number cannot be converted to a 1-indexed page")]
PageOverflow,
/// Native input repeated a page number.
#[error("native page {page} appears more than once")]
DuplicateNativePage {
/// Repeated 1-indexed page number.
page: u32,
},
/// OCR input repeated a page number.
#[error("OCR page {page} appears more than once")]
DuplicateOcrPage {
/// Repeated 1-indexed page number.
page: u32,
},
/// OCR returned a page that was not part of native extraction.
#[error("OCR page {page} is not present in native page extraction")]
UnexpectedOcrPage {
/// Unexpected 1-indexed page number.
page: u32,
},
/// Render resolution is non-finite or non-positive.
#[error("render DPI must be positive and finite, got {value}")]
InvalidRenderDpi {
/// Invalid value.
value: f32,
},
/// Hosted recommendation confidence is outside 01.
#[error("hosted recommendation confidence must be between 0 and 1, got {value}")]
InvalidHostedConfidence {
/// Invalid value.
value: f32,
},
}
#[cfg(test)]
mod tests {
use super::*;
use crate::vision::{
ImagePoint, ImageQuad, ModelIdentity, OcrPage, OcrSpan, PageTransform, RenderPixelFormat,
RenderedPage,
};
fn native(page: u32, markdown: &str, needs_ocr: bool) -> PageMarkdown {
PageMarkdown {
page,
markdown: markdown.to_string(),
needs_ocr,
ocr_reason: needs_ocr.then(|| "scanned".to_string()),
}
}
fn rendered_page(page: u32) -> RenderedPage {
let transform =
PageTransform::from_corners(200, 100, (0.0, 100.0), (200.0, 100.0), (0.0, 0.0))
.unwrap();
RenderedPage::new(
page,
200.0,
100.0,
200,
100,
600,
RenderPixelFormat::Rgb8,
vec![255; 60_000],
transform,
)
.unwrap()
}
fn span(text: &str, top: f32, confidence: f32) -> OcrSpan {
OcrSpan {
text: text.to_string(),
polygon: ImageQuad::new([
ImagePoint::new(10.0, top),
ImagePoint::new(190.0, top),
ImagePoint::new(190.0, top + 10.0),
ImagePoint::new(10.0, top + 10.0),
]),
confidence,
orientation_degrees: None,
}
}
fn routed_page(page: u32, spans: Vec<OcrSpan>, confidence: Option<f32>) -> RoutedOcrPage {
RoutedOcrPage {
rendered: rendered_page(page),
ocr: OcrPage {
page,
spans,
mean_confidence: confidence,
model: ModelIdentity::new("test-ocr", "v1"),
processing_time_ms: 7,
warnings: Vec::new(),
},
}
}
fn run(pages: Vec<RoutedOcrPage>) -> OcrRun {
OcrRun {
pages,
render_time_ms: 5,
ocr_time_ms: 7,
}
}
#[test]
fn scanned_page_uses_geometry_ordered_ocr_and_provenance() {
let native = [native(0, "", true)];
let run = run(vec![routed_page(
1,
vec![
span("Second line", 30.0, 0.9),
span("First line", 10.0, 0.9),
],
Some(0.9),
)]);
let result = fuse_ocr_pages(&native, &run, 1, &OcrFusionOptions::new()).unwrap();
assert!(
result.pages[0].markdown.find("First").unwrap()
< result.pages[0].markdown.find("Second").unwrap()
);
assert_eq!(result.pages[0].provenance.source, PageContentSource::Ocr);
assert_eq!(result.pages[0].page, 1);
assert_eq!(result.pages[0].page, result.pages[0].provenance.page);
assert_eq!(
result.pages[0].provenance.ocr_model.as_ref().unwrap().name,
"test-ocr"
);
assert!(!result.pages[0].provenance.hosted_recommended);
}
#[test]
fn force_mode_deduplicates_native_content() {
let native = [native(0, "Hello, world!\n", false)];
let run = run(vec![routed_page(
1,
vec![span("Hello world", 10.0, 0.9)],
Some(0.9),
)]);
let result = fuse_ocr_pages(&native, &run, 1, &OcrFusionOptions::new()).unwrap();
assert_eq!(result.pages[0].markdown, "Hello, world!\n");
assert_eq!(result.pages[0].provenance.source, PageContentSource::Native);
}
#[test]
fn force_mode_appends_only_additional_ocr_blocks() {
let native = [native(0, "Native title\n", false)];
let run = run(vec![routed_page(
1,
vec![
span("Native title", 10.0, 0.9),
span("Image-only label", 30.0, 0.9),
],
Some(0.9),
)]);
let result = fuse_ocr_pages(&native, &run, 1, &OcrFusionOptions::new()).unwrap();
assert_eq!(result.pages[0].provenance.source, PageContentSource::Fused);
assert_eq!(result.pages[0].markdown.matches("Native title").count(), 1);
assert!(result.pages[0].markdown.contains("Image-only label"));
}
#[test]
fn force_mode_preserves_punctuation_and_short_distinct_blocks() {
let (markdown, source) = merge_native_and_ocr("Figure 1\n", "1\n\n***");
assert_eq!(source, PageContentSource::Fused);
assert_eq!(markdown, "Figure 1\n\n1\n\n***\n");
}
#[test]
fn force_mode_extracts_only_novel_tokens_from_overlapping_blocks() {
let (markdown, source) = merge_native_and_ocr("A C\n", "## A B C");
assert_eq!(source, PageContentSource::Fused);
assert_eq!(markdown, "A C\n\nB\n");
assert_eq!(markdown.matches('A').count(), 1);
assert_eq!(markdown.matches('C').count(), 1);
}
#[test]
fn force_mode_deduplicates_reordered_tokens() {
let native = [native(0, "Alpha Beta Gamma\n", false)];
let run = run(vec![routed_page(
1,
vec![span("Gamma Alpha Beta", 10.0, 0.9)],
Some(0.9),
)]);
let result = fuse_ocr_pages(&native, &run, 1, &OcrFusionOptions::new()).unwrap();
assert_eq!(result.pages[0].provenance.source, PageContentSource::Native);
assert_eq!(result.pages[0].markdown, "Alpha Beta Gamma\n");
}
#[test]
fn fusion_preserves_native_leading_indentation() {
let native = [native(0, " indented code\n", false)];
let duplicate = run(vec![routed_page(
1,
vec![span("indented code", 10.0, 0.9)],
Some(0.9),
)]);
let result = fuse_ocr_pages(&native, &duplicate, 1, &OcrFusionOptions::new()).unwrap();
assert_eq!(result.pages[0].markdown, " indented code\n");
let addition = run(vec![routed_page(
1,
vec![span("image label", 10.0, 0.9)],
Some(0.9),
)]);
let result = fuse_ocr_pages(&native, &addition, 1, &OcrFusionOptions::new()).unwrap();
assert_eq!(
result.pages[0].markdown,
" indented code\n\nimage label\n"
);
}
#[test]
fn missing_or_weak_required_ocr_recommends_hosted() {
let native = [native(0, "", true), native(1, "", true)];
let run = run(vec![routed_page(
2,
vec![span("uncertain", 10.0, 0.3)],
Some(0.3),
)]);
let result = fuse_ocr_pages(&native, &run, 2, &OcrFusionOptions::new()).unwrap();
assert!(result.pages[0].provenance.hosted_recommended);
assert!(result.pages[1].provenance.hosted_recommended);
}
#[test]
fn rejects_ocr_pages_outside_native_selection() {
let error = fuse_ocr_pages(
&[native(0, "text", false)],
&run(vec![routed_page(2, Vec::new(), None)]),
2,
&OcrFusionOptions::new(),
)
.unwrap_err();
assert_eq!(error, OcrFusionError::UnexpectedOcrPage { page: 2 });
}
#[test]
fn time_shares_preserve_batch_total() {
assert_eq!(equal_time_shares(8, 3), vec![3, 3, 2]);
assert_eq!(equal_time_shares(8, 0), Vec::<u64>::new());
}
}
+60
View File
@@ -0,0 +1,60 @@
//! Optional native vision primitives used by OCR pipelines.
//!
//! The existing lopdf extractor remains the default path. Native page
//! rendering is available only with the `render-pdfium` feature. Engine
//! contracts are available with `vision`, while checksum-verified model
//! resolution is a separate `model-cache` feature. The `ocr-oar` feature adds
//! a CPU PP-OCRv6 Small implementation of [`OcrEngine`]. These remain separate
//! so browser WASM, text-only consumers, and renderer-only users take on no
//! model-management or inference dependencies.
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
mod contracts;
#[cfg(all(feature = "model-download", not(target_arch = "wasm32")))]
mod download;
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
mod fusion;
#[cfg(all(feature = "model-cache", not(target_arch = "wasm32")))]
mod models;
#[cfg(all(feature = "ocr-oar", not(target_arch = "wasm32")))]
mod oar;
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
mod render;
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
mod routing;
#[cfg(all(feature = "render-pdfium", not(target_arch = "wasm32")))]
mod pdfium;
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
pub use contracts::{
ImagePoint, ImageQuad, LayoutEngine, LayoutOptions, LayoutPage, LayoutRegion, LayoutRegionKind,
ModelDownloadPolicy, ModelIdentity, OcrEngine, OcrMode, OcrOptions, OcrPage, OcrProfile,
OcrSpan, PageContentSource, PageProvenance, PageRenderer, VisionTimings,
};
#[cfg(all(feature = "model-download", not(target_arch = "wasm32")))]
pub use download::{HttpModelDownloadError, HttpModelDownloader, DEFAULT_MODEL_DOWNLOAD_TIMEOUT};
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
pub use fusion::{
fuse_ocr_pages, ocr_page_to_markdown, FusedPageMarkdown, FusedPages, OcrFusionError,
OcrFusionOptions,
};
#[cfg(all(feature = "model-cache", not(target_arch = "wasm32")))]
pub use models::{
ModelAcquireError, ModelArtifact, ModelArtifactKind, ModelDownloader, ModelManifest,
ModelPaths, ModelStore, ModelStoreError, PP_OCR_V6_SMALL,
};
#[cfg(all(feature = "ocr-oar", not(target_arch = "wasm32")))]
pub use oar::{OarOcrEngine, OarOcrError, ONNX_RUNTIME_LIBRARY_ENV};
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
pub use render::{
PagePoint, PageTransform, RenderBufferError, RenderOptions, RenderPixelFormat, RenderedPage,
DEFAULT_RENDER_DPI,
};
#[cfg(all(feature = "vision", not(target_arch = "wasm32")))]
pub use routing::{
route_ocr_pages, run_ocr_pages, OcrRoutingError, OcrRun, OcrRunError, RoutedOcrPage,
};
#[cfg(all(feature = "render-pdfium", not(target_arch = "wasm32")))]
pub use pdfium::{PdfiumRenderer, RenderError};
+1004
View File
File diff suppressed because it is too large Load Diff
+469
View File
@@ -0,0 +1,469 @@
//! PP-OCRv6 Small implementation backed by OAR and ONNX Runtime.
use std::path::PathBuf;
use std::time::Instant;
use image::RgbImage;
use oar_ocr::oarocr::{OAROCRBuilder, OAROCR};
use oar_ocr::processors::BoundingBox;
use thiserror::Error;
use super::{
ImagePoint, ImageQuad, ModelArtifactKind, ModelIdentity, ModelPaths, OcrEngine, OcrMode,
OcrOptions, OcrPage, OcrSpan, RenderPixelFormat, RenderedPage,
};
/// Environment variable selecting the ONNX Runtime shared library.
pub const ONNX_RUNTIME_LIBRARY_ENV: &str = "ORT_DYLIB_PATH";
/// Failures while constructing or running the OAR OCR backend.
#[derive(Debug, Error)]
#[non_exhaustive]
pub enum OarOcrError {
/// A required file is missing from the resolved model set.
#[error("resolved OCR model set is missing {kind:?}")]
MissingModelArtifact {
/// Missing artifact role.
kind: ModelArtifactKind,
},
/// OCR was invoked while the caller explicitly disabled it.
#[error("OCR is disabled; select Auto or Force before invoking the engine")]
OcrDisabled,
/// Confidence thresholds must match the normalized engine output range.
#[error("minimum OCR confidence must be finite and between 0 and 1, got {value}")]
InvalidMinimumConfidence {
/// Invalid threshold.
value: f32,
},
/// Bitmap dimension arithmetic exceeded the host address space.
#[error("rendered page {page} bitmap dimensions overflow the host address space")]
ImageSizeOverflow {
/// 1-indexed page number.
page: u32,
},
/// A validated renderer buffer could not be represented as an RGB image.
#[error("rendered page {page} could not be converted to an RGB image")]
InvalidImageBuffer {
/// 1-indexed page number.
page: u32,
},
/// The external ONNX Runtime shared library could not be loaded.
#[error("failed to load ONNX Runtime from {path}: {source}")]
OnnxRuntimeLoad {
/// Requested shared-library path or platform library name.
path: PathBuf,
/// Dynamic-loader failure.
#[source]
source: ort::LoadDynamicError,
},
/// OAR returned no result for a submitted page.
#[error("OAR returned no result for rendered page {page}")]
MissingPageResult {
/// 1-indexed page number.
page: u32,
},
/// OAR or ONNX Runtime rejected the models or failed during inference.
#[error(transparent)]
Backend(#[from] oar_ocr::core::OCRError),
}
/// CPU PP-OCRv6 Small engine using OAR's detection and recognition pipeline.
///
/// Construction accepts only [`ModelPaths`] that have already passed
/// pdf-inspector's manifest size and SHA-256 verification. OAR's independent
/// model auto-download feature is deliberately not enabled.
#[derive(Debug)]
pub struct OarOcrEngine {
pipeline: OAROCR,
model: ModelIdentity,
}
impl OarOcrEngine {
/// Loads PP-OCRv6 Small from a resolved, verified model set.
pub fn from_models(models: &ModelPaths) -> Result<Self, OarOcrError> {
load_onnx_runtime()?;
let detection = required_model(models, ModelArtifactKind::TextDetection)?;
let recognition = required_model(models, ModelArtifactKind::TextRecognition)?;
let dictionary = required_model(models, ModelArtifactKind::CharacterDictionary)?;
let pipeline = OAROCRBuilder::new(detection, recognition, dictionary).build()?;
let model = ModelIdentity::new(models.manifest_id(), models.revision());
Ok(Self { pipeline, model })
}
fn recognize_page(
&self,
page: &RenderedPage,
options: &OcrOptions,
) -> Result<OcrPage, OarOcrError> {
let started = Instant::now();
let image = rendered_page_to_rgb(page)?;
let result = self
.pipeline
.predict(vec![image])?
.into_iter()
.next()
.ok_or(OarOcrError::MissingPageResult { page: page.page() })?;
let mut spans = Vec::with_capacity(result.text_regions.len());
let mut invalid_geometry = 0usize;
let mut missing_recognition = 0usize;
for region in result.text_regions {
let (Some(text), Some(confidence)) = (region.text, region.confidence) else {
missing_recognition += 1;
continue;
};
if text.trim().is_empty() || !confidence.is_finite() {
missing_recognition += 1;
continue;
}
let confidence = confidence.clamp(0.0, 1.0);
if confidence < options.minimum_confidence {
continue;
}
let polygon = region.dt_poly.as_ref().unwrap_or(&region.bounding_box);
let Some(polygon) = bounding_box_to_quad(polygon, page.width(), page.height()) else {
invalid_geometry += 1;
continue;
};
spans.push(OcrSpan {
text: text.to_string(),
polygon,
confidence,
orientation_degrees: region.orientation_angle,
});
}
let mut warnings = Vec::new();
if !options.languages.is_empty() {
warnings
.push("language hints are not used by the PP-OCRv6 Small OAR backend".to_string());
}
if missing_recognition > 0 {
warnings.push(format!(
"discarded {missing_recognition} regions without usable recognition output"
));
}
if invalid_geometry > 0 {
warnings.push(format!(
"discarded {invalid_geometry} recognized regions with invalid geometry"
));
}
let mean_confidence = if spans.is_empty() {
None
} else {
Some(spans.iter().map(|span| span.confidence).sum::<f32>() / spans.len() as f32)
};
let processing_time_ms = u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX);
Ok(OcrPage {
page: page.page(),
spans,
mean_confidence,
model: self.model.clone(),
processing_time_ms,
warnings,
})
}
}
fn load_onnx_runtime() -> Result<(), OarOcrError> {
let path = std::env::var_os(ONNX_RUNTIME_LIBRARY_ENV)
.filter(|path| !path.is_empty())
.map(PathBuf::from)
.unwrap_or_else(default_onnx_runtime_library);
drop(
ort::init_from(&path).map_err(|source| OarOcrError::OnnxRuntimeLoad {
path: path.clone(),
source,
})?,
);
Ok(())
}
fn default_onnx_runtime_library() -> PathBuf {
#[cfg(target_os = "windows")]
const NAME: &str = "onnxruntime.dll";
#[cfg(any(target_os = "linux", target_os = "android", target_os = "freebsd"))]
const NAME: &str = "libonnxruntime.so";
#[cfg(any(target_os = "macos", target_os = "ios"))]
const NAME: &str = "libonnxruntime.dylib";
PathBuf::from(NAME)
}
impl OcrEngine for OarOcrEngine {
type Error = OarOcrError;
fn model(&self) -> &ModelIdentity {
&self.model
}
fn recognize(
&self,
pages: &[RenderedPage],
options: &OcrOptions,
) -> Result<Vec<OcrPage>, Self::Error> {
validate_options(options)?;
pages
.iter()
.map(|page| self.recognize_page(page, options))
.collect()
}
}
fn validate_options(options: &OcrOptions) -> Result<(), OarOcrError> {
if options.mode == OcrMode::Off {
return Err(OarOcrError::OcrDisabled);
}
if !options.minimum_confidence.is_finite() || !(0.0..=1.0).contains(&options.minimum_confidence)
{
return Err(OarOcrError::InvalidMinimumConfidence {
value: options.minimum_confidence,
});
}
Ok(())
}
fn required_model(
models: &ModelPaths,
kind: ModelArtifactKind,
) -> Result<&std::path::Path, OarOcrError> {
models
.get(kind)
.ok_or(OarOcrError::MissingModelArtifact { kind })
}
fn rendered_page_to_rgb(page: &RenderedPage) -> Result<RgbImage, OarOcrError> {
let width = usize::try_from(page.width())
.map_err(|_| OarOcrError::ImageSizeOverflow { page: page.page() })?;
let height = usize::try_from(page.height())
.map_err(|_| OarOcrError::ImageSizeOverflow { page: page.page() })?;
let output_len = width
.checked_mul(height)
.and_then(|pixels| pixels.checked_mul(3))
.ok_or(OarOcrError::ImageSizeOverflow { page: page.page() })?;
let input_bpp = page.format().bytes_per_pixel();
let active_input_row = width
.checked_mul(input_bpp)
.ok_or(OarOcrError::ImageSizeOverflow { page: page.page() })?;
let output_row = width
.checked_mul(3)
.ok_or(OarOcrError::ImageSizeOverflow { page: page.page() })?;
let mut rgb = vec![0u8; output_len];
for row in 0..height {
let input_start = row * page.stride();
let input = &page.pixels()[input_start..input_start + active_input_row];
let output_start = row * output_row;
let output = &mut rgb[output_start..output_start + output_row];
match page.format() {
RenderPixelFormat::Rgb8 => output.copy_from_slice(input),
RenderPixelFormat::Rgba8 => {
for (rgba, rgb) in input.chunks_exact(4).zip(output.chunks_exact_mut(3)) {
rgb.copy_from_slice(&rgba[..3]);
}
}
RenderPixelFormat::Gray8 => {
for (&gray, rgb) in input.iter().zip(output.chunks_exact_mut(3)) {
rgb.fill(gray);
}
}
}
}
RgbImage::from_raw(page.width(), page.height(), rgb)
.ok_or(OarOcrError::InvalidImageBuffer { page: page.page() })
}
fn bounding_box_to_quad(bounding_box: &BoundingBox, width: u32, height: u32) -> Option<ImageQuad> {
let points: Vec<ImagePoint> = bounding_box
.points
.iter()
.filter(|point| point.x.is_finite() && point.y.is_finite())
.map(|point| {
ImagePoint::new(
point.x.clamp(0.0, width as f32),
point.y.clamp(0.0, height as f32),
)
})
.collect();
if bounding_box.points.len() == 4 && points.len() == 4 && is_ordered_convex_quad(&points) {
return Some(ImageQuad::new([points[0], points[1], points[2], points[3]]));
}
if points.len() < 3 {
return None;
}
let min_x = points
.iter()
.map(|point| point.x)
.fold(f32::INFINITY, f32::min);
let max_x = points
.iter()
.map(|point| point.x)
.fold(f32::NEG_INFINITY, f32::max);
let min_y = points
.iter()
.map(|point| point.y)
.fold(f32::INFINITY, f32::min);
let max_y = points
.iter()
.map(|point| point.y)
.fold(f32::NEG_INFINITY, f32::max);
if max_x <= min_x || max_y <= min_y {
return None;
}
Some(ImageQuad::new([
ImagePoint::new(min_x, min_y),
ImagePoint::new(max_x, min_y),
ImagePoint::new(max_x, max_y),
ImagePoint::new(min_x, max_y),
]))
}
fn is_ordered_convex_quad(points: &[ImagePoint]) -> bool {
if points.len() != 4 {
return false;
}
let mut orientation = 0.0_f32;
for index in 0..4 {
let first = points[index];
let second = points[(index + 1) % 4];
let third = points[(index + 2) % 4];
let cross = (second.x - first.x) * (third.y - second.y)
- (second.y - first.y) * (third.x - second.x);
if cross.abs() <= f32::EPSILON {
return false;
}
if orientation == 0.0 {
orientation = cross.signum();
} else if cross.signum() != orientation {
return false;
}
}
true
}
#[cfg(test)]
mod tests {
use oar_ocr::processors::Point;
use super::*;
use crate::vision::PageTransform;
fn page(format: RenderPixelFormat, stride: usize, pixels: Vec<u8>) -> RenderedPage {
let transform =
PageTransform::from_corners(2, 2, (0.0, 2.0), (2.0, 2.0), (0.0, 0.0)).unwrap();
RenderedPage::new(1, 2.0, 2.0, 2, 2, stride, format, pixels, transform).unwrap()
}
#[test]
fn converts_padded_rgb_without_exposing_padding() {
let page = page(
RenderPixelFormat::Rgb8,
8,
vec![1, 2, 3, 4, 5, 6, 99, 99, 7, 8, 9, 10, 11, 12, 99, 99],
);
let image = rendered_page_to_rgb(&page).unwrap();
assert_eq!(image.as_raw(), &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]);
}
#[test]
fn converts_rgba_and_gray_to_rgb() {
let rgba = page(
RenderPixelFormat::Rgba8,
8,
vec![1, 2, 3, 44, 4, 5, 6, 55, 7, 8, 9, 66, 10, 11, 12, 77],
);
assert_eq!(
rendered_page_to_rgb(&rgba).unwrap().as_raw(),
&[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
);
let gray = page(RenderPixelFormat::Gray8, 2, vec![1, 2, 3, 4]);
assert_eq!(
rendered_page_to_rgb(&gray).unwrap().as_raw(),
&[1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4]
);
}
#[test]
fn preserves_quads_and_clamps_them_to_the_bitmap() {
let bbox = BoundingBox::new(vec![
Point::new(-1.0, 2.0),
Point::new(11.0, 2.0),
Point::new(11.0, 9.0),
Point::new(-1.0, 9.0),
]);
let quad = bounding_box_to_quad(&bbox, 10, 8).unwrap();
assert_eq!(quad.points[0], ImagePoint::new(0.0, 2.0));
assert_eq!(quad.points[2], ImagePoint::new(10.0, 8.0));
}
#[test]
fn reduces_polygons_to_a_stable_axis_aligned_quad() {
let bbox = BoundingBox::new(vec![
Point::new(2.0, 1.0),
Point::new(7.0, 2.0),
Point::new(8.0, 6.0),
Point::new(5.0, 9.0),
Point::new(1.0, 5.0),
]);
let quad = bounding_box_to_quad(&bbox, 10, 10).unwrap();
assert_eq!(quad.points[0], ImagePoint::new(1.0, 1.0));
assert_eq!(quad.points[2], ImagePoint::new(8.0, 9.0));
}
#[test]
fn normalizes_unordered_or_partially_invalid_quads() {
let unordered = BoundingBox::new(vec![
Point::new(1.0, 1.0),
Point::new(8.0, 8.0),
Point::new(8.0, 1.0),
Point::new(1.0, 8.0),
]);
let quad = bounding_box_to_quad(&unordered, 10, 10).unwrap();
assert_eq!(quad.points[0], ImagePoint::new(1.0, 1.0));
assert_eq!(quad.points[1], ImagePoint::new(8.0, 1.0));
assert_eq!(quad.points[2], ImagePoint::new(8.0, 8.0));
let partially_invalid = BoundingBox::new(vec![
Point::new(8.0, 8.0),
Point::new(f32::NAN, 4.0),
Point::new(1.0, 8.0),
Point::new(8.0, 1.0),
Point::new(1.0, 1.0),
]);
let quad = bounding_box_to_quad(&partially_invalid, 10, 10).unwrap();
assert_eq!(quad.points[0], ImagePoint::new(1.0, 1.0));
assert_eq!(quad.points[1], ImagePoint::new(8.0, 1.0));
assert_eq!(quad.points[2], ImagePoint::new(8.0, 8.0));
}
#[test]
fn refuses_disabled_or_invalid_options_before_inference() {
assert!(matches!(
validate_options(&OcrOptions::new()),
Err(OarOcrError::OcrDisabled)
));
for value in [-0.1, 1.1, f32::NAN, f32::INFINITY] {
let options = OcrOptions::new()
.mode(OcrMode::Force)
.minimum_confidence(value);
assert!(matches!(
validate_options(&options),
Err(OarOcrError::InvalidMinimumConfidence { .. })
));
}
assert!(validate_options(
&OcrOptions::new()
.mode(OcrMode::Auto)
.minimum_confidence(1.0)
)
.is_ok());
}
}
+266
View File
@@ -0,0 +1,266 @@
//! PDFium-backed implementation of the renderer-neutral page contract.
use std::path::Path;
use firecrawl_pdfium::{Pdfium, PixelFormat, PixelPoint, RenderConfig};
use thiserror::Error;
use super::{
PageRenderer, PageTransform, RenderBufferError, RenderOptions, RenderPixelFormat, RenderedPage,
};
impl RenderPixelFormat {
fn pdfium_format(self) -> PixelFormat {
match self {
// PDFium produces BGR directly; `rendered_page_from_pdfium`
// swaps the red and blue channels in place.
Self::Rgb8 => PixelFormat::Bgr8,
Self::Rgba8 => PixelFormat::Rgba8,
Self::Gray8 => PixelFormat::Gray8,
}
}
}
impl RenderOptions {
fn pdfium_config(&self) -> RenderConfig {
RenderConfig::new()
.dpi(self.dpi)
.pixel_format(self.pixel_format.pdfium_format())
.annotations(self.annotations)
.form_fields(self.form_fields)
.max_output_bytes(self.max_output_bytes_per_page)
}
}
/// Errors produced by the optional local renderer.
#[derive(Debug, Error)]
#[non_exhaustive]
pub enum RenderError {
/// Page numbers in pdf-inspector APIs are 1-indexed, so zero is invalid.
#[error("page numbers are 1-indexed; page 0 is invalid")]
InvalidPageNumber,
/// The requested 1-indexed page is not present in the document.
#[error("page {page} is out of bounds for a {page_count}-page document")]
PageOutOfBounds {
/// Requested 1-indexed page.
page: u32,
/// Number of pages in the document.
page_count: usize,
},
/// PDFium loading, document parsing, form setup, or rendering failed.
#[error(transparent)]
Pdfium(#[from] firecrawl_pdfium::Error),
/// PDFium returned an internally inconsistent bitmap or transform.
#[error(transparent)]
Buffer(#[from] RenderBufferError),
}
/// Loaded PDFium renderer used to prepare pages for OCR.
///
/// PDFium calls are safe from concurrent threads but serialize inside the
/// underlying binding. Returned [`RenderedPage`] values are ordinary owned
/// data and can be processed concurrently after rendering.
#[derive(Debug, Clone, Copy)]
pub struct PdfiumRenderer {
pdfium: Pdfium,
}
impl PdfiumRenderer {
/// Loads PDFium using `firecrawl-pdfium`'s documented discovery chain.
pub fn load() -> Result<Self, RenderError> {
Ok(Self {
pdfium: Pdfium::load()?,
})
}
/// Loads PDFium from an explicit native library path.
pub fn load_from_path(path: impl AsRef<Path>) -> Result<Self, RenderError> {
Ok(Self {
pdfium: Pdfium::load_from_path(path)?,
})
}
/// Path of the active PDFium library, if it was loaded from a concrete
/// file rather than through the system loader.
pub fn loaded_from(&self) -> Option<&Path> {
self.pdfium.loaded_from()
}
/// Renders selected 1-indexed pages in the same order as `pages`.
///
/// This inherent method mirrors [`PageRenderer`] so existing callers do
/// not need to import the trait.
pub fn render_pages(
&self,
pdf_bytes: &[u8],
pages: &[u32],
password: Option<&str>,
options: &RenderOptions,
) -> Result<Vec<RenderedPage>, RenderError> {
self.render_pages_impl(pdf_bytes, pages, password, options)
}
fn render_pages_impl(
&self,
pdf_bytes: &[u8],
pages: &[u32],
password: Option<&str>,
options: &RenderOptions,
) -> Result<Vec<RenderedPage>, RenderError> {
if pages.is_empty() {
return Ok(Vec::new());
}
if pages.contains(&0) {
return Err(RenderError::InvalidPageNumber);
}
let document = self.pdfium.load_document(pdf_bytes.to_vec(), password)?;
let page_count = document.page_count();
if let Some(&page) = pages.iter().find(|&&page| page as usize > page_count) {
return Err(RenderError::PageOutOfBounds { page, page_count });
}
if options.form_fields {
document.enable_form_rendering()?;
}
let config = options.pdfium_config();
let mut rendered_pages = Vec::with_capacity(pages.len());
for &page_number in pages {
let page = document.page(page_number as usize - 1)?;
let size = page.size();
let rendered = page.render(&config)?;
rendered_pages.push(rendered_page_from_pdfium(
page_number,
size.width,
size.height,
options.pixel_format,
rendered,
)?);
}
Ok(rendered_pages)
}
}
impl PageRenderer for PdfiumRenderer {
type Error = RenderError;
fn render_pages(
&self,
pdf_bytes: &[u8],
pages: &[u32],
password: Option<&str>,
options: &RenderOptions,
) -> Result<Vec<RenderedPage>, Self::Error> {
self.render_pages_impl(pdf_bytes, pages, password, options)
}
}
fn rendered_page_from_pdfium(
page: u32,
page_width: f32,
page_height: f32,
format: RenderPixelFormat,
rendered: firecrawl_pdfium::RenderedPage,
) -> Result<RenderedPage, RenderBufferError> {
let width = rendered.width();
let height = rendered.height();
let stride = rendered.stride();
let pdfium_transform = *rendered.transform();
let corner = |x, y| {
let point = pdfium_transform.pixel_to_page(PixelPoint::new(x, y));
(point.x, point.y)
};
let transform = PageTransform::from_corners(
width,
height,
corner(0.0, 0.0),
corner(f64::from(width), 0.0),
corner(0.0, f64::from(height)),
)
.ok_or(RenderBufferError::InvalidTransform)?;
let mut pixels = rendered.into_pixels();
if format == RenderPixelFormat::Rgb8 {
bgr_to_rgb_in_place(&mut pixels, width, height, stride)?;
}
RenderedPage::new(
page,
page_width,
page_height,
width,
height,
stride,
format,
pixels,
transform,
)
}
fn bgr_to_rgb_in_place(
pixels: &mut [u8],
width: u32,
height: u32,
stride: usize,
) -> Result<(), RenderBufferError> {
let row_bytes = (width as usize)
.checked_mul(RenderPixelFormat::Rgb8.bytes_per_pixel())
.ok_or(RenderBufferError::SizeOverflow)?;
if stride < row_bytes {
return Err(RenderBufferError::InvalidStride {
stride,
minimum: row_bytes,
});
}
let expected = stride
.checked_mul(height as usize)
.ok_or(RenderBufferError::SizeOverflow)?;
if pixels.len() != expected {
return Err(RenderBufferError::InvalidBufferLength {
actual: pixels.len(),
expected,
});
}
for row in pixels.chunks_exact_mut(stride) {
for pixel in row[..row_bytes].chunks_exact_mut(3) {
pixel.swap(0, 2);
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn bgr_pixels_are_converted_to_rgb_in_place() {
let mut pixels = vec![1, 2, 3, 4, 5, 6];
bgr_to_rgb_in_place(&mut pixels, 2, 1, 6).unwrap();
assert_eq!(pixels, [3, 2, 1, 6, 5, 4]);
}
#[test]
fn bgr_conversion_skips_row_padding() {
let mut pixels = vec![1, 2, 3, 9, 7, 8, 9, 6];
bgr_to_rgb_in_place(&mut pixels, 1, 2, 4).unwrap();
assert_eq!(pixels, [3, 2, 1, 9, 9, 8, 7, 6]);
}
#[test]
fn malformed_bgr_buffers_return_errors() {
assert!(matches!(
bgr_to_rgb_in_place(&mut [0; 6], 2, 1, 5),
Err(RenderBufferError::InvalidStride { .. })
));
assert!(matches!(
bgr_to_rgb_in_place(&mut [0; 5], 1, 2, 3),
Err(RenderBufferError::InvalidBufferLength { .. })
));
}
}
+552
View File
@@ -0,0 +1,552 @@
//! Renderer-neutral page bitmap and coordinate types.
use thiserror::Error;
use crate::PdfRect;
/// Default rendering resolution for OCR.
pub const DEFAULT_RENDER_DPI: f32 = 150.0;
/// Default maximum size of one rendered page: 256 MiB.
pub const DEFAULT_MAX_OUTPUT_BYTES: u64 = 256 * 1024 * 1024;
/// Pixel layout returned by [`RenderedPage`].
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
#[non_exhaustive]
pub enum RenderPixelFormat {
/// Three bytes per pixel in red, green, blue order. This is the default
/// because OCR preprocessors conventionally consume RGB images.
#[default]
Rgb8,
/// Four bytes per pixel in red, green, blue, alpha order.
Rgba8,
/// One luminance byte per pixel.
Gray8,
}
impl RenderPixelFormat {
/// Number of bytes used by one pixel.
pub fn bytes_per_pixel(self) -> usize {
match self {
Self::Rgb8 => 3,
Self::Rgba8 => 4,
Self::Gray8 => 1,
}
}
}
/// Configuration for pages rendered as input to a local vision pipeline.
#[derive(Debug, Clone, PartialEq)]
pub struct RenderOptions {
/// Output resolution. Defaults to 150 DPI.
pub dpi: f32,
/// Pixel layout. Defaults to three-channel RGB.
pub pixel_format: RenderPixelFormat,
/// Include PDF annotations in the rendered bitmap.
pub annotations: bool,
/// Include visible static AcroForm field appearances.
pub form_fields: bool,
/// Maximum allocation for each rendered page.
pub max_output_bytes_per_page: u64,
}
impl Default for RenderOptions {
fn default() -> Self {
Self {
dpi: DEFAULT_RENDER_DPI,
pixel_format: RenderPixelFormat::Rgb8,
annotations: true,
form_fields: true,
max_output_bytes_per_page: DEFAULT_MAX_OUTPUT_BYTES,
}
}
}
impl RenderOptions {
/// Creates local-rendering options with OCR-oriented defaults.
pub fn new() -> Self {
Self::default()
}
/// Sets the output resolution in dots per inch.
pub fn dpi(mut self, dpi: f32) -> Self {
self.dpi = dpi;
self
}
/// Sets the output pixel layout.
pub fn pixel_format(mut self, pixel_format: RenderPixelFormat) -> Self {
self.pixel_format = pixel_format;
self
}
/// Toggles annotation rendering.
pub fn annotations(mut self, annotations: bool) -> Self {
self.annotations = annotations;
self
}
/// Toggles visible static form-field rendering.
pub fn form_fields(mut self, form_fields: bool) -> Self {
self.form_fields = form_fields;
self
}
/// Sets the maximum allocation for each rendered page.
pub fn max_output_bytes_per_page(mut self, bytes: u64) -> Self {
self.max_output_bytes_per_page = bytes;
self
}
}
/// A point in PDF page space, measured in points from the bottom-left.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct PagePoint {
/// Horizontal position in PDF points.
pub x: f32,
/// Vertical position in PDF points, increasing upward.
pub y: f32,
}
/// Affine transform between top-left pixel space and PDF page space.
///
/// Renderers create this from the page-space images of the bitmap corners.
/// Keeping the coefficients in pdf-inspector makes [`RenderedPage`] neutral
/// to the renderer implementation that produced it.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct PageTransform {
forward: [f64; 6],
inverse: [f64; 6],
pixel_width: u32,
pixel_height: u32,
}
impl PageTransform {
/// Builds a transform from the PDF-space images of device corners
/// `(0, 0)`, `(pixel_width, 0)`, and `(0, pixel_height)`.
pub fn from_corners(
pixel_width: u32,
pixel_height: u32,
origin: (f64, f64),
x_axis: (f64, f64),
y_axis: (f64, f64),
) -> Option<Self> {
if pixel_width == 0 || pixel_height == 0 {
return None;
}
let values = [origin.0, origin.1, x_axis.0, x_axis.1, y_axis.0, y_axis.1];
if values.iter().any(|value| !value.is_finite()) {
return None;
}
let width = f64::from(pixel_width);
let height = f64::from(pixel_height);
let a = (x_axis.0 - origin.0) / width;
let c = (x_axis.1 - origin.1) / width;
let b = (y_axis.0 - origin.0) / height;
let d = (y_axis.1 - origin.1) / height;
let (e, f) = origin;
let forward = [a, b, c, d, e, f];
if forward.iter().any(|coefficient| !coefficient.is_finite()) {
return None;
}
let determinant = a * d - b * c;
if determinant == 0.0 || !determinant.is_finite() {
return None;
}
let inverse_a = d / determinant;
let inverse_b = -b / determinant;
let inverse_c = -c / determinant;
let inverse_d = a / determinant;
let inverse_e = -(inverse_a * e + inverse_b * f);
let inverse_f = -(inverse_c * e + inverse_d * f);
let inverse = [
inverse_a, inverse_b, inverse_c, inverse_d, inverse_e, inverse_f,
];
if inverse.iter().any(|coefficient| !coefficient.is_finite()) {
return None;
}
Some(Self {
forward,
inverse,
pixel_width,
pixel_height,
})
}
/// Width of the bitmap this transform describes.
pub fn pixel_width(&self) -> u32 {
self.pixel_width
}
/// Height of the bitmap this transform describes.
pub fn pixel_height(&self) -> u32 {
self.pixel_height
}
/// Converts a bitmap point to PDF page space.
pub fn pixel_to_page(&self, x: f64, y: f64) -> PagePoint {
let [a, b, c, d, e, f] = self.forward;
PagePoint {
x: (a * x + b * y + e) as f32,
y: (c * x + d * y + f) as f32,
}
}
/// Converts a PDF page-space point to bitmap coordinates.
pub fn page_to_pixel(&self, x: f64, y: f64) -> (f64, f64) {
let [a, b, c, d, e, f] = self.inverse;
(a * x + b * y + e, c * x + d * y + f)
}
}
/// Invalid renderer output rejected by [`RenderedPage::new`].
#[derive(Debug, Error, Clone, PartialEq, Eq)]
#[non_exhaustive]
pub enum RenderBufferError {
/// Page numbers are 1-indexed.
#[error("rendered page number must be at least 1")]
InvalidPageNumber,
/// Bitmap dimensions must be non-zero.
#[error("rendered bitmap dimensions must be non-zero")]
InvalidDimensions,
/// Page dimensions must be positive finite numbers.
#[error("rendered PDF page dimensions must be positive and finite")]
InvalidPageDimensions,
/// Transform dimensions must match the bitmap dimensions.
#[error("coordinate transform dimensions do not match the rendered bitmap")]
TransformDimensions,
/// Renderer did not provide an invertible finite coordinate transform.
#[error("renderer returned an invalid coordinate transform")]
InvalidTransform,
/// The stride cannot hold one active row of pixels.
#[error("pixel stride {stride} is shorter than the active row size {minimum}")]
InvalidStride {
/// Supplied bytes per row.
stride: usize,
/// Minimum bytes required for one row.
minimum: usize,
},
/// Pixel buffer size is inconsistent with height and stride.
#[error("pixel buffer has {actual} bytes; expected {expected}")]
InvalidBufferLength {
/// Actual byte count.
actual: usize,
/// Required byte count.
expected: usize,
},
/// Dimension arithmetic overflowed the host address space.
#[error("rendered bitmap dimensions overflow the host address space")]
SizeOverflow,
}
/// One rendered page with owned pixels and its pixel-to-PDF transform.
///
/// The value contains no live renderer, page, or document handles. It can be
/// moved to an OCR worker and retained after rendering returns.
#[derive(Debug, Clone)]
pub struct RenderedPage {
page: u32,
page_width: f32,
page_height: f32,
width: u32,
height: u32,
stride: usize,
format: RenderPixelFormat,
pixels: Vec<u8>,
transform: PageTransform,
}
impl RenderedPage {
/// Creates a renderer-neutral owned page after validating its buffer.
#[allow(clippy::too_many_arguments)]
pub fn new(
page: u32,
page_width: f32,
page_height: f32,
width: u32,
height: u32,
stride: usize,
format: RenderPixelFormat,
pixels: Vec<u8>,
transform: PageTransform,
) -> Result<Self, RenderBufferError> {
if page == 0 {
return Err(RenderBufferError::InvalidPageNumber);
}
if width == 0 || height == 0 {
return Err(RenderBufferError::InvalidDimensions);
}
if page_width <= 0.0
|| page_height <= 0.0
|| !page_width.is_finite()
|| !page_height.is_finite()
{
return Err(RenderBufferError::InvalidPageDimensions);
}
if transform.pixel_width() != width || transform.pixel_height() != height {
return Err(RenderBufferError::TransformDimensions);
}
let row_bytes = (width as usize)
.checked_mul(format.bytes_per_pixel())
.ok_or(RenderBufferError::SizeOverflow)?;
if stride < row_bytes {
return Err(RenderBufferError::InvalidStride {
stride,
minimum: row_bytes,
});
}
let expected = stride
.checked_mul(height as usize)
.ok_or(RenderBufferError::SizeOverflow)?;
if pixels.len() != expected {
return Err(RenderBufferError::InvalidBufferLength {
actual: pixels.len(),
expected,
});
}
Ok(Self {
page,
page_width,
page_height,
width,
height,
stride,
format,
pixels,
transform,
})
}
/// 1-indexed page number.
pub fn page(&self) -> u32 {
self.page
}
/// Page width in PDF points after applying the page's rotation.
pub fn page_width(&self) -> f32 {
self.page_width
}
/// Page height in PDF points after applying the page's rotation.
pub fn page_height(&self) -> f32 {
self.page_height
}
/// Bitmap width in pixels.
pub fn width(&self) -> u32 {
self.width
}
/// Bitmap height in pixels.
pub fn height(&self) -> u32 {
self.height
}
/// Number of bytes between adjacent bitmap rows.
pub fn stride(&self) -> usize {
self.stride
}
/// Pixel layout of [`pixels`](Self::pixels).
pub fn format(&self) -> RenderPixelFormat {
self.format
}
/// Owned bitmap bytes, with rows ordered top-to-bottom.
pub fn pixels(&self) -> &[u8] {
&self.pixels
}
/// Consumes the page and returns its pixel buffer.
pub fn into_pixels(self) -> Vec<u8> {
self.pixels
}
/// Coordinate transform associated with the rendered page.
pub fn transform(&self) -> PageTransform {
self.transform
}
/// Converts a bitmap point (top-left origin, y-down) to PDF page space
/// (bottom-left origin, y-up).
pub fn pixel_to_page(&self, x: f64, y: f64) -> PagePoint {
self.transform.pixel_to_page(x, y)
}
/// Converts a bitmap rectangle to the repository's existing PDF-space
/// rectangle type. The returned page number remains 1-indexed.
pub fn pixel_rect_to_pdf_rect(&self, x: f64, y: f64, width: f64, height: f64) -> PdfRect {
let points = [
self.transform.pixel_to_page(x, y),
self.transform.pixel_to_page(x + width, y),
self.transform.pixel_to_page(x, y + height),
self.transform.pixel_to_page(x + width, y + height),
];
let left = points
.iter()
.map(|point| point.x)
.fold(f32::INFINITY, f32::min);
let right = points
.iter()
.map(|point| point.x)
.fold(f32::NEG_INFINITY, f32::max);
let bottom = points
.iter()
.map(|point| point.y)
.fold(f32::INFINITY, f32::min);
let top = points
.iter()
.map(|point| point.y)
.fold(f32::NEG_INFINITY, f32::max);
PdfRect {
x: left,
y: bottom,
width: right - left,
height: top - bottom,
page: self.page,
}
}
/// Converts a PDF-space rectangle to bitmap coordinates
/// `(x, y, width, height)` with a top-left origin.
pub fn pdf_rect_to_pixel(&self, rect: &PdfRect) -> (f64, f64, f64, f64) {
let left = f64::from(rect.x);
let right = f64::from(rect.x + rect.width);
let bottom = f64::from(rect.y);
let top = f64::from(rect.y + rect.height);
let points = [
self.transform.page_to_pixel(left, bottom),
self.transform.page_to_pixel(right, bottom),
self.transform.page_to_pixel(left, top),
self.transform.page_to_pixel(right, top),
];
let min_x = points
.iter()
.map(|point| point.0)
.fold(f64::INFINITY, f64::min);
let max_x = points
.iter()
.map(|point| point.0)
.fold(f64::NEG_INFINITY, f64::max);
let min_y = points
.iter()
.map(|point| point.1)
.fold(f64::INFINITY, f64::min);
let max_y = points
.iter()
.map(|point| point.1)
.fold(f64::NEG_INFINITY, f64::max);
(min_x, min_y, max_x - min_x, max_y - min_y)
}
}
#[cfg(test)]
mod tests {
use super::*;
fn transform() -> PageTransform {
PageTransform::from_corners(400, 200, (0.0, 100.0), (200.0, 100.0), (0.0, 0.0)).unwrap()
}
#[test]
fn transform_maps_both_directions_at_non_identity_scale() {
let transform = transform();
let point = transform.pixel_to_page(100.0, 50.0);
assert!((point.x - 50.0).abs() < 1e-6);
assert!((point.y - 75.0).abs() < 1e-6);
let pixel = transform.page_to_pixel(f64::from(point.x), f64::from(point.y));
assert!((pixel.0 - 100.0).abs() < 1e-6);
assert!((pixel.1 - 50.0).abs() < 1e-6);
}
#[test]
fn rendered_page_accepts_padding_and_validates_length() {
let page = RenderedPage::new(
1,
200.0,
100.0,
400,
200,
1_204,
RenderPixelFormat::Rgb8,
vec![0; 1_204 * 200],
transform(),
)
.unwrap();
assert_eq!(page.stride(), 1_204);
assert!(matches!(
RenderedPage::new(
1,
200.0,
100.0,
400,
200,
1_204,
RenderPixelFormat::Rgb8,
vec![0; 5],
transform(),
),
Err(RenderBufferError::InvalidBufferLength { .. })
));
}
#[test]
fn rotated_transform_round_trips_rectangles() {
let transform =
PageTransform::from_corners(100, 200, (0.0, 0.0), (0.0, 100.0), (200.0, 0.0)).unwrap();
let page = RenderedPage::new(
1,
200.0,
100.0,
100,
200,
300,
RenderPixelFormat::Rgb8,
vec![0; 300 * 200],
transform,
)
.unwrap();
let pdf = page.pixel_rect_to_pdf_rect(10.0, 20.0, 30.0, 40.0);
let pixel = page.pdf_rect_to_pixel(&pdf);
assert!((pixel.0 - 10.0).abs() < 1e-5);
assert!((pixel.1 - 20.0).abs() < 1e-5);
assert!((pixel.2 - 30.0).abs() < 1e-5);
assert!((pixel.3 - 40.0).abs() < 1e-5);
}
#[test]
fn skewed_transform_bounds_all_rectangle_corners() {
let transform =
PageTransform::from_corners(100, 100, (0.0, 100.0), (100.0, 125.0), (25.0, 0.0))
.unwrap();
let page = RenderedPage::new(
1,
125.0,
125.0,
100,
100,
300,
RenderPixelFormat::Rgb8,
vec![0; 30_000],
transform,
)
.unwrap();
let pdf = page.pixel_rect_to_pdf_rect(10.0, 20.0, 30.0, 40.0);
assert!((pdf.x - 15.0).abs() < 1e-5);
assert!((pdf.y - 42.5).abs() < 1e-5);
assert!((pdf.width - 40.0).abs() < 1e-5);
assert!((pdf.height - 47.5).abs() < 1e-5);
let pixels = page.pdf_rect_to_pixel(&pdf);
assert!(pixels.0 <= 10.0 && pixels.1 <= 20.0);
assert!(pixels.0 + pixels.2 >= 40.0 && pixels.1 + pixels.3 >= 60.0);
}
}
+380
View File
@@ -0,0 +1,380 @@
//! Page routing and renderer/OCR orchestration without Markdown fusion.
use std::collections::BTreeSet;
use std::error::Error;
use std::time::Instant;
use thiserror::Error;
use super::{OcrEngine, OcrMode, OcrOptions, OcrPage, PageRenderer, RenderOptions, RenderedPage};
/// A rendered page paired with OCR output in the same bitmap coordinate space.
#[derive(Debug)]
pub struct RoutedOcrPage {
/// Renderer-owned bitmap and pixel↔PDF transform.
pub rendered: RenderedPage,
/// Positioned OCR spans for the bitmap.
pub ocr: OcrPage,
}
/// Output of one selective OCR invocation.
#[derive(Debug)]
pub struct OcrRun {
/// Pages processed in ascending document order.
pub pages: Vec<RoutedOcrPage>,
/// Total page-rendering wall time.
pub render_time_ms: u64,
/// Total engine wall time.
pub ocr_time_ms: u64,
}
/// Selects 1-indexed pages for OCR.
///
/// `recommended_pages` comes from pdf-inspector's existing detector/text
/// quality signals. `selected_pages` is an optional user page filter. Results
/// are validated, deduplicated, and returned in document order.
pub fn route_ocr_pages(
mode: OcrMode,
page_count: u32,
recommended_pages: &[u32],
selected_pages: Option<&[u32]>,
) -> Result<Vec<u32>, OcrRoutingError> {
match mode {
OcrMode::Off => Ok(Vec::new()),
OcrMode::Auto => {
let mut routed = validated_page_set("recommended", recommended_pages, page_count)?;
if let Some(selected) = selected_pages {
let selected = validated_page_set("selected", selected, page_count)?;
routed.retain(|page| selected.contains(page));
}
Ok(routed.into_iter().collect())
}
OcrMode::Force => {
let routed = selected_pages
.map(|pages| validated_page_set("selected", pages, page_count))
.transpose()?
.unwrap_or_else(|| (1..=page_count).collect());
Ok(routed.into_iter().collect())
}
}
}
/// Renders and recognizes already-routed pages while retaining transforms for
/// the following fusion layer.
///
/// An empty page list returns without calling either dependency, which keeps
/// model resolution and inference lazy when Auto routing finds no OCR work.
#[allow(clippy::too_many_arguments)]
pub fn run_ocr_pages<R, O>(
renderer: &R,
engine: &O,
pdf_bytes: &[u8],
pages: &[u32],
password: Option<&str>,
render_options: &RenderOptions,
ocr_options: &OcrOptions,
) -> Result<OcrRun, OcrRunError>
where
R: PageRenderer,
O: OcrEngine,
{
if pages.is_empty() {
return Ok(OcrRun {
pages: Vec::new(),
render_time_ms: 0,
ocr_time_ms: 0,
});
}
if ocr_options.mode == OcrMode::Off {
return Err(OcrRunError::OcrDisabled);
}
let render_started = Instant::now();
let rendered = renderer
.render_pages(pdf_bytes, pages, password, render_options)
.map_err(|source| OcrRunError::Render {
source: Box::new(source),
})?;
let render_time_ms = elapsed_ms(render_started);
validate_page_order("renderer", pages, rendered.iter().map(RenderedPage::page))?;
let ocr_started = Instant::now();
let recognized =
engine
.recognize(&rendered, ocr_options)
.map_err(|source| OcrRunError::Ocr {
source: Box::new(source),
})?;
let ocr_time_ms = elapsed_ms(ocr_started);
validate_page_order("OCR engine", pages, recognized.iter().map(|page| page.page))?;
Ok(OcrRun {
pages: rendered
.into_iter()
.zip(recognized)
.map(|(rendered, ocr)| RoutedOcrPage { rendered, ocr })
.collect(),
render_time_ms,
ocr_time_ms,
})
}
/// Invalid page routing or renderer/engine contract output.
#[derive(Debug, Error)]
#[non_exhaustive]
pub enum OcrRoutingError {
/// A page list contained zero or a page beyond the document.
#[error("{source_name} OCR page {page} is outside the valid range 1..={page_count}")]
InvalidPage {
/// Page-list source.
source_name: &'static str,
/// Invalid 1-indexed page.
page: u32,
/// Document page count.
page_count: u32,
},
}
/// Failures while rendering and recognizing a routed page set.
#[derive(Debug, Error)]
#[non_exhaustive]
pub enum OcrRunError {
/// A non-empty route cannot execute with OCR disabled.
#[error("cannot process routed pages while OCR mode is Off")]
OcrDisabled,
/// Page rasterization failed.
#[error("page rendering failed: {source}")]
Render {
/// Renderer-specific failure.
#[source]
source: Box<dyn Error + Send + Sync>,
},
/// OCR inference failed.
#[error("OCR inference failed: {source}")]
Ocr {
/// Engine-specific failure.
#[source]
source: Box<dyn Error + Send + Sync>,
},
/// A dependency returned the wrong count or order.
#[error("{stage} returned pages {actual:?}; expected {expected:?}")]
PageOrderMismatch {
/// Dependency boundary that violated the contract.
stage: &'static str,
/// Requested 1-indexed pages.
expected: Vec<u32>,
/// Returned 1-indexed pages.
actual: Vec<u32>,
},
}
fn validated_page_set(
source_name: &'static str,
pages: &[u32],
page_count: u32,
) -> Result<BTreeSet<u32>, OcrRoutingError> {
let mut result = BTreeSet::new();
for &page in pages {
if page == 0 || page > page_count {
return Err(OcrRoutingError::InvalidPage {
source_name,
page,
page_count,
});
}
result.insert(page);
}
Ok(result)
}
fn validate_page_order(
stage: &'static str,
expected: &[u32],
actual: impl IntoIterator<Item = u32>,
) -> Result<(), OcrRunError> {
let actual: Vec<u32> = actual.into_iter().collect();
if actual == expected {
Ok(())
} else {
Err(OcrRunError::PageOrderMismatch {
stage,
expected: expected.to_vec(),
actual,
})
}
}
fn elapsed_ms(started: Instant) -> u64 {
u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::vision::{
ImagePoint, ImageQuad, ModelIdentity, OcrSpan, PageTransform, RenderBufferError,
RenderPixelFormat,
};
#[derive(Debug, Error)]
#[error("fake failure")]
struct FakeError;
struct FakeRenderer;
impl PageRenderer for FakeRenderer {
type Error = FakeError;
fn render_pages(
&self,
_pdf_bytes: &[u8],
pages: &[u32],
_password: Option<&str>,
_options: &RenderOptions,
) -> Result<Vec<RenderedPage>, Self::Error> {
pages
.iter()
.copied()
.map(rendered_page)
.collect::<Result<_, _>>()
.map_err(|_| FakeError)
}
}
struct FakeEngine {
model: ModelIdentity,
}
impl FakeEngine {
fn new() -> Self {
Self {
model: ModelIdentity::new("fake", "v1"),
}
}
}
impl OcrEngine for FakeEngine {
type Error = FakeError;
fn model(&self) -> &ModelIdentity {
&self.model
}
fn recognize(
&self,
pages: &[RenderedPage],
_options: &OcrOptions,
) -> Result<Vec<OcrPage>, Self::Error> {
Ok(pages
.iter()
.map(|page| OcrPage {
page: page.page(),
spans: vec![OcrSpan {
text: format!("page {}", page.page()),
polygon: ImageQuad::new([
ImagePoint::new(0.0, 0.0),
ImagePoint::new(1.0, 0.0),
ImagePoint::new(1.0, 1.0),
ImagePoint::new(0.0, 1.0),
]),
confidence: 0.9,
orientation_degrees: None,
}],
mean_confidence: Some(0.9),
model: self.model.clone(),
processing_time_ms: 1,
warnings: Vec::new(),
})
.collect())
}
}
fn rendered_page(page: u32) -> Result<RenderedPage, RenderBufferError> {
let transform =
PageTransform::from_corners(1, 1, (0.0, 1.0), (1.0, 1.0), (0.0, 0.0)).unwrap();
RenderedPage::new(
page,
1.0,
1.0,
1,
1,
3,
RenderPixelFormat::Rgb8,
vec![255; 3],
transform,
)
}
#[test]
fn off_auto_and_force_route_expected_pages() {
assert_eq!(
route_ocr_pages(OcrMode::Off, 0, &[99], Some(&[0])).unwrap(),
Vec::<u32>::new()
);
assert_eq!(
route_ocr_pages(OcrMode::Auto, 5, &[5, 3, 3, 1], Some(&[2, 3, 5])).unwrap(),
vec![3, 5]
);
assert_eq!(
route_ocr_pages(OcrMode::Force, 4, &[], None).unwrap(),
vec![1, 2, 3, 4]
);
assert_eq!(
route_ocr_pages(OcrMode::Force, 4, &[], Some(&[4, 2, 2])).unwrap(),
vec![2, 4]
);
}
#[test]
fn routing_rejects_invalid_page_numbers() {
assert!(matches!(
route_ocr_pages(OcrMode::Auto, 2, &[0], None),
Err(OcrRoutingError::InvalidPage { page: 0, .. })
));
assert!(matches!(
route_ocr_pages(OcrMode::Force, 2, &[], Some(&[3])),
Err(OcrRoutingError::InvalidPage { page: 3, .. })
));
}
#[test]
fn run_retains_render_transforms_and_input_order() {
let options = OcrOptions::new().mode(OcrMode::Auto);
let run = run_ocr_pages(
&FakeRenderer,
&FakeEngine::new(),
b"pdf",
&[2, 4],
None,
&RenderOptions::new(),
&options,
)
.unwrap();
assert_eq!(
run.pages
.iter()
.map(|page| page.rendered.page())
.collect::<Vec<_>>(),
vec![2, 4]
);
assert_eq!(run.pages[1].ocr.spans[0].text, "page 4");
assert_eq!(run.pages[1].rendered.pixel_to_page(0.0, 0.0).y, 1.0);
}
#[test]
fn empty_route_is_a_noop_even_when_ocr_is_off() {
let run = run_ocr_pages(
&FakeRenderer,
&FakeEngine::new(),
b"pdf",
&[],
None,
&RenderOptions::new(),
&OcrOptions::new(),
)
.unwrap();
assert!(run.pages.is_empty());
assert_eq!(run.render_time_ms, 0);
assert_eq!(run.ocr_time_ms, 0);
}
}
+65
View File
@@ -0,0 +1,65 @@
#![cfg(all(feature = "render-pdfium", not(target_arch = "wasm32")))]
use pdf_inspector::vision::{PdfiumRenderer, RenderError, RenderOptions, RenderPixelFormat};
fn load_renderer() -> Option<PdfiumRenderer> {
match PdfiumRenderer::load() {
Ok(renderer) => Some(renderer),
Err(RenderError::Pdfium(firecrawl_pdfium::Error::Load(
firecrawl_pdfium::LoadError::LibraryNotFound { .. },
))) => {
eprintln!("skipping PDFium runtime test because no native library is installed");
None
}
Err(error) => panic!("failed to load PDFium: {error}"),
}
}
#[test]
fn renders_owned_rgb_page_and_round_trips_coordinates() {
let Some(renderer) = load_renderer() else {
return;
};
let bytes = std::fs::read("tests/fixtures/thermo-freon12.pdf").unwrap();
let pages = renderer
.render_pages(
&bytes,
&[1],
None,
&RenderOptions::new().dpi(150.0).form_fields(false),
)
.unwrap();
assert_eq!(pages.len(), 1);
let page = &pages[0];
assert_eq!(page.page(), 1);
assert_eq!(page.format(), RenderPixelFormat::Rgb8);
assert_eq!(page.stride(), page.width() as usize * 3);
assert_eq!(page.pixels().len(), page.stride() * page.height() as usize);
assert!((page.width() as f32 - page.page_width()).abs() > 1.0);
let pdf_rect = page.pixel_rect_to_pdf_rect(10.0, 10.0, 20.0, 12.0);
let pixel_rect = page.pdf_rect_to_pixel(&pdf_rect);
assert!((pixel_rect.0 - 10.0).abs() < 0.01);
assert!((pixel_rect.1 - 10.0).abs() < 0.01);
assert!((pixel_rect.2 - 20.0).abs() < 0.01);
assert!((pixel_rect.3 - 12.0).abs() < 0.01);
}
#[test]
fn rejects_zero_and_out_of_range_page_numbers() {
let Some(renderer) = load_renderer() else {
return;
};
let bytes = std::fs::read("tests/fixtures/thermo-freon12.pdf").unwrap();
assert!(matches!(
renderer.render_pages(&bytes, &[0], None, &RenderOptions::new()),
Err(RenderError::InvalidPageNumber)
));
assert!(matches!(
renderer.render_pages(&bytes, &[u32::MAX], None, &RenderOptions::new()),
Err(RenderError::PageOutOfBounds { .. })
));
}
+127
View File
@@ -0,0 +1,127 @@
#![cfg(all(feature = "ocr-oar", not(target_arch = "wasm32")))]
use pdf_inspector::vision::{
ModelStore, OarOcrEngine, OcrEngine, OcrMode, OcrOptions, PageTransform, RenderPixelFormat,
RenderedPage, PP_OCR_V6_SMALL,
};
#[cfg(feature = "render-pdfium")]
use pdf_inspector::vision::{PdfiumRenderer, RenderError, RenderOptions};
const MODEL_DIRECTORY_ENV: &str = "PDF_INSPECTOR_OCR_TEST_MODELS";
const IMAGE_ENV: &str = "PDF_INSPECTOR_OCR_TEST_IMAGE";
const EXPECTED_TEXT_ENV: &str = "PDF_INSPECTOR_OCR_TEST_EXPECTED";
#[cfg(feature = "render-pdfium")]
fn load_renderer() -> Option<PdfiumRenderer> {
match PdfiumRenderer::load() {
Ok(renderer) => Some(renderer),
Err(RenderError::Pdfium(firecrawl_pdfium::Error::Load(
firecrawl_pdfium::LoadError::LibraryNotFound { .. },
))) => {
eprintln!("skipping OCR runtime test because no native PDFium library is installed");
None
}
Err(error) => panic!("failed to load PDFium: {error}"),
}
}
#[test]
fn recognizes_an_rgb_image_with_verified_models() {
let Some(model_directory) = std::env::var_os(MODEL_DIRECTORY_ENV) else {
eprintln!("skipping OCR runtime test because {MODEL_DIRECTORY_ENV} is not set");
return;
};
let Some(image_path) = std::env::var_os(IMAGE_ENV) else {
eprintln!("skipping OCR runtime test because {IMAGE_ENV} is not set");
return;
};
let image = image::open(image_path).unwrap().into_rgb8();
let (width, height) = image.dimensions();
let transform = PageTransform::from_corners(
width,
height,
(0.0, f64::from(height)),
(f64::from(width), f64::from(height)),
(0.0, 0.0),
)
.unwrap();
let page = RenderedPage::new(
1,
width as f32,
height as f32,
width,
height,
width as usize * 3,
RenderPixelFormat::Rgb8,
image.into_raw(),
transform,
)
.unwrap();
let results = recognize(&model_directory, &[page]);
assert_usable_result(&results);
let text = results[0]
.spans
.iter()
.map(|span| span.text.as_str())
.collect::<Vec<_>>()
.join(" ");
eprintln!("recognized: {text}");
if let Ok(expected) = std::env::var(EXPECTED_TEXT_ENV) {
assert!(
text.to_lowercase().contains(&expected.to_lowercase()),
"expected OCR output to contain {expected:?}, got {text:?}"
);
}
}
#[cfg(feature = "render-pdfium")]
#[test]
fn recognizes_a_pdfium_rendered_fixture_with_verified_models() {
let Some(model_directory) = std::env::var_os(MODEL_DIRECTORY_ENV) else {
eprintln!("skipping OCR runtime test because {MODEL_DIRECTORY_ENV} is not set");
return;
};
let Some(renderer) = load_renderer() else {
return;
};
let bytes = std::fs::read("tests/fixtures/thermo-freon12.pdf").unwrap();
let pages = renderer
.render_pages(
&bytes,
&[1],
None,
&RenderOptions::new().dpi(150.0).form_fields(false),
)
.unwrap();
let results = recognize(&model_directory, &pages);
assert_usable_result(&results);
}
fn recognize(
model_directory: &std::ffi::OsStr,
pages: &[RenderedPage],
) -> Vec<pdf_inspector::vision::OcrPage> {
let store = ModelStore::new(model_directory).override_root(model_directory);
let models = store.resolve(&PP_OCR_V6_SMALL).unwrap();
let engine = OarOcrEngine::from_models(&models).unwrap();
engine
.recognize(
pages,
&OcrOptions::new()
.mode(OcrMode::Force)
.minimum_confidence(0.3),
)
.unwrap()
}
fn assert_usable_result(results: &[pdf_inspector::vision::OcrPage]) {
assert_eq!(results.len(), 1);
assert_eq!(results[0].page, 1);
assert_eq!(results[0].model.name, PP_OCR_V6_SMALL.id);
assert_eq!(results[0].model.revision, PP_OCR_V6_SMALL.revision);
assert!(!results[0].spans.is_empty());
assert!(results[0].spans.iter().all(|span| span.confidence >= 0.3));
}