Abimael MartellandClaude Opus 4.7
647ea5c7cd
Some PDFs have fonts whose ToUnicode CMap is missing or broken —
Identity-H fonts without unicode metadata, Type-3 fonts where every
glyph maps to garbage. The page extractor returns punctuation-only
fragments or single-glyph repeats; the rendered image still carries
the visible text, so the region should fall back to OCR rather than
serve a partial table.
The existing captured_only_a_fragment guard can't catch this case
because region_text_chars itself collapses under font-decode failure —
captured vs extracted is symmetrically low, and the ratio still looks
acceptable.
Add a complementary area-based density guard: when a region has lots
of pixel real estate but very few text chars, the page extractor
hit a font failure. Bbox area is independent of extraction success,
so the symmetry breaks.
Threshold 0.003 chars/sq pt sits between observed clean extractions
(≥0.005 on full-page A4 ledgers, key/value layouts, archival
catalogs) and observed font-decode failures (≤0.0014 on prod-traffic
samples). Three guards keep it from misfiring:
- text_chars < 20 skipped: synthetic / fragmentary fixtures
- area < 30,000 sq pt skipped: tiny stat blocks
- area > 400,000 sq pt skipped: near-whole-A4 bboxes where
density is unreliable (large white-space margins)
Verified against three reproducible cases from prod shadow logs
that previously served partial output:
- Cyrillic page with punctuation-only decode (46 chars, density
0.00045) → flagged, routes to OCR
- Cyrillic page where every glyph collapsed to one letter (89
chars, density 0.00025) → flagged, routes to OCR
- Materials-test region where text extracted fine but the table
body extends beyond the bbox (96 chars, density 0.00131)
→ flagged, routes to OCR
Existing fixtures (full-page A4 ledger, multi-row key/value with
paragraph values, archival catalog, bits_pilani whole-page tests,
synthetic line-grid test) all retain identical behavior.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
56 lines
1.2 KiB
JSON
56 lines
1.2 KiB
JSON
{
|
|
"name": "@firecrawl/pdf-inspector",
|
|
"version": "1.8.14",
|
|
"description": "Fast PDF classification and text extraction. Detect text-based vs scanned PDFs, extract text by region with quality checks. Native Rust performance via napi-rs.",
|
|
"main": "index.js",
|
|
"types": "index.d.ts",
|
|
"bin": {
|
|
"pdf-inspector": "bin/pdf-inspector.mjs"
|
|
},
|
|
"license": "MIT",
|
|
"keywords": [
|
|
"pdf",
|
|
"pdf-extraction",
|
|
"pdf-parser",
|
|
"text-extraction",
|
|
"ocr",
|
|
"pdf-classification",
|
|
"napi",
|
|
"rust",
|
|
"firecrawl"
|
|
],
|
|
"files": [
|
|
"index.js",
|
|
"index.d.ts",
|
|
"*.node",
|
|
"bin/",
|
|
"README.md"
|
|
],
|
|
"repository": {
|
|
"type": "git",
|
|
"url": "https://github.com/firecrawl/pdf-inspector"
|
|
},
|
|
"homepage": "https://github.com/firecrawl/pdf-inspector",
|
|
"publishConfig": {
|
|
"access": "public"
|
|
},
|
|
"napi": {
|
|
"binaryName": "pdf-inspector",
|
|
"targets": [
|
|
"x86_64-unknown-linux-gnu",
|
|
"aarch64-apple-darwin",
|
|
"x86_64-pc-windows-msvc"
|
|
],
|
|
"package": {
|
|
"name": "@firecrawl/pdf-inspector-js"
|
|
}
|
|
},
|
|
"scripts": {
|
|
"build": "napi build --platform --release",
|
|
"build:debug": "napi build --platform"
|
|
},
|
|
"devDependencies": {
|
|
"@napi-rs/cli": "^3.4.1"
|
|
}
|
|
}
|