extract_tables: reject font-decode failures via text-density floor (#92)
Some PDFs have fonts whose ToUnicode CMap is missing or broken —
Identity-H fonts without unicode metadata, Type-3 fonts where every
glyph maps to garbage. The page extractor returns punctuation-only
fragments or single-glyph repeats; the rendered image still carries
the visible text, so the region should fall back to OCR rather than
serve a partial table.
The existing captured_only_a_fragment guard can't catch this case
because region_text_chars itself collapses under font-decode failure —
captured vs extracted is symmetrically low, and the ratio still looks
acceptable.
Add a complementary area-based density guard: when a region has lots
of pixel real estate but very few text chars, the page extractor
hit a font failure. Bbox area is independent of extraction success,
so the symmetry breaks.
Threshold 0.003 chars/sq pt sits between observed clean extractions
(≥0.005 on full-page A4 ledgers, key/value layouts, archival
catalogs) and observed font-decode failures (≤0.0014 on prod-traffic
samples). Three guards keep it from misfiring:
- text_chars < 20 skipped: synthetic / fragmentary fixtures
- area < 30,000 sq pt skipped: tiny stat blocks
- area > 400,000 sq pt skipped: near-whole-A4 bboxes where
density is unreliable (large white-space margins)
Verified against three reproducible cases from prod shadow logs
that previously served partial output:
- Cyrillic page with punctuation-only decode (46 chars, density
0.00045) → flagged, routes to OCR
- Cyrillic page where every glyph collapsed to one letter (89
chars, density 0.00025) → flagged, routes to OCR
- Materials-test region where text extracted fine but the table
body extends beyond the bbox (96 chars, density 0.00131)
→ flagged, routes to OCR
Existing fixtures (full-page A4 ledger, multi-row key/value with
paragraph values, archival catalog, bits_pilani whole-page tests,
synthetic line-grid test) all retain identical behavior.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
36445595d0
commit
647ea5c7cd
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@firecrawl/pdf-inspector",
|
||||
"version": "1.8.13",
|
||||
"version": "1.8.14",
|
||||
"description": "Fast PDF classification and text extraction. Detect text-based vs scanned PDFs, extract text by region with quality checks. Native Rust performance via napi-rs.",
|
||||
"main": "index.js",
|
||||
"types": "index.d.ts",
|
||||
|
||||
Reference in New Issue
Block a user