extract_tables_in_regions: detect paragraph-as-table misreads (0.4.2)
Adds a 5th failure-mode check to looks_like_partial_table: when the heuristic mis-detects text-wrapped paragraph prose as a multi-column table, cells in the same column tend to start with lowercase letters or continuation punctuation (commas, closing quotes) — because they're actually sentence fragments. Real tables almost never have most data cells starting lowercase. Trigger: ≥2 cols, ≥4 data rows, ≥60% of non-empty data cells start with lowercase or continuation punctuation → return needs_ocr=true. Caught in the eval as the next-largest failure mode after the 0.4.1 fix: PDFs 088, 182, 090 — heuristic produced "tables" like: |Approval is needed from the|Acquisitions of| |Treasurer if the acquisition|residential and| |constitutes a "significant|agricultural| |action," including acquiring an|land by foreign| Reading column 1 top-to-bottom: "Approval is needed from the Treasurer if the acquisition constitutes a 'significant action,' including acquiring an interest..." — a paragraph, not tabular data. Tests: 2 new tests (the 088-style failure case + a real multi-word table that must NOT be flagged). All 11 looks_like_partial_table tests pass; 323 unit + 91 integration tests still green. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
d0dd067e70
commit
780efdb955
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "firecrawl-pdf-inspector",
|
||||
"version": "0.4.1",
|
||||
"version": "0.4.2",
|
||||
"description": "Fast PDF classification and text extraction. Detect text-based vs scanned PDFs, extract text by region with quality checks. Native Rust performance via napi-rs.",
|
||||
"main": "index.js",
|
||||
"types": "index.d.ts",
|
||||
|
||||
Reference in New Issue
Block a user