fix: improve table detection for numeric columns and multi-line headers (#35)
* fix: improve heuristic table detection for numeric columns and multi-line headers Two fixes for tables that have clean extractable text but fail heuristic structure detection: 1. Numeric column merge pass (grid.rs): After initial X-position clustering, adjacent clusters are merged when one is sparse (header text) and the other is dense with >50% numeric items (data column). Multi-line wrapped headers often land slightly offset from their data column — the merge closes gaps within 1.5× the clustering threshold. New is_numeric_text() helper matches decimals, percentages, negative numbers, and comma-separated thousands. 2. Duplicate-header skip (detect_heuristic.rs): Spanning super-headers like "First Degree | First Degree | Higher Degree" contain duplicate cells that trigger looks_like_partial_table_ex rejection. Now skips rows with duplicate cells when a better header candidate exists within the next 3 rows (higher fill ratio or numeric cells). Tested on BITS Pilani university report (430 pages, 314 table pages). Page 4 (multi-line header + numeric data) previously returned needs_ocr=true; now correctly detects the table structure. Eval: 197 PDFs, zero regressions, all 104+ tests pass, zero clippy. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * bump version to 0.7.1 --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
35445c3208
commit
7c8b09be67
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "firecrawl-pdf-inspector",
|
||||
"version": "0.7.0",
|
||||
"version": "0.7.1",
|
||||
"description": "Fast PDF classification and text extraction. Detect text-based vs scanned PDFs, extract text by region with quality checks. Native Rust performance via napi-rs.",
|
||||
"main": "index.js",
|
||||
"types": "index.d.ts",
|
||||
|
||||
Reference in New Issue
Block a user