feat: TSR-aware table extraction (extract_tables_with_structure_mem) (#61)

* feat: TSR-aware table extraction (extract_tables_with_structure_mem)

New public function that consumes raw structure-recovery output (HTML
structure tokens + per-cell bboxes from a model like SLANet) and assembles
markdown tables by pulling cell text from the native PDF — no OCR, no
geometry inference.

Why: the existing extract_tables_in_regions_mem infers grid geometry from
text positions only and can't distinguish merged cells from multiple narrow
columns. Pairing structure recovery from a layout/TSR model with native
PDF text gets perfect text quality with proper row/col/span structure.

- New module src/tables/structured.rs: token state machine, polygon→AABB,
  crop-px→page-pt, rowspan/colspan-aware cell layout, markdown emitter.
  Accepts both 4-element rects and 8-element 4-corner polygons.
- New public extract_tables_with_structure_mem in src/lib.rs that reuses
  extract_page_text_items, region_overlaps_item, and the shared region
  text-collection helper. No existing public function modified.
- napi binding extractTablesWithStructure mirroring the existing
  extractTablesInRegions shape (f64 in JS → f32 internally).
- 14 unit tests + 5 integration tests, including a real-PDF gold-standard
  match against bits_pilani_feedback.pdf.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* TSR follow-ups: header-aware separator, cells API, v1.6.0

- cells_to_markdown emits the separator after the LAST row that contains
  is_header=true cells, falling back to "after row 0" when no header is
  flagged. Multi-row theads now render correctly. Three new unit tests
  cover: multi-row header, header not on row 0, no headers (fallback).
- New public extract_tables_with_structure_cells_mem returning
  Vec<Vec<StructuredCell>> so callers can drive their own rendering or
  debug overlays without re-doing the parse + extraction. The markdown
  variant now wraps it. The previously-unused page_pt_bbox field is
  surfaced through this API.
- New napi binding extractTablesWithStructureCells + StructuredCellJs.
- Bump @firecrawl/pdf-inspector to 1.6.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Abimael Martell
2026-04-26 00:55:39 -07:00
committed by GitHub
co-authored by Claude Opus 4.7
parent 5c4c6e8d33
commit f6d5e214f1
6 changed files with 1370 additions and 4 deletions
+2
View File
@@ -9,6 +9,7 @@ mod detect_struct;
mod financial;
mod format;
mod grid;
pub mod structured;
pub use detect_heuristic::detect_tables;
pub(crate) use detect_heuristic::is_table_of_contents;
@@ -17,6 +18,7 @@ pub(crate) use detect_rects::cluster_rects;
pub use detect_rects::{detect_tables_from_rects, RectHintRegion};
pub use detect_struct::detect_tables_from_struct_tree;
pub use format::table_to_markdown;
pub use structured::{cells_to_markdown, StructuredCell};
use crate::types::TextItem;