Compare commits

...
Author SHA1 Message Date
Abimael MartellandClaude Opus 4.7 329c671de8 Stage 1 exclusive item-to-cell assignment, v1.6.4
Closes the row-merge pattern observed against FNBO branch list at the
College/Fairway boundary: when SLANet emits cells whose y-extents
overlap between consecutive rows, an item whose center fell in the
overlap region got pulled into BOTH cells, producing run-on cells
(e.g. "Kansas Kansas" + concatenated addresses).

Cause: stage 1's "for each cell, gather items inside" loop allowed an
item to match multiple cells. `normalize_cell_bands` reduces overlap
but is biased when cell-mean-center is offset from actual text baseline
(SLANet bboxes are typically taller than their text content), so the
midpoint-clamp can land on the wrong side of the row boundary, and
items at the boundary still match two cells.

Fix: invert the matching. For each PDF text item, find candidate cells
(those whose bbox satisfies tsr_region_contains_item — center inside
OR >=60% overlap on both axes), and assign to the cell whose CENTER
is geometrically closest. Build per-cell text from the assigned items.
Stage 2 (orphan recovery) is unchanged.

Exclusivity prevents item duplication across cells. The closest-center
rule disambiguates the cell-overlap case naturally without aggressive
band clamping. normalize_cell_bands stays — it tightens cells before
matching (smaller overlap → fewer ambiguous candidates) but is no
longer load-bearing for correctness of the overlap case.

Local replay against FNBO via api/scripts/local-tsr-replay.ts (which
exercises the full layout-pod → table-pod → pdf-inspector chain
in-process):

  Pre-1.6.4 (deployed 1.6.3):
    |LITH West|Illinois Illinois|11700 S. IL Route 47, Huntley IL...
    |College|||0534.03|
    |Fairway|Kansas Kansas|4650 College Blvd... 2828 Shawnee Mission...

  Post-1.6.4 (this change):
    |Huntley|Illinois|11700 S. IL Route 47, Huntley IL...|8711.15|
    |LITH West|Illinois|4520 W Algonquin Rd, Lake in the Hills...
    |College|Kansas|4650 College Blvd, Overland Park KS...|0532.01|
    |Fairway|Kansas|2828 Shawnee Mission Pkwy, Fairway KS...|0500.00|

One row (Shawnee in the Kansas section) can still get dropped under
SLANet detection variance — the model occasionally under-detects rows
and emits N structure rows for N+1 PDF rows. The squeezed row's text
gets routed to the structurally-nearest existing cell. This is a
SLANet limitation, not addressable in pdf-inspector without
synthesizing rows from PDF text geometry; deferred.

Tests: 416 lib + 120 integration + 2 doctests pass. clippy + fmt clean.

Bump @firecrawl/pdf-inspector to 1.6.4 (patch — refines stage 1's
matching strategy; no API changes).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:19:26 -07:00
2 changed files with 65 additions and 16 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@firecrawl/pdf-inspector",
"version": "1.6.3",
"version": "1.6.4",
"description": "Fast PDF classification and text extraction. Detect text-based vs scanned PDFs, extract text by region with quality checks. Native Rust performance via napi-rs.",
"main": "index.js",
"types": "index.d.ts",
+64 -15
View File
@@ -931,24 +931,73 @@ pub fn extract_tables_with_structure_cells_mem(
normalize_cell_bands(&mut cells);
// Stage 1: strict text fill — each cell gets the items whose centers
// fall inside its (normalized) bbox or whose >=60% overlap rule fires.
// Track which item indices any cell claimed so the orphan pass below
// doesn't double-assign.
// Stage 1: exclusive per-item assignment. For each PDF text item,
// find the cell(s) whose (band-clamped) bbox satisfies the strict
// membership rule (`tsr_region_contains_item`: center inside OR
// >=60% overlap on both axes). If multiple cells qualify, assign
// the item to the cell whose center is geometrically closest. If
// exactly one qualifies, assign to that. If none, the item is an
// orphan and stage 2 below tries to recover it.
//
// The exclusivity (one item → one cell) prevents the cell-overlap
// bug where SLANet emits cells whose y-extents overlap between
// rows: under the previous "for each cell, gather items" approach,
// an item whose center fell in two cells' overlap got duplicated
// into both. Closest-center disambiguation routes it to the
// correct row.
let mut claimed: std::collections::HashSet<usize> = std::collections::HashSet::new();
for cell in &mut cells {
let [x1, y1, x2, y2] = cell.page_pt_bbox;
let bounds = region_bounds(x1, y1, x2, y2, page_h, coords);
let mut matched: Vec<TextItem> = Vec::new();
for (i, item) in items.iter().enumerate() {
if tsr_region_contains_item(item, bounds) {
claimed.insert(i);
matched.push(item.clone());
let mut item_to_cell: std::collections::HashMap<usize, usize> =
std::collections::HashMap::new();
// Pre-compute each cell's bounds + center (in PDF-pt-flipped space)
// so we don't redo the work per item.
let cell_meta: Vec<Option<(RegionBounds, f32, f32)>> = cells
.iter()
.map(|cell| {
let [x1, y1, x2, y2] = cell.page_pt_bbox;
if x1 >= x2 || y1 >= y2 {
return None;
}
let bounds = region_bounds(x1, y1, x2, y2, page_h, coords);
let cx = (bounds.x_min + bounds.x_max) * 0.5;
let cy = (bounds.y_min + bounds.y_max) * 0.5;
Some((bounds, cx, cy))
})
.collect();
for (item_idx, item) in items.iter().enumerate() {
let item_w = text_utils::effective_width(item);
let item_cx = item.x + item_w * 0.5;
let item_cy = item.y + item.height * 0.5;
let mut best: Option<(usize, f32)> = None;
for (cell_idx, meta) in cell_meta.iter().enumerate() {
let Some((bounds, ccx, ccy)) = meta else {
continue;
};
if !tsr_region_contains_item(item, *bounds) {
continue;
}
let dx = item_cx - ccx;
let dy = item_cy - ccy;
let dist_sq = dx * dx + dy * dy;
if best.is_none_or(|(_, d)| dist_sq < d) {
best = Some((cell_idx, dist_sq));
}
}
// Markdown cells must be one line — collapse line breaks produced
// by the line-grouping pass.
cell.text = collect_text_from_matched_items(matched, adaptive_threshold)
if let Some((ci, _)) = best {
claimed.insert(item_idx);
item_to_cell.insert(item_idx, ci);
}
}
// Build per-cell text from the assigned items. Markdown cells must
// be one line — collapse line breaks from the line-grouping pass.
let mut per_cell_items: Vec<Vec<TextItem>> = vec![Vec::new(); cells.len()];
for (&item_idx, &cell_idx) in &item_to_cell {
per_cell_items[cell_idx].push(items[item_idx].clone());
}
for (cell_idx, matched) in per_cell_items.into_iter().enumerate() {
cells[cell_idx].text = collect_text_from_matched_items(matched, adaptive_threshold)
.replace(['\n', '\r'], " ");
}