Compare commits

..
Author SHA1 Message Date
Abimael Martell dfe64b80e7 fix(layout): re-guard the ungated XY-cut fallthrough on table pages
Removing page_has_table from the relative-valley condition also
unlocked the XY-cut fallback inside that block, which has no prose
gate — a table page whose valley candidate was just rejected could
take an unvalidated split. The guard is restored on that call
specifically; the relative-valley path keeps its prose-gated access.
Also rewrite the stale dense-page comment for the 30-item floor.
Bench and corpus unchanged (462/884, corpus byte-identical).
2026-08-18 16:30:27 -07:00
Abimael Martell 2401a77c5f fix(layout): lower the relative-valley floor to sparse pages
The 100-item floor guarded against shallow histogram dips on sparse
pages, but OCR'd multi-column pages produce few long line-runs (a
two-column French academic page arrives as ~60 items) and were falling
to single-column Y-sorting, weaving their columns line by line. The
prose gate inside relative-valley acceptance is the real defense
against spurious dips; 30 items is enough for it to judge.
2026-08-18 16:18:00 -07:00
Abimael Martell 7a374a3867 fix(layout): keep relative-valley column detection on table pages
The page_has_table guard predates item claiming: by grouping time the
detected table's items have already left the flow, so a table cannot
fake a gutter in the histogram this guard protects, and the prose gate
inside relative-valley acceptance rejects any residual table-shaped
split. Without the fallback, the prose remainder of a table-bearing
two-column page fell to single-column Y-sorting and its columns
interleaved line by line.
2026-08-18 16:10:50 -07:00
Abimael Martell 0027b048ce fix(tables): reject parallel-prose grids on all unsplit pages (#429)
* fix(tables): reject parallel-prose grids on all unsplit pages

The body-font heuristic pass projects multi-column text pages onto
table grids: on a two-column reference section, every line pair across
the gutter looks like a row with two X-clusters, and the page is
emitted as a woven table. The parallel-prose rejector — which requires
transition evidence (unterminated cells flowing into lowercase starts
in the same column), not mere cell length — was gated to chart pages;
it now runs for every unsplit page.

A compact header row still blocks the rejection, except when cross-row
prose continuations outnumber the rows: no genuine table produces a
continuation on average in every row, so the 'header' there is just
short line fragments atop parallel prose columns.

Band-split retries stay exempt: they exist for tables that only
assemble after recombining bands.

* fix(tables): header bypass requires continuations to strictly outnumber rows

Align the code with its stated rule (the comparison allowed the bypass
at exact equality) and add the dedicated positive-path test: a compact
header atop parallel prose columns whose cross-row continuations
outnumber the rows is flagged as parallel prose. Bench unchanged.
2026-08-18 15:58:20 -07:00
+21 -8
View File
@@ -225,11 +225,19 @@ pub(crate) fn detect_columns(
// Justified text can leave gutter bins non-empty because item widths extend
// to the column edge. Look for local minima that are significantly lower
// than the peaks on either side.
// Only attempt this for dense pages (>=100 items) — sparse pages with shallow
// histogram dips are likely not multi-column.
// Skip on pages with detected tables — table column gaps look like gutters
// in the histogram but the table pipeline already handles reading order.
if valleys.is_empty() && page_items.len() >= 100 && !page_has_table {
//
// The 30-item floor admits sparse pages: OCR'd multi-column pages arrive
// as few long line-runs and were falling to single-column Y-sorting.
// Below 30 items the histogram is too shallow for even the prose gate
// to judge a dip.
//
// Pages with detected tables take the relative-valley path too: the
// table's items have already left the flow by the time grouping runs,
// so a table cannot fake a gutter here, and the prose gate below
// rejects any residual table-shaped split. Without this, the prose
// REMAINDER of a table-bearing two-column page falls to single-column
// Y-sorting and the columns interleave line by line.
if valleys.is_empty() && page_items.len() >= 30 {
let rel_valleys = find_relative_valleys(
&histogram,
num_bins,
@@ -270,9 +278,14 @@ pub(crate) fn detect_columns(
}
}
}
// Try XY-cut fallback before giving up
if let Some(columns) = try_xy_cut_split(&page_items, x_min, x_max, page) {
return columns;
// Try XY-cut fallback before giving up. Unlike the relative-valley
// path above, XY-cut has no prose gate, so the table-page guard
// stays here: without it a table page whose valley candidate was
// just rejected could take an unvalidated split.
if !page_has_table {
if let Some(columns) = try_xy_cut_split(&page_items, x_min, x_max, page) {
return columns;
}
}
return vec![ColumnRegion { x_min, x_max }];
}