Compare commits

...
Author SHA1 Message Date
Abimael Martell dfe64b80e7 fix(layout): re-guard the ungated XY-cut fallthrough on table pages
Removing page_has_table from the relative-valley condition also
unlocked the XY-cut fallback inside that block, which has no prose
gate — a table page whose valley candidate was just rejected could
take an unvalidated split. The guard is restored on that call
specifically; the relative-valley path keeps its prose-gated access.
Also rewrite the stale dense-page comment for the 30-item floor.
Bench and corpus unchanged (462/884, corpus byte-identical).
2026-08-18 16:30:27 -07:00
Abimael Martell 2401a77c5f fix(layout): lower the relative-valley floor to sparse pages
The 100-item floor guarded against shallow histogram dips on sparse
pages, but OCR'd multi-column pages produce few long line-runs (a
two-column French academic page arrives as ~60 items) and were falling
to single-column Y-sorting, weaving their columns line by line. The
prose gate inside relative-valley acceptance is the real defense
against spurious dips; 30 items is enough for it to judge.
2026-08-18 16:18:00 -07:00
Abimael Martell 7a374a3867 fix(layout): keep relative-valley column detection on table pages
The page_has_table guard predates item claiming: by grouping time the
detected table's items have already left the flow, so a table cannot
fake a gutter in the histogram this guard protects, and the prose gate
inside relative-valley acceptance rejects any residual table-shaped
split. Without the fallback, the prose remainder of a table-bearing
two-column page fell to single-column Y-sorting and its columns
interleaved line by line.
2026-08-18 16:10:50 -07:00
+21 -8
View File
@@ -225,11 +225,19 @@ pub(crate) fn detect_columns(
// Justified text can leave gutter bins non-empty because item widths extend
// to the column edge. Look for local minima that are significantly lower
// than the peaks on either side.
// Only attempt this for dense pages (>=100 items) — sparse pages with shallow
// histogram dips are likely not multi-column.
// Skip on pages with detected tables — table column gaps look like gutters
// in the histogram but the table pipeline already handles reading order.
if valleys.is_empty() && page_items.len() >= 100 && !page_has_table {
//
// The 30-item floor admits sparse pages: OCR'd multi-column pages arrive
// as few long line-runs and were falling to single-column Y-sorting.
// Below 30 items the histogram is too shallow for even the prose gate
// to judge a dip.
//
// Pages with detected tables take the relative-valley path too: the
// table's items have already left the flow by the time grouping runs,
// so a table cannot fake a gutter here, and the prose gate below
// rejects any residual table-shaped split. Without this, the prose
// REMAINDER of a table-bearing two-column page falls to single-column
// Y-sorting and the columns interleave line by line.
if valleys.is_empty() && page_items.len() >= 30 {
let rel_valleys = find_relative_valleys(
&histogram,
num_bins,
@@ -270,9 +278,14 @@ pub(crate) fn detect_columns(
}
}
}
// Try XY-cut fallback before giving up
if let Some(columns) = try_xy_cut_split(&page_items, x_min, x_max, page) {
return columns;
// Try XY-cut fallback before giving up. Unlike the relative-valley
// path above, XY-cut has no prose gate, so the table-page guard
// stays here: without it a table page whose valley candidate was
// just rejected could take an unvalidated split.
if !page_has_table {
if let Some(columns) = try_xy_cut_split(&page_items, x_min, x_max, page) {
return columns;
}
}
return vec![ColumnRegion { x_min, x_max }];
}