Files
Abimael Martell 841513d3fb
CI / Test (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Clippy (push) Has been cancelled
CI / Build (macos-latest) (push) Has been cancelled
CI / Build (ubuntu-latest) (push) Has been cancelled
CI / OCR (macos-latest) (push) Has been cancelled
CI / OCR (ubuntu-latest) (push) Has been cancelled
CI / OCR (windows-latest) (push) Has been cancelled
CI / OCR runtime smoke (push) Has been cancelled
CI / WebAssembly (push) Has been cancelled
fix(extract): pass page lines to per-page markdown conversion (#434)
* fix(layout): keep relative-valley column detection on table pages

The page_has_table guard predates item claiming: by grouping time the
detected table's items have already left the flow, so a table cannot
fake a gutter in the histogram this guard protects, and the prose gate
inside relative-valley acceptance rejects any residual table-shaped
split. Without the fallback, the prose remainder of a table-bearing
two-column page fell to single-column Y-sorting and its columns
interleaved line by line.

* fix(layout): lower the relative-valley floor to sparse pages

The 100-item floor guarded against shallow histogram dips on sparse
pages, but OCR'd multi-column pages produce few long line-runs (a
two-column French academic page arrives as ~60 items) and were falling
to single-column Y-sorting, weaving their columns line by line. The
prose gate inside relative-valley acceptance is the real defense
against spurious dips; 30 items is enough for it to judge.

* fix(layout): re-guard the ungated XY-cut fallthrough on table pages

Removing page_has_table from the relative-valley condition also
unlocked the XY-cut fallback inside that block, which has no prose
gate — a table page whose valley candidate was just rejected could
take an unvalidated split. The guard is restored on that call
specifically; the relative-valley path keeps its prose-gated access.
Also rewrite the stale dense-page comment for the 30-item floor.
Bench and corpus unchanged (462/884, corpus byte-identical).

* fix(extract): pass page lines to per-page markdown conversion

The per-page extraction path (extract_pages_markdown_mem_impl, used by the
pages API and every --ocr auto run) partitioned each page's rects but
passed an empty slice for PDF lines to markdown conversion, while the
extracted all_lines sat unused. Every table only the line-based detector
finds (text-anchor rule tables, ruled grids) was silently dropped in that
mode, even though the whole-document path emitted it fine.

Partition all_lines per page like rects and pass them through. Regression
test: a rule-anchored table fixture must survive the pages API.
2026-08-19 09:45:21 -07:00
..