fix(tables): Prevent paragraph text from being falsely detected as tables

Add pairwise cross-row column alignment check to find_table_regions_strict().
Real tables have fixed column X positions that repeat across rows (score ~1.0),
while paragraph text has varying word positions (score ~0.3-0.4). Regions with
average pairwise alignment score < 0.5 are now rejected.

Also remove the num_cols >= 5 content-check bypass for BodyFont mode, requiring
body-font tables to always have ≥30% data-like cell content.

Fixes false table detection across 9+ PDFs including AI_Cultural_Collections
(-730 false lines), 131212888 (-740), Data-Processing-Agreement (-108),
CEP_DN_Interest_Rates (-93). No regressions on legitimate tables.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

This commit is contained in:

Abimael Martell

2026-02-12 15:10:10 -08:00

co-authored by

Claude Opus 4.6

parent d0e14fe4cd

commit 56cfd895b5

2 changed files with 119 additions and 25 deletions

									
										Cargo.toml
									
		+4
		
												View File
												
				@@ -41,3 +41,7 @@ path = "src/bin/pdf2md.rs"

				name = "detect-pdf"

				path = "src/bin/detect_pdf.rs"

				[[bin]]

				name = "debug_ygaps"

				path = "src/bin/debug_ygaps.rs"