mu-ref/pdf-inspector
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(text): Improve word spacing, paragraph joining, and page breaks

Browse Source 
- Preserve leading whitespace in text_with_formatting() to fix ~80% of
  missing-space issues (e.g. "holdermeans" -> "holder means")
- Tighten word boundary gap threshold from 0.05 to 0.01 and add
  word-count heuristic to distinguish CID word-level operators from
  Type1 line-level operators, fixing spurious spaces (e.g. "t emporary")
- Replace fixed paragraph threshold (1.8x base_size) with dynamic
  median-based computation for double-spaced documents
- Remove --- page break markers between pages
- Fix page number y-threshold for US Letter (720pt vs 800pt)

Eval: +1.4% mean word_sim, zero char_sim regressions across 63 PDFs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Abimael Martell
2026-02-13 12:55:00 -08:00
co-authored by Claude Opus 4.6
parent b94df22d37
commit 8c4b9d2d07
3 changed files with 122 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/integration_tests.rs
+6 -3
View File
@@ -440,11 +440,14 @@ fn test_markdown_from_items_monospace_code() {
fn test_markdown_from_items_page_breaks() {
use pdf_inspector::markdown::to_markdown_from_items;
let items = vec![
make_text_item("Page 1", 100.0, 700.0, 12.0, 1),
make_text_item("Page 2", 100.0, 700.0, 12.0, 2),
make_text_item("Content on first page", 100.0, 700.0, 12.0, 1),
make_text_item("Content on second page", 100.0, 700.0, 12.0, 2),
];
let md = to_markdown_from_items(items, MarkdownOptions::default());
assert!(md.contains("---")); // Page break marker
// Pages should be separated by blank lines (no --- markers)
assert!(!md.contains("---"));
assert!(md.contains("Content on first page"));
assert!(md.contains("Content on second page"));
}
// ============================================================================