refactor(tounicode): Replace raw byte scanning with lopdf document model for CMap extraction

Eliminates double-parsing of PDFs by using the lopdf document model exclusively
for ToUnicode CMap extraction. The old raw byte scanner parsed PDFs separately
from lopdf and only handled FlateDecode, while lopdf handles FlateDecode + LZW +
ASCII85. The new approach walks page fonts and Form XObject fonts via the document
API, yielding ~7x speedup on text-heavy PDFs and ~1.2x overall.

- Add FontCMaps::from_doc() with recursive Form XObject font walking
- Remove ~270 lines of raw byte scanning code (from_pdf_bytes, extract_stream_from_raw_pdf, etc.)
- Remove flate2 dependency (lopdf handles decompression internally)
- Remove dead CMap lookup fallback branches (by_name, get_with_obj, base_font_name)
- Remove unused font_base_names parameter from extract_text_from_operand
- Skip U+FFFD replacement characters in CMap decode (PDF notdef glyph markers)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

This commit is contained in:

Abimael Martell

2026-02-18 13:33:32 -08:00

co-authored by

Claude Opus 4.6

parent 64d2438e2e

commit a00ce46ab0

6 changed files with 158 additions and 371 deletions

									
										Cargo.toml
									
		-3
	
												View File
												
				@@ -12,9 +12,6 @@ repository = "https://github.com/firecrawl/pdf-inspector"

				# PDF parsing

				lopdf = { git = "https://github.com/J-F-Liu/lopdf", features = ["rayon"] }

				# Compression

				flate2 = "1.0"

				# Error handling

				thiserror = "2.0"