Abimael MartellandClaude Opus 4.6
579bd4f3b1
Three changes to improve heading quality: 1. Raise heading threshold from 1.1x to 1.2x base font size, reducing false positives where slightly larger body text was promoted to headers. 2. Add word count guard (max 15 words) to skip heading detection for long body paragraphs that happen to use a larger font. 3. Add merge_heading_lines() preprocessing that joins consecutive lines at the same heading level on the same page (e.g., "About Glenair, the Mission-Critical" + "Interconnect Company" → single heading). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
pdf-inspector
Fast Rust library for PDF inspection, classification, and text extraction. Intelligently detects scanned vs text-based PDFs to enable smart routing decisions.
Supported Features
| Category | Feature | Description |
|---|---|---|
| Detection | Fast Classification | ~10-50ms by sampling content streams |
| PDF Types | TextBased, Scanned, ImageBased, Mixed | |
| Confidence Scoring | 0.0-1.0 scale for classification certainty | |
| Configurable Thresholds | Tune sampling depth and detection sensitivity | |
| Metadata Extraction | Document title from PDF Info dictionary | |
| Text Extraction | Plain Text | Direct extraction from text-based PDFs |
| Position-Aware | Text with X/Y coordinates, font info, page numbers | |
| Multi-Column Support | Automatic detection and proper reading order | |
| Text Encoding | UTF-16BE, UTF-8, and Latin-1 | |
| ToUnicode CMap | Proper decoding of CID-keyed fonts (Type0/Identity-H) | |
| Linearized PDFs | Raw stream extraction for optimized PDFs | |
| Headers | Auto Detection | H1-H4 based on font size ratios |
| Lists | Bullet Points | •, -, *, ○, ●, ◦ |
| Numbered Lists | 1., 1), (1) |
|
| Letter Lists | a., a), (a) |
|
| Code Blocks | Monospace Fonts | Courier, Consolas, Monaco, Menlo, Fira Code, JetBrains Mono |
| Keyword Detection | Language keywords and syntax patterns | |
| Tables | Region Detection | Automatic table boundary identification |
| Column/Row Detection | Position clustering for structure | |
| Markdown Output | Proper alignment and formatting | |
| Footnotes | Extraction and formatting | |
| Text Processing | Subscript/Superscript | Font size and Y-offset detection |
| Hyphenation Fixing | Rejoins words broken across lines | |
| Page Number Filtering | Removes isolated page numbers | |
| URL Formatting | Converts URLs to markdown links | |
| Drop Cap Merging | Handles large initial letters |
Output Formats
| Format | Description |
|---|---|
| Markdown | Headers, lists, code blocks, tables, page breaks |
| Plain Text | Basic text extraction |
| JSON | Metadata with type, confidence, page count, timing |
| Positioned Items | Low-level text with coordinates and font info |
CLI Tools
| Tool | Description |
|---|---|
pdf2md |
Convert PDF to Markdown (supports --json output) |
detect-pdf |
Detect PDF type without conversion (supports --json output) |
API Overview
Functions
| Function | Description |
|---|---|
process_pdf / process_pdf_mem |
Detect, extract, and convert to markdown |
detect_pdf_type / detect_pdf_type_mem |
Fast type detection only |
extract_text / extract_text_mem |
Plain text extraction |
extract_text_with_positions |
Text with coordinates |
to_markdown |
Convert text to markdown |
Types
| Type | Description |
|---|---|
PdfType |
TextBased, Scanned, ImageBased, Mixed |
PdfProcessResult |
Full result with text, markdown, and metadata |
PdfTypeResult |
Detection result with type, confidence, page count |
TextItem |
Text with position, font info, and page number |
TextLine |
Grouped items on the same line |
MarkdownOptions |
Configuration for markdown conversion |
DetectionConfig |
Configuration for PDF type detection |
PdfError |
Io, Parse, Encrypted, InvalidStructure |
How Detection Works
- Load only metadata (xref table, trailer, page count)
- Sample first ~5 pages' content streams
- Scan raw bytes for
Tj/TJ(text) andDo(image) operators - Classify based on text operator presence
This allows detecting 300+ page PDFs in milliseconds.
License
MIT