pdf-inspector

Fast Rust library for PDF inspection, classification, and text extraction. Intelligently detects scanned vs text-based PDFs to enable smart routing decisions.

Supported Features

Category	Feature	Description
Detection	Fast Classification	~10-50ms by sampling content streams
	PDF Types	TextBased, Scanned, ImageBased, Mixed
	Confidence Scoring	0.0-1.0 scale for classification certainty
	Configurable Thresholds	Tune sampling depth and detection sensitivity
	Metadata Extraction	Document title from PDF Info dictionary
Text Extraction	Plain Text	Direct extraction from text-based PDFs
	Position-Aware	Text with X/Y coordinates, font info, page numbers
	Multi-Column Support	Automatic detection and proper reading order
	Text Encoding	UTF-16BE, UTF-8, and Latin-1
	ToUnicode CMap	Proper decoding of CID-keyed fonts (Type0/Identity-H)
	Linearized PDFs	Raw stream extraction for optimized PDFs
Headers	Auto Detection	H1-H4 based on font size ratios
Lists	Bullet Points	•, -, *, ○, ●, ◦
	Numbered Lists	1., 1), (1)
	Letter Lists	a., a), (a)
Code Blocks	Monospace Fonts	Courier, Consolas, Monaco, Menlo, Fira Code, JetBrains Mono
	Keyword Detection	Language keywords and syntax patterns
Tables	Region Detection	Automatic table boundary identification
	Column/Row Detection	Position clustering for structure
	Markdown Output	Proper alignment and formatting
	Footnotes	Extraction and formatting
Text Processing	Subscript/Superscript	Font size and Y-offset detection
	Hyphenation Fixing	Rejoins words broken across lines
	Page Number Filtering	Removes isolated page numbers
	URL Formatting	Converts URLs to markdown links
	Drop Cap Merging	Handles large initial letters

Output Formats

Format	Description
Markdown	Headers, lists, code blocks, tables, page breaks
Plain Text	Basic text extraction
JSON	Metadata with type, confidence, page count, timing
Positioned Items	Low-level text with coordinates and font info

CLI Tools

Tool	Description
pdf2md	Convert PDF to Markdown (supports --json output)
detect-pdf	Detect PDF type without conversion (supports --json output)

API Overview

Functions

Function	Description
process_pdf / process_pdf_mem	Detect, extract, and convert to markdown
detect_pdf_type / detect_pdf_type_mem	Fast type detection only
extract_text / extract_text_mem	Plain text extraction
extract_text_with_positions	Text with coordinates
to_markdown	Convert text to markdown

Types

Type	Description
PdfType	TextBased, Scanned, ImageBased, Mixed
PdfProcessResult	Full result with text, markdown, and metadata
PdfTypeResult	Detection result with type, confidence, page count
TextItem	Text with position, font info, and page number
TextLine	Grouped items on the same line
MarkdownOptions	Configuration for markdown conversion
DetectionConfig	Configuration for PDF type detection
PdfError	Io, Parse, Encrypted, InvalidStructure

How Detection Works

1. Load only metadata (xref table, trailer, page count)
2. Sample first ~5 pages' content streams
3. Scan raw bytes for Tj/TJ (text) and Do (image) operators
4. Classify based on text operator presence

This allows detecting 300+ page PDFs in milliseconds.

License

MIT