text_cluster_column_undercount previously fired only on tables with
6+ markdown columns. That missed a common production failure shape:
4-column page geometry where the heuristic detector's x-position
clustering collapses 2 narrow numeric columns (dates, IDs, amounts)
into adjacent wider columns, producing 2-column markdown.
The original 6-column floor existed because raw x-cluster count is
noisy on small tables — wrapped continuations, bullet indents, and
within-cell text variation produce many small x-clusters that don't
correspond to real columns. Examples:
- Pcmso-style 2-column key/value layout with multi-line values
shows 10 raw x-clusters but only 1 has more than a single item.
- Yale-style archival catalog with 3 columns shows 6 raw
x-clusters but most clusters are single-item continuations.
Replace the raw cluster count with a "significant cluster" count:
clusters whose item count is at least 1/4 of the dominant cluster
(and ≥2 items). That filters the within-cell-variation noise while
preserving signal from real columns, which consistently have one
item per row.
With significant-cluster counting:
- Narrow-undercount fires when significant_clusters >= 3 AND
significant_clusters >= table_cols * 2 (catches 4-col-collapsed-
to-2 cases without misfiring on pcmso 10-clusters or yale 6
where the significant subset matches markdown).
- Legacy wide-undercount path (table_cols >= 6) keeps the same
+2 / 1.2x thresholds, now applied to significant clusters
instead of raw clusters.
Verified against representative regression and must-not-regress
cases from prior PR cycles: narrow-undercount catches genuine
column-drop cases (significant=4, markdown=2 → routes to OCR);
must-not-regress fixtures (italian-gov 6520 chars, pcmso 1733
chars, yale 1754 chars, doc200/182/189 still routing to OCR via
PR #91 guards) unchanged.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PDF Inspector
Fast PDF classification and region-based text extraction for Node.js/Bun. Native Rust performance via napi-rs.
Built by Firecrawl for hybrid OCR pipelines — extract text from PDF structure where possible, fall back to OCR only when needed.
Install
npm install @firecrawl/pdf-inspector
# or
bun add @firecrawl/pdf-inspector
Prebuilt binaries included for linux-x64 and macOS ARM64. No Rust toolchain needed.
API
classifyPdf(buffer: Buffer): PdfClassification
Classify a PDF as TextBased, Scanned, Mixed, or ImageBased (~10-50ms). Returns which pages need OCR.
import { classifyPdf } from '@firecrawl/pdf-inspector'
import { readFileSync } from 'fs'
const pdf = readFileSync('document.pdf')
const result = classifyPdf(pdf)
console.log(result.pdfType) // "TextBased" | "Scanned" | "Mixed" | "ImageBased"
console.log(result.pageCount) // 42
console.log(result.pagesNeedingOcr) // [5, 12, 15] (0-indexed)
console.log(result.confidence) // 0.875
extractTextInRegions(buffer: Buffer, pageRegions: PageRegions[]): PageRegionTexts[]
Extract text within bounding-box regions from a PDF. Designed for hybrid OCR pipelines where a layout model detects regions in rendered page images, and this function extracts text from the PDF structure for text-based pages — skipping GPU OCR.
Each region result includes a needsOcr flag that signals unreliable extraction (empty text, GID-encoded fonts, garbage text, encoding issues).
import { extractTextInRegions } from '@firecrawl/pdf-inspector'
const result = extractTextInRegions(pdf, [
{
page: 0, // 0-indexed
regions: [
[0, 0, 300, 400], // [x1, y1, x2, y2] in PDF points, top-left origin
[300, 0, 612, 400],
]
}
])
for (const region of result[0].regions) {
if (region.needsOcr) {
// Unreliable text — send this region to OCR instead
} else {
console.log(region.text) // Extracted text in reading order
}
}
Types
interface PdfClassification {
pdfType: string // "TextBased" | "Scanned" | "Mixed" | "ImageBased"
pageCount: number
pagesNeedingOcr: number[] // 0-indexed page numbers
confidence: number // 0.0 - 1.0
}
interface PageRegions {
page: number // 0-indexed
regions: number[][] // [[x1, y1, x2, y2], ...] in PDF points, top-left origin
}
interface PageRegionTexts {
page: number
regions: RegionText[]
}
interface RegionText {
text: string
needsOcr: boolean // true when text is unreliable
}
Platforms
| Platform | Architecture | Supported |
|---|---|---|
| Linux | x64 | Yes |
| macOS | ARM64 | Yes |
License
MIT