Tagged PDFs carry a structure tree with real heading roles (H1..H6), and
the core already parses it (structure_tree::StructTree) and threads MCIDs
onto TextItem — but neither surfaced through the bindings.
- Expose TextItem.mcid (Option<i64>) through the napi and pyo3 bindings,
matching the core field added with the marked-content extractor.
- Add StructRole::name(), the inverse of from_name, so roles have a
stable string form.
- Add extract_structure_elements / extract_structure_elements_mem to the
core: one (page, mcid, role) entry per marked-content reference, sorted
by (page, mcid), empty for untagged PDFs. Pages are 1-indexed to match
TextItem.page, so results join directly against
extract_text_with_positions output.
- Bind it as extractStructureElements (napi) and
extract_structure_elements / extract_structure_elements_bytes (pyo3),
with type-stub updates in pdf_inspector.pyi.
- Cover the join in Rust integration tests, napi test.mjs, and pytest,
using the existing firecrawl_docs_tagged.pdf fixture (tagged) and
thermo-freon12.pdf (untagged).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## Bug 1: Python type stubs missing 5 fields/classes
The pdf_inspector.pyi file was out of sync with the actual Python
bindings exposed via #[pyo3(get)] in src/python.rs. This breaks
IDE autocompletion and type checking (PyCharm, VS Code/Pylance, mypy)
for all Python users.
Added:
- PdfResult.ocr_reasons_by_page (python.rs:35)
- PageOcrReasons class with page and
easons fields (python.rs:67-86)
- RegionText.ocr_reason (python.rs:136)
- PageMarkdown.ocr_reason (python.rs:193)
- PagesExtractionResult.ocr_reasons_by_page (python.rs:226)
## Bug 2: PdfResult.pages_needing_ocr indexing undocumented
PdfResult.pages_needing_ocr is 1-indexed (per python.rs:30) but
neither the .pyi stubs nor docs/python.md annotated this, while the
same field on PdfClassification was annotated as 0-indexed. Users
mixing both APIs would get wrong page numbers.
## Bug 3: README.md duplicate bullet character
The Markdown features table listed * twice in bullet prefixes.
The first should be ullet (U+2022), matching the actual source code in
src/markdown/mod.rs:34 which uses: ullet, dash, sterisk, circle, illed-circle, open-circle.
## Bug 4: docs/python.md missing fields in type reference
The Types section was missing PageOcrReasons, RegionText class
definition, ocr_reason fields, and ocr_reasons_by_page fields.
## Evidence
Cross-referenced every #[pyo3(get)] attribute in src/python.rs
against the .pyi declarations and docs/python.md type reference.
* feat(extractor): descriptor/embedded-font style flags + geometric strikeout detection
Two style-recall gaps, both invisible to the existing name-based
heuristics:
1. Subset fonts with opaque BaseFont names ("Tc1", "AAAAAB+Amplitude")
defeat is_italic_font/is_bold_font. New descriptor_style_flags reads
the FontDescriptor (ItalicAngle beyond 4 degrees, Flags bit 7 Italic,
bit 19 ForceBold) and, when the descriptor claims upright, falls back
to the embedded font file: ttf-parser's OS/2 fsSelection + post
italicAngle for sfnt fonts, and the CFF Name INDEX PostScript name
for bare-CFF FontFile3 (descriptor rewritten to ItalicAngle 0 while
embedding "Amplitude-LightItalic" was observed in the wild).
ORed into is_bold/is_italic at item creation (content streams and
form XObjects).
2. No strikeout signal existed. New is_strikeout on TextItem, detected
in the same pass as underline: same rules pipeline (stroked lines /
thin filled rects, table-ruling suppression), different vertical
window — a rule crossing the glyphs at 12-55% of the em above the
baseline instead of sitting at it. Exposed through napi and python
bindings and pdf2md --items-json.
Verified on public ParseBench corpus docs: previously-missed italic
council titles and bold CJK itinerary headings now flagged (render-
checked); 24/508 docs gain flags, none lose any; 35 strikeout items
detected corpus-wide, disjoint from underline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): quote-op advance width, Ts text rise, doc-level font style cache (PR #125 review)
Address three valid findings from review:
- The ' (move-to-next-line-and-show-text) operator emitted zero-width
items and never advanced the text matrix, so geometric underline/
strikeout detection (which requires width > 0) could never mark its
text, and following show ops overlapped it. Reuse Tj's advance-width
computation and matrix advance.
- Ts (text rise) was dropped entirely: raised/lowered runs kept the
unshifted baseline, so rules drawn at the risen glyph position missed
the strike/underline windows. Track rise in the text state (saved and
restored with q/Q) and shift the rendering position through the text
matrix's y column; advances stay on the unshifted matrix per spec.
- descriptor_style_flags re-decompressed and re-parsed the same embedded
font program on every page whenever the descriptor left a style flag
unset (the common case). Add a document-scoped FontStyleCache keyed by
the FontFile2/FontFile3 object id, threaded through page and form
extraction alongside the existing CMapDecisionCache.
The fourth finding (Form XObject rules never reach geometric detection)
is real but pre-existing for underline and needs the form walker to grow
path/paint tracking plus a new return type; deferred as a follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): ActualText items render at their glyphs' text rise (PR #125 review)
The EMC-built ActualText item used the captured text matrix without the
rise adjustment the ordinary Tj/TJ/' emission sites apply, so a tagged
run shown with Ts landed on the unshifted baseline — off the strikeout/
underline windows and inconsistent with untagged runs. The rise is
captured together with the first-glyph matrix (and at BDC for the
entry-position fallback): the item must render at the rise of its
GLYPHS, not whatever rise is set by EMC time.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): capture ActualText glyph position after the quote op's line move (PR #125 review)
The `'` handler skipped the entire suppressed-extraction block, so a
tagged span whose show op is `'` never captured its glyph matrix/rise —
the EMC item fell back to the BDC-entry matrix, which sits on the
PREVIOUS line (the `'` line move happens after BDC) with no rise. The
capture now happens right after the line move, matching the Tj/TJ
paths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): style-boundary gate on subscript merge + strikeout suppression coverage (PR #125 review)
merge_subscript_items absorbed a script digit into its parent
regardless of underline/strikeout flags — dropping the digit's own mark
or widening the parent's over it. The merged item carries one flag, so
differing marks now break the merge, mirroring merge_text_items'
style-boundary rule (pre-existing for underline as well).
Also extends the table-suppression test to assert is_strikeout is
cleared alongside is_underline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(extractor): geometric underline detection on TextItem (ENG-5015)
PDFs carry no underline font flag — underlines are stroked horizontal
lines or thin filled rects drawn under the baseline. Correlate those
graphics (already parsed from the content stream) with text items in a
post-pass: a rule within ~0.35em below the baseline covering >=60% of
an item's width marks is_underline.
Exposed through the napi and python bindings. Verified on real docs:
4/4 underlined sentences flagged on a Japanese report, links/headings
flagged on 8 of 10 underline-bearing eval docs, zero flags on docs
without underlines. Known FP source (table cell borders) documented —
downstream applies inline styling only to plain-text regions.
napi 1.9.8 -> 1.9.9.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(extractor): underline rules only from painted rects, normalized extents (review)
Two review fixes: (1) normalize rect extents before the thickness/width
checks — `re` operands pass through the CTM so width/height can be
negative, which missed negative-width rules and let negative-height
bands pass as thin; (2) only feed painted rects to underline detection —
`re` rects now wait in a pending list until a paint operator (S/s, f/F/
f*, B/B*/b/b*) confirms them, and `re W n` clip-only paths are discarded
at `n`, so invisible clip boundaries no longer underline nearby text.
Marking moved into content_stream where paint state lives (pre-rotation,
consistent device space).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(extractor): harden underline detection
* feat(cli): export positioned text item json
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: expose per-page markdown extraction to Python and Node (#49)
Implements the feature requested in issue #49: a list-of-pages markdown output
from the Python API. Matching the existing project pattern, the feature lives
in the Rust core and is surfaced through every binding.
- Rust core: `extract_pages_markdown` (path) and `extract_pages_markdown_mem`
(bytes) now take `Option<&[u32]>` — `None` returns every page in document
order; a slice restricts and preserves caller order.
- Python: new `extract_pages_markdown(path, pages=None)` and
`extract_pages_markdown_bytes(data, pages=None)` functions plus
`PageMarkdown` / `PagesExtractionResult` classes; stub file updated.
- Node: `extractPagesMarkdown(buffer, pages?)` — `pages` is now optional.
- Tests: 2 new Rust integration tests, 9 new Python tests, 2 new Node
assertions. All 372 unit + 107 integration + 53 Python tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump version from 1.3.0 to 1.4.0
Minor bump for the new per-page markdown extraction API exposed through
the Python and Node bindings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both bindings now expose the same 6 function families: process, detect,
classify, extractText, extractTextWithPositions, and extractTextInRegions.
Bumps PyO3 from 0.22 to 0.25 for Python 3.14 support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expose the pdf-inspector Rust library as a Python package using PyO3 + maturin.
Python users can now `pip install` and use `import pdf_inspector` for PDF
classification, text extraction, and markdown conversion with native Rust speed.
Adds:
- src/python.rs: PyO3 bindings (process_pdf, detect_pdf, extract_text, etc.)
- pyproject.toml: maturin build configuration
- pdf_inspector.pyi: type stubs for IDE support
- tests/test_python.py: 21 pytest tests covering all Python API functions
- examples/basic_usage.py: example script demonstrating all features
- Updated README with Python quick start and API reference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Abimael Martell