Tagged PDFs carry a structure tree with real heading roles (H1..H6), and
the core already parses it (structure_tree::StructTree) and threads MCIDs
onto TextItem — but neither surfaced through the bindings.
- Expose TextItem.mcid (Option<i64>) through the napi and pyo3 bindings,
matching the core field added with the marked-content extractor.
- Add StructRole::name(), the inverse of from_name, so roles have a
stable string form.
- Add extract_structure_elements / extract_structure_elements_mem to the
core: one (page, mcid, role) entry per marked-content reference, sorted
by (page, mcid), empty for untagged PDFs. Pages are 1-indexed to match
TextItem.page, so results join directly against
extract_text_with_positions output.
- Bind it as extractStructureElements (napi) and
extract_structure_elements / extract_structure_elements_bytes (pyo3),
with type-stub updates in pdf_inspector.pyi.
- Cover the join in Rust integration tests, napi test.mjs, and pytest,
using the existing firecrawl_docs_tagged.pdf fixture (tagged) and
thermo-freon12.pdf (untagged).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(napi): add processPdfAsync, classifyPdfAsync, extractPagesMarkdownAsync
The Node bindings are synchronous, so every call parses on the event
loop thread — up to hundreds of milliseconds of dead loop per document
in a server. Add additive AsyncTask-based variants that run the same
shared implementations on the libuv thread pool and return promises.
The existing synchronous exports keep their names, signatures, and
behaviour; each sync/async pair shares one implementation. Panics in
compute() are caught and surfaced as rejections, matching the sync
error contract.
Closes#336
Co-authored-by: Abimael Martell <abimaelmartell@users.noreply.github.com>
* fix(napi): read async task buffers in place instead of copying
Review feedback on #337: buffer.to_vec() copied the whole PDF on the
event loop before the task was queued, so large inputs still stalled
the loop and doubled peak memory. The tasks now hold the napi Buffer
itself — its ref pins the JS allocation for the task's lifetime and
the backing store is stable, so compute() reads it directly from the
worker thread. Callers must not mutate the buffer until the promise
settles (same contract as Node's async fs APIs); documented on each
export and in the README.
The suggested removal of ts_return_type was checked and rejected:
without it napi-rs generates Promise<unknown> for AsyncTask returns.
A comment now records that finding.
Co-authored-by: Abimael Martell <abimaelmartell@users.noreply.github.com>
* fix(napi): copy async task input on the JS thread for soundness
Review feedback on #337: holding the napi Buffer and reading it from
the libuv worker was unsound. Buffer derefs straight to the JS-side
allocation, so a caller mutating it before the promise settled would
race the worker's reads — undefined behavior, not a recoverable error,
and the documented don't-mutate contract was unenforceable. Deferring
the copy to compute() would not help: any off-thread read races the
same way. The JS thread is the only race-free place to take the copy,
because JS is single-threaded and nothing can mutate the buffer during
the synchronous part of the call.
Revert to an owned Vec<u8> copied at call time. The cost is one memcpy,
negligible next to the parse the async variants exist to unblock. Docs
now state the buffer may be reused or mutated immediately, and a test
locks in the copy semantics by mutating the input while a parse is in
flight.
Co-authored-by: Abimael Martell <abimaelmartell@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Abimael Martell <abimaelmartell@users.noreply.github.com>
* feat: expose per-page markdown extraction to Python and Node (#49)
Implements the feature requested in issue #49: a list-of-pages markdown output
from the Python API. Matching the existing project pattern, the feature lives
in the Rust core and is surfaced through every binding.
- Rust core: `extract_pages_markdown` (path) and `extract_pages_markdown_mem`
(bytes) now take `Option<&[u32]>` — `None` returns every page in document
order; a slice restricts and preserves caller order.
- Python: new `extract_pages_markdown(path, pages=None)` and
`extract_pages_markdown_bytes(data, pages=None)` functions plus
`PageMarkdown` / `PagesExtractionResult` classes; stub file updated.
- Node: `extractPagesMarkdown(buffer, pages?)` — `pages` is now optional.
- Tests: 2 new Rust integration tests, 9 new Python tests, 2 new Node
assertions. All 372 unit + 107 integration + 53 Python tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump version from 1.3.0 to 1.4.0
Minor bump for the new per-page markdown extraction API exposed through
the Python and Node bindings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both bindings now expose the same 6 function families: process, detect,
classify, extractText, extractTextWithPositions, and extractTextInRegions.
Bumps PyO3 from 0.22 to 0.25 for Python 3.14 support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Abimael Martell