* fix: recover from a corrupted startxref pointer
Fixes#228.
A PDF whose startxref pointer has been corrupted to point at the wrong
byte offset — a single flipped digit, which is what damaged writers
emit in the wild — was entirely unprocessable: every entry point
(classify_pdf, extract_pages_markdown, process_pdf) raised "Invalid
PDF structure", even though the file's object data, real xref table,
and trailer were all completely intact just past the wrong pointer.
Both pypdf and pdfium recover from this by locating the real table
directly instead of trusting the pointer; lopdf doesn't.
Added a new repair candidate (alongside the existing
missing-%%EOF-marker and stripped-leading-bytes repairs in
repair_pdf_container_candidates): scan the buffer for the real,
standalone `xref` keyword and append a corrected trailing
`startxref`/`%%EOF` block. lopdf's own get_xref_start always reads the
*last* `%%EOF` in the final 512 bytes of the buffer and the
`startxref` value immediately before it, so the appended block
transparently supersedes the corrupted one already in the file — no
in-place byte surgery on content the original writer produced.
Scoped to classic (non-stream) xref tables, matching the reported
repro and the common case; a corrupted pointer into a cross-reference
*stream* (`N 0 obj << /Type /XRef ...>>`, some PDF 1.5+ writers) would
need the containing object's number, not just a byte offset — out of
scope here.
Verified against the issue's exact repro (a valid one-page PDF with a
single corrupted byte in its startxref offset): before this fix,
process_pdf/classify_pdf/extract_pages_markdown all raised "Invalid
PDF structure"; after, both the page count and the real extracted text
("Order Detail Report by Account", "WIDGET ASSEMBLY", the dollar
amount) come back correctly. New regression test added.
Full suite (859 tests, 1 new) passes; cargo clippy --all-targets
-- -D warnings unchanged at 28 pre-existing/unrelated errors.
* fix: validate xref table shape and scan in a single reverse pass
Addresses cubic-dev-ai's review of #230.
- P2 (correctness/safety): the recovery candidate trusted the last
standalone "xref" token unconditionally, without confirming it's
actually a cross-reference table. A coincidental "xref" substring
inside unrelated content — a stream, a string, uncompressed
metadata — could get "repaired" against a bogus offset, letting
lopdf load successfully against garbage instead of returning a
clean error: a real failure turned into silent data corruption on
the fallback path. Added looks_like_xref_subsection_header, which
confirms a plausible classic xref subsection header (`<start-id>
<count>`, e.g. "0 6" — the shape every real classic table starts
with) actually follows the candidate token before accepting it.
find_last_valid_xref_table_start now walks backward from the end of
the buffer until it finds a token that both stands alone *and*
validates, rather than accepting the first (rightmost) standalone
match unconditionally.
- P2 (performance): the old scan re-invoked
`buf[..search_end].windows(4).rposition(...)` on a shrinking prefix
every time a candidate token failed the boundary check, which is
quadratic on a pathological buffer with many non-standalone "xref"
occurrences. Rewrote as a single reverse byte-index walk — O(n)
regardless of how many false candidates it has to reject along the
way.
Added direct unit tests on the byte-level scan (more precise than
constructing adversarial full PDFs, and the coincidental-match
scenario can't be represented in an integration-test fixture anyway
since reportlab compresses page content by default): a coincidental
standalone "xref" with no subsection header is rejected; a real
classic table is found; a coincidental match positioned *after* the
real table in the buffer doesn't shadow it; "xref" as a substring of
"startxref" still doesn't match. The original #228 repro (corrupted
startxref pointer, real table otherwise intact) is unaffected —
verified manually in addition to the existing integration test.
Full suite (863 tests, 5 new) passes; cargo clippy --all-targets
-- -D warnings unchanged at 28 pre-existing/unrelated errors.
* fix: reject xref subsection count runs with trailing garbage
looks_like_xref_subsection_header validated that a count run of digits
followed the whitespace separator, but never checked what came after
it. A coincidental "xref\n0 6garbage" in stream/literal content would
still validate as a real subsection header shape and get repaired
against a bogus offset.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Abimael Martell <1450169+abimaelmartell@users.noreply.github.com>
