Bar charts drawn as filled rects read as cell rects or aligned text:
the cell-rect fallback gridded their axis labels into phantom tables,
and when rect paths rejected them, hint regions and the gap-histogram
heuristic re-gridded the same text. On survey-report pages this
scrambled reading order and swallowed section headings.
Detection (is_chart_bar_cluster): the dominant equal-breadth rect
family arranged in >=2 spaced positions (bars; cell rects touch),
data-driven extent variation (>=1.3x), no same-offset/same-extent
partners across positions (grid rows pair up, chart segments don't),
and only numeric labels inside. Mirrored predicate covers horizontal
bar charts.
Chart clusters are skipped in detect_tables_from_rects (no table, no
hint), and a new detect_chart_regions pass lets the markdown pipeline
pre-claim chart items so heuristic/line/column detection and the
merged-band retry all skip them — the text flows out as plain lines.
opendataloader-bench: overall 0.8389 -> 0.8446, TEDS 0.699 -> 0.708,
MHS 0.742 -> 0.750, NID 0.889 -> 0.894; 7 docs up (best +0.459), none
down. pdf-evals changed-set composite +0.013, TEDS +0.036.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A 4-column compliance table (labels + Small/Medium/Large) was emitted
as two separate tables: split_side_by_side read the text gap between
the ruled and remaining columns as a page-layout gutter, and each band
then detected its own fragment.
Before accepting a side-by-side split, check rect clusters near the
boundary: if a table-shaped cluster spans it (or ends at it with
cell-like text row-aligned beyond), and those table rows account for
the majority of far-side text, the split runs through a table — veto
it. The majority guard keeps legitimate splits on pages where a figure
spans two prose columns.
opendataloader-bench: overall 0.8306 -> 0.8362, TEDS 0.656 -> 0.675,
target doc +0.506, no regressions.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(tables): flatten page-number tables of contents instead of gridding
A title-based contents page ("About the Publisher vii", "Experiment #1
… 3") with no dot leaders and no section numbers was detected as a
2-column data table and rendered as a markdown grid, scrambling the
linear reading order (a top cause of NID loss on affected docs) and
scoring 0 on table structure.
Add is_page_number_toc: a narrow (2-3 col) list whose last column is
mostly page numbers (short integers or roman numerals) that are mostly
non-decreasing, with a text-title first column and NO header row (a
TOC's first row is already an entry). Such tables now route through the
existing flat-list TOC renderer.
The no-header + narrow-width + monotonic guards keep real data tables
intact — e.g. a 4-column regional table, or a 2-column "Mineral | CEC"
table with a header row and ascending values.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: bump reading-order (NID) benchmark to 0.89
Reflects the phantom-TOC fix in this PR: NID 0.88 -> 0.89 on the
200-doc benchmark. Other cells are unchanged at 2-decimal precision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: canonical roman validation, real-first-row header check, roman page cells
- page_number_value now requires a *canonical* roman numeral (re-encode
and compare), so words like "civil"/"mix"/"ill" are no longer parsed
as page numbers.
- The no-header guard checks the actual first row's last cell instead of
the first non-empty one, so a blank header cell ("Category | ") still
rejects the TOC heuristic.
- format::is_page_number_cell recognizes canonical roman numerals, so
roman front-matter pages (vii, ix) get proper title/page separation in
the flat TOC list.
- Fix the non-monotonic test to use 5 rows so it exercises the
monotonicity guard rather than the row-count early return; add
roman-lookalike and blank-header rejection tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: share roman helper, widen length to 8, require page-span for TOC
- Extract canonical_roman_value + to_roman_lower into tables/mod.rs and
use them from both the TOC detector and the formatter, removing the
duplicated mapping/loop and keeping them in sync. The shared helper
accepts ≤8 chars, so longer front-matter numerals (xxxviii) flatten
consistently on both sides.
- Add a page-span guard to is_page_number_toc: real page numbers skip
through the document (range >> entry count), so a dense consecutive
ordinal/rank/ID column (1,2,3,…) is rejected — monotonicity alone did
not separate those data tables from contents.
Costs ~0.001 aggregate on the benchmark (NID 0.888->0.887) for the added
precision; still a clear win over baseline (NID 0.883, TEDS unchanged).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: recover consecutive-page TOCs via a title signal
The strict page-span rule rejected legitimate one-page-per-entry TOCs
(range ~= entry count). Relax it: accept any page sequence with a gap
(nearly all real contents). Only a *perfectly dense* consecutive run —
which rank/ID/ordinal columns produce, but a chapter-per-page TOC can
too — falls back to a title signal: flatten when the first-column
entries average multi-word headings, keep as a table when they are the
short single-word labels typical of leaderboards/ID lists.
Recovers the ~0.001 the range-only rule cost (NID back to 0.888) while
still rejecting dense ordinal data tables.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(extractor): descriptor/embedded-font style flags + geometric strikeout detection
Two style-recall gaps, both invisible to the existing name-based
heuristics:
1. Subset fonts with opaque BaseFont names ("Tc1", "AAAAAB+Amplitude")
defeat is_italic_font/is_bold_font. New descriptor_style_flags reads
the FontDescriptor (ItalicAngle beyond 4 degrees, Flags bit 7 Italic,
bit 19 ForceBold) and, when the descriptor claims upright, falls back
to the embedded font file: ttf-parser's OS/2 fsSelection + post
italicAngle for sfnt fonts, and the CFF Name INDEX PostScript name
for bare-CFF FontFile3 (descriptor rewritten to ItalicAngle 0 while
embedding "Amplitude-LightItalic" was observed in the wild).
ORed into is_bold/is_italic at item creation (content streams and
form XObjects).
2. No strikeout signal existed. New is_strikeout on TextItem, detected
in the same pass as underline: same rules pipeline (stroked lines /
thin filled rects, table-ruling suppression), different vertical
window — a rule crossing the glyphs at 12-55% of the em above the
baseline instead of sitting at it. Exposed through napi and python
bindings and pdf2md --items-json.
Verified on public ParseBench corpus docs: previously-missed italic
council titles and bold CJK itinerary headings now flagged (render-
checked); 24/508 docs gain flags, none lose any; 35 strikeout items
detected corpus-wide, disjoint from underline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): quote-op advance width, Ts text rise, doc-level font style cache (PR #125 review)
Address three valid findings from review:
- The ' (move-to-next-line-and-show-text) operator emitted zero-width
items and never advanced the text matrix, so geometric underline/
strikeout detection (which requires width > 0) could never mark its
text, and following show ops overlapped it. Reuse Tj's advance-width
computation and matrix advance.
- Ts (text rise) was dropped entirely: raised/lowered runs kept the
unshifted baseline, so rules drawn at the risen glyph position missed
the strike/underline windows. Track rise in the text state (saved and
restored with q/Q) and shift the rendering position through the text
matrix's y column; advances stay on the unshifted matrix per spec.
- descriptor_style_flags re-decompressed and re-parsed the same embedded
font program on every page whenever the descriptor left a style flag
unset (the common case). Add a document-scoped FontStyleCache keyed by
the FontFile2/FontFile3 object id, threaded through page and form
extraction alongside the existing CMapDecisionCache.
The fourth finding (Form XObject rules never reach geometric detection)
is real but pre-existing for underline and needs the form walker to grow
path/paint tracking plus a new return type; deferred as a follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): ActualText items render at their glyphs' text rise (PR #125 review)
The EMC-built ActualText item used the captured text matrix without the
rise adjustment the ordinary Tj/TJ/' emission sites apply, so a tagged
run shown with Ts landed on the unshifted baseline — off the strikeout/
underline windows and inconsistent with untagged runs. The rise is
captured together with the first-glyph matrix (and at BDC for the
entry-position fallback): the item must render at the rise of its
GLYPHS, not whatever rise is set by EMC time.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): capture ActualText glyph position after the quote op's line move (PR #125 review)
The `'` handler skipped the entire suppressed-extraction block, so a
tagged span whose show op is `'` never captured its glyph matrix/rise —
the EMC item fell back to the BDC-entry matrix, which sits on the
PREVIOUS line (the `'` line move happens after BDC) with no rise. The
capture now happens right after the line move, matching the Tj/TJ
paths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
* fix(review): style-boundary gate on subscript merge + strikeout suppression coverage (PR #125 review)
merge_subscript_items absorbed a script digit into its parent
regardless of underline/strikeout flags — dropping the digit's own mark
or widening the parent's over it. The merged item carries one flag, so
differing marks now break the merge, mirroring merge_text_items'
style-boundary rule (pre-existing for underline as well).
Also extends the table-suppression test to assert is_strikeout is
cleared alongside is_underline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(extractor): geometric underline detection on TextItem (ENG-5015)
PDFs carry no underline font flag — underlines are stroked horizontal
lines or thin filled rects drawn under the baseline. Correlate those
graphics (already parsed from the content stream) with text items in a
post-pass: a rule within ~0.35em below the baseline covering >=60% of
an item's width marks is_underline.
Exposed through the napi and python bindings. Verified on real docs:
4/4 underlined sentences flagged on a Japanese report, links/headings
flagged on 8 of 10 underline-bearing eval docs, zero flags on docs
without underlines. Known FP source (table cell borders) documented —
downstream applies inline styling only to plain-text regions.
napi 1.9.8 -> 1.9.9.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(extractor): underline rules only from painted rects, normalized extents (review)
Two review fixes: (1) normalize rect extents before the thickness/width
checks — `re` operands pass through the CTM so width/height can be
negative, which missed negative-width rules and let negative-height
bands pass as thin; (2) only feed painted rects to underline detection —
`re` rects now wait in a pending list until a paint operator (S/s, f/F/
f*, B/B*/b/b*) confirms them, and `re W n` clip-only paths are discarded
at `n`, so invisible clip boundaries no longer underline nearby text.
Marking moved into content_stream where paint state lives (pre-rotation,
consistent device space).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(extractor): harden underline detection
* feat(cli): export positioned text item json
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: TSR-aware table extraction (extract_tables_with_structure_mem)
New public function that consumes raw structure-recovery output (HTML
structure tokens + per-cell bboxes from a model like SLANet) and assembles
markdown tables by pulling cell text from the native PDF — no OCR, no
geometry inference.
Why: the existing extract_tables_in_regions_mem infers grid geometry from
text positions only and can't distinguish merged cells from multiple narrow
columns. Pairing structure recovery from a layout/TSR model with native
PDF text gets perfect text quality with proper row/col/span structure.
- New module src/tables/structured.rs: token state machine, polygon→AABB,
crop-px→page-pt, rowspan/colspan-aware cell layout, markdown emitter.
Accepts both 4-element rects and 8-element 4-corner polygons.
- New public extract_tables_with_structure_mem in src/lib.rs that reuses
extract_page_text_items, region_overlaps_item, and the shared region
text-collection helper. No existing public function modified.
- napi binding extractTablesWithStructure mirroring the existing
extractTablesInRegions shape (f64 in JS → f32 internally).
- 14 unit tests + 5 integration tests, including a real-PDF gold-standard
match against bits_pilani_feedback.pdf.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* TSR follow-ups: header-aware separator, cells API, v1.6.0
- cells_to_markdown emits the separator after the LAST row that contains
is_header=true cells, falling back to "after row 0" when no header is
flagged. Multi-row theads now render correctly. Three new unit tests
cover: multi-row header, header not on row 0, no headers (fallback).
- New public extract_tables_with_structure_cells_mem returning
Vec<Vec<StructuredCell>> so callers can drive their own rendering or
debug overlays without re-doing the parse + extraction. The markdown
variant now wraps it. The previously-unused page_pt_bbox field is
surfaced through this API.
- New napi binding extractTablesWithStructureCells + StructuredCellJs.
- Bump @firecrawl/pdf-inspector to 1.6.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The TOC/data-table distinction was being recomputed at every consumer:
- format.rs::table_to_markdown ran is_table_of_contents to decide between
flat-list and markdown-table rendering.
- compute_layout_complexity ran it again to filter TOCs out of
pages_with_tables.
- detect_heuristic validations used it to decide whether to relax val 1/9.
Each caller had to remember tables can be either kind, which leaks the
TOC concept across the codebase.
Add `TableKind { Data, Toc }` and a `Table::new` constructor that classifies
once from the cells. All five detectors (heuristic, rect, line, struct,
columns) now go through `Table::new`. Consumers match on `kind` instead of
re-running classification.
Pure refactor — no behavior change. Verified: pdf-evals output is byte-for-
byte identical (0 changed snapshots).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: detect hierarchical-indent TOCs as tables
Validation 1 (≥25% of rows have first col) and validation 9 (paragraph
content) were rejecting TOC pages where top-level chapters indent at the
leftmost X but subsections cascade further right. The leftmost column ends
up sparse (only chapter rows land there) and most cells are empty, but the
structure is still an unambiguous TOC.
Skip both validations when cells form a TOC pattern AND the table is narrow
(≤5 cols). The width cap preserves existing handling of wide multi-column
TOCs (e.g. back-of-book 2-up indices) where format_toc_as_list would mash
adjacent visual entries together.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: exclude TOC pages from pages_with_tables metadata
TOC detection routes through the table pipeline (detected as a table, then
formatted as a flat list with tab-aligned page numbers via
format_toc_as_list). That made TOC pages appear in LayoutComplexity's
pages_with_tables, which:
- Misleads downstream consumers that read this field as "this page has a
data table".
- Trips the table-page guard in column detection, which switches to a
different valley-detection threshold for table pages.
Add an is_table_of_contents check in compute_layout_complexity so TOC-shaped
detections don't count toward the table flag. The TOC still renders correctly
as a flat list — only the metadata classification changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix sort panics on NaN values from bogus PDF font metrics
Replace all `partial_cmp(...).unwrap_or(Ordering::Equal)` and bare
`partial_cmp(...).unwrap()` with `total_cmp()` across the codebase.
`partial_cmp` returns `None` for NaN, and mapping that to `Equal`
violates total ordering: `a == NaN` and `NaN == b` but `a != b`.
Rust 1.81+ detects this and panics in sort_by. `total_cmp` handles
NaN deterministically (sorts to end) and guarantees total ordering.
The critical crash was in `extract_text_in_regions` (lib.rs:478)
where PDFs with bogus font ascent/descent values produced NaN in
text item coordinates, causing process abort via NAPI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix missed partial_cmp in layout.rs and restore napi exports
- Convert two remaining b.y.partial_cmp(&a.y) calls to total_cmp
in group_single_column and column layout sorting
- Restore missing napi exports: detectPdf, extractText,
extractTextWithPositions, processPdf
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a PDF has a well-formed structure tree with /Table > /TR > /TD|TH
elements linked to MCIDs, build tables directly from the semantic
hierarchy. Runs as highest-priority detection (step 0) before rect-based,
line-based, and heuristic strategies.
- Add StructTree::extract_tables() to walk the tree and collect table
descriptors with row/cell/MCID info
- Add detect_tables_from_struct_tree() to match MCIDs to TextItems
- Reject tables with <30% MCID cell coverage (stale structure trees)
- Update 2013-app2 snapshot (struct-tree gives valid but different
column ordering)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(layout): relative valley column detection for justified text
Add fallback column detection using relative valley analysis for PDFs with
justified text where item widths extend past gutter boundaries. The absolute
valley detector fails on these layouts because gutter bins are at ~40% of
peak (well above the 15% noise threshold).
The relative valley detector smooths the histogram with a 5-bin moving
average, finds local minima where contrast < 0.60 of surrounding peaks,
and validates with peak balance >= 0.40. Limited to single best valley
(max 2 columns) and requires >= 100 items per page.
Tested on IRS Publication 17 (2002), a 289-page 2-column justified text
document: column detection went from ~40 pages to 165 pages.
190 passed, 0 regressions across 191 eval PDFs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(layout): tighten relative valley thresholds to reduce false positives
Reduce PEAK_WINDOW from 40 to 25 bins (50pt) so valleys are only validated
against nearby peaks, not distant ones. Add MIN_PEAK_HEIGHT of 20 (smoothed)
to reject sparse pages where histogram peaks are too low to indicate dense
two-column text.
Previous thresholds caused 13 regressions across the eval suite by splitting
tables, TOCs, checklists, and forms. Now: 188 passed, 0 regressions (2 minor
metadata-only diffs on IRS P17 and 9978293).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(layout): skip relative valley detection on pages with tables
Table column gaps in the histogram look identical to text column gutters
but the table pipeline already handles reading order for those pages.
Pass page_has_table flag through detect_columns to suppress the relative
valley fallback on pages where tables were detected.
This eliminates all remaining regressions from relative valley detection:
190 passed, 0 regressions across 191 eval PDFs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(layout): prose density validation for relative valley detection
Add columns_have_prose() to validate relative valley column splits.
Checks that both sides of a proposed split contain paragraph-like
content (fill ratio >= 40%, avg items/line <= 3.5) before committing
to a column split. Combined with the table-page guard, this prevents
false column splits on financial statements, forms, and tabular
layouts where long labels or dot leaders fill the column width.
Also tightens find_relative_valleys() thresholds (PEAK_WINDOW 40->25,
MIN_PEAK_HEIGHT 5->20) to reduce false positive valley candidates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When rect-based, line-based, and heuristic detection all fail to find
tables on a page, try building a table directly from the layout
engine's column boundaries. Handles borderless tabular layouts like
exam/reference grids where columns are defined purely by text
alignment.
Includes header-row column refinement: when a detected column
contains multiple header items, it gets split at the gap between
them to recover the correct number of columns.
Guards against false positives: requires ≥4 columns, ≤40 rows,
>50% multi-column rows, short cells (avg ≤40 chars), no prose
content, no dominant single column, and no structural elements
(≥6 rects or ≥4 lines) on the page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: tagged PDF structure tree support for semantic markdown generation
Parse /StructTreeRoot from tagged PDFs and use semantic roles (H1-H6, P,
LI, BlockQuote, Code, Caption) to improve markdown output. Structure tree
headings add to font-size heuristics without suppressing them. Coverage
threshold (≥50%) ensures only properly tagged PDFs activate this path.
Phase 1: Parse structure tree with role maps, MCID collection, flattening
Phase 2: Capture MCIDs from BMC/BDC operators, tag TextItems
Phase 3: Structure-aware markdown generation in convert loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: accumulate consecutive code lines into single fenced block
Per-line code fencing produced broken markdown for multi-line code
blocks (separate ``` open/close per line). Unify struct-tree Code
role and font-based monospace detection into a single is_code_line
check with in_code_block state for proper accumulation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add tagged PDF fixture with Firecrawl docs content
Synthetic 7-page PDF with rich structure tree exercising H1, H2, H3,
P, Code, LI, Caption, TH, TD roles. Generated via fpdf2 script.
Integration test verifies struct tree parsing and code fence output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove python PDF generator script from repo
Keep the generated fixture PDF but don't track the generator script.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: handle malformed bare-name struct types in tagged PDFs
Some PDF generators (e.g. fpdf2) write /S Code instead of /S /Code
in structure elements. lopdf silently drops these objects since bare
tokens are invalid PDF syntax. Add a pre-processor that scans for
known bare struct type names and prepends / before loading.
Unifies path and memory loading through the same fix pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update lopdf dependency to main branch
The firecrawl/zlib-checksum-encrypted branch was merged and deleted.
Point to main which includes all previously merged fixes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: switch lopdf to upstream repo pinned at 845cd3d
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Drop column-width padding from table output. Saves ~37% bytes on
table-heavy PDFs. Markdown renders identically — padding was purely
cosmetic. Optimizes token efficiency for AI agent consumers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rect-guided tables:
- Add try_build_rect_guided_table() to build tables from rect cluster
X positions as column boundaries, bypassing heuristic detection
- Split merged multi-number TextItems (e.g. "10 11 12...31") into
individual day-column cells using column-advancing boundary assignment
- Interpolate missing column boundaries for holiday/non-work days
that lack colored rects
- Strip tilde-leader noise from cells (legend text bleeding)
- Filter legend text beyond table area via max-X threshold
- Pass cluster_rects through RectHintRegion for downstream use
Extractor improvements:
- Deduplicate clip-path and fill-path rects before using as table hints
- Add font width fallback for missing glyph metrics via average width
- Improve column detection scoring to prefer balanced gutters
- Handle side-by-side layouts with hint-region derived split points
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Many IRS forms and government PDFs draw table gridlines using path
operators (m/l/S) instead of rectangle (re) operators. This adds
line-based table detection to capture these tables.
- Add PdfLine type for line segments from path operators
- Capture m/l/h/S/s/B/b/f/n path operators in content_stream.rs
- Thread Vec<PdfLine> through extraction pipeline
- New detect_lines.rs: classify lines, snap to grid, validate and
assign items with extensive false-positive filters
- Integrate in markdown pipeline: rects first, then lines as fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Unit test verifying short sub-header rows (e.g. month names) are not
merged as continuation rows in table formatting. Snapshot test for
2013_app2.pdf pinning the full row-stripe table output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a PDF page has a graph directly above a table (e.g. Figure 2 above
Table II in real-estate-pricing.pdf), the heuristic table detector would
merge graph axis labels and legend items into the table, producing a
corrupted result.
Add rect hint regions: when a small cluster of cell-border rects (4-6)
fails full grid validation (e.g. only row borders, no column dividers),
extract their Y bounding box as a "hint region". The heuristic detector
then runs separately on items inside vs outside hint regions, preventing
unrelated content from being merged into tables.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
