fix TSR cell text assignment for overlapping bboxes

Made-with: Cursor

.github/workflows/publish.yml

@@ -2,14 +2,41 @@ name: Publish npm package
 
 on:
   push:
-    tags: ['v*']
+    branches: [main]
+    paths: ['napi/package.json']
 
 permissions:
   contents: read
   id-token: write
 
 jobs:
+  check-version:
+    name: Check version change
+    runs-on: ubuntu-latest
+    outputs:
+      changed: ${{ steps.check.outputs.changed }}
+      version: ${{ steps.check.outputs.version }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 2
+
+      - name: Check if version changed
+        id: check
+        run: |
+          NEW_VERSION=$(node -p "require('./napi/package.json').version")
+          OLD_VERSION=$(git show HEAD~1:napi/package.json | node -p "JSON.parse(require('fs').readFileSync('/dev/stdin','utf8')).version")
+          echo "old=$OLD_VERSION new=$NEW_VERSION"
+          if [ "$NEW_VERSION" != "$OLD_VERSION" ]; then
+            echo "changed=true" >> "$GITHUB_OUTPUT"
+            echo "version=$NEW_VERSION" >> "$GITHUB_OUTPUT"
+          else
+            echo "changed=false" >> "$GITHUB_OUTPUT"
+          fi
+
   build:
+    needs: check-version
+    if: needs.check-version.outputs.changed == 'true'
     name: Build ${{ matrix.target }}
     runs-on: ${{ matrix.os }}
     strategy:
@@ -19,6 +46,8 @@ jobs:
             target: x86_64-unknown-linux-gnu
           - os: macos-14
             target: aarch64-apple-darwin
+          - os: windows-latest
+            target: x86_64-pc-windows-msvc
     steps:
       - uses: actions/checkout@v4
 
@@ -68,7 +97,7 @@ jobs:
 
   publish:
     name: Publish to npm
-    needs: build
+    needs: [check-version, build]
     runs-on: ubuntu-latest
     permissions:
       contents: read

.gitignore

@@ -35,3 +35,8 @@ scripts/
 # Test output
 test_output/
 
+# Python
+__pycache__/
+*.pyc
+.pytest_cache/
+

AGENTS.md

@@ -0,0 +1,79 @@
+# pdf-inspector
+
+Fast PDF text extraction to structured Markdown. CLI binary: `pdf2md`. Detection binary: `detect-pdf`.
+
+## Build & Test
+
+```bash
+cargo fmt                                    # format
+cargo clippy -- -D warnings                  # lint (enforced, zero warnings)
+cargo test                                   # unit + integration tests (267+ unit, 73+ integration)
+cargo build --release                        # release binary for benchmarks
+```
+
+All three must pass before committing.
+
+## Binaries
+
+- `pdf2md` — extract PDF → Markdown. Supports `--json` for structured output.
+- `detect-pdf` — classify PDF type (TextBased/Scanned/Mixed/ImageBased). Supports `--analyze --json`.
+
+## Architecture
+
+```
+src/
+  lib.rs                        – public API, process_pdf_with_options, encoding issue detection
+  detector.rs                   – PDF type classification, tiled-scan detection, page sampling
+  types.rs                      – TextItem, TextLine, PdfRect, PdfLine
+  tounicode.rs                  – CMap/ToUnicode parsing, CID decoding
+  text_utils.rs                 – CJK/RTL handling, Otsu threshold, ligature expansion, NFKC
+  extractor/
+    mod.rs                      – top-level extraction orchestrator
+    content_stream.rs           – PDF operator state machine (Tj/TJ/Td/Tm/q/Q)
+    fonts.rs                    – font width/encoding, CMapDecisionCache, TrueType cmap fallback
+    layout.rs                   – column detection (histogram), newspaper/tabular classification,
+                                  spanning-line pre-masking, sidebar detection
+  tables/
+    detect_rects.rs             – rect-based table detection (union-find clustering)
+    detect_heuristic.rs         – heuristic table detection (gap-histogram, body-font tables)
+    detect_lines.rs             – line-based table detection (H/V line grids)
+    grid.rs                     – column/row boundaries, cell assignment
+    format.rs                   – table→Markdown formatting, continuation row merging
+  markdown/
+    convert.rs                  – core line→Markdown loop, struct-tree role support
+    analysis.rs                 – font stats, heading tiers, paragraph thresholds
+    classify.rs                 – line classification (header, list, code, caption)
+    preprocess.rs               – drop cap merging, heading line merging
+    postprocess.rs              – dot leaders, hyphenation, page numbers, URL formatting
+```
+
+## Key design decisions
+
+- **Primary audience is AI agents.** Output optimized for token efficiency and semantic quality, not visual formatting. No cosmetic padding.
+- **Three table detection strategies** run in priority order: rect-based → line-based → heuristic. First valid result wins.
+- **Column detection** uses horizontal projection histograms with valley detection. Multi-item spanning lines (titles, headers) are pre-masked using column-aware thresholds before column assignment.
+- **Newspaper vs tabular** classification determines reading order: newspaper reads columns sequentially, tabular Y-interleaves them.
+- **Tiled-scan detection** catches scanned PDFs with JBIG2/strip images where no single tile exceeds the template threshold but aggregate area does (≥2M pixels).
+- **Garbage text upgrade** reclassifies Mixed PDFs as Scanned when extracted text is <50% alphanumeric.
+- **Tagged PDF support** uses structure tree roles (H1-H6, P, L, Code, BlockQuote) when available, falling back to font-size heuristics.
+
+## Testing
+
+- **Unit tests**: inline `#[cfg(test)] mod tests` in each module with synthetic data.
+- **Integration tests**: `tests/integration_tests.rs` with fixture PDFs in `tests/fixtures/`.
+- **Regression suite**: sibling repo `pdf-evals` with 179+ snapshot PDFs. Run `cargo build --release` then `bench.py test` in that repo before committing.
+
+## Debugging
+
+```bash
+RUST_LOG=pdf_inspector::extractor::layout=debug cargo run --bin pdf2md -- file.pdf
+RUST_LOG=pdf_inspector::tables=debug cargo run --bin pdf2md -- file.pdf
+RUST_LOG=pdf_inspector::detector=debug cargo run --release --bin detect-pdf -- file.pdf
+```
+
+## Conventions
+
+- Clippy: use `is_some_and(...)` not `map_or(false, ...)`
+- lopdf quirk: `ParseError` is private — match by string for `InvalidFileHeader`
+- Column limit for tables: 25 (wide statistical tables)
+- `propagate_merged_cells` skipped for >10 columns (spanning rects = background fills)

CLAUDE.md

@@ -61,7 +61,8 @@ src/
 
 - **Unit tests**: inline `#[cfg(test)] mod tests` in each module with synthetic data.
 - **Integration tests**: `tests/integration_tests.rs` with fixture PDFs in `tests/fixtures/`.
-- **Regression suite**: sibling repo `pdf-evals` with 179+ snapshot PDFs. Run `cargo build --release` then `bench.py test` in that repo before committing.
+- **Regression suite**: sibling repo `pdf-evals` with 187+ snapshot PDFs. Run `cargo build --release` then `bench.py test` in that repo before committing.
+- **Semantic quality**: run `bench.py score` in `pdf-evals` for the semantic verdict (TEDS + MHS + reading order + char/word + list preservation, composited). Character-level diff alone misclassifies structural improvements (e.g., column-detection rewrites) as regressions — `score` is the tie-breaker. See `pdf-evals/CLAUDE.md` "Semantic scoring".
 
 ## Debugging
 

README.md

@@ -55,12 +55,12 @@ print(result.markdown)   # Markdown string or None
 ### Node.js
 
 ```bash
-npm install @firecrawl/pdf-inspector-js
+npm install @firecrawl/pdf-inspector
 ```
 
 ```javascript
 import { readFileSync } from 'fs';
-import { processPdf, classifyPdf } from '@firecrawl/pdf-inspector-js';
+import { processPdf, classifyPdf } from '@firecrawl/pdf-inspector';
 
 const result = processPdf(readFileSync('document.pdf'));
 console.log(result.pdfType);   // "TextBased", "Scanned", "ImageBased", "Mixed"

docs/python.md

@@ -42,6 +42,14 @@ text = pdf_inspector.extract_text("document.pdf")
 items = pdf_inspector.extract_text_with_positions("document.pdf")
 for item in items[:5]:
     print(f"'{item.text}' at ({item.x:.0f}, {item.y:.0f}) size={item.font_size}")
+
+# Per-page markdown (one Markdown string per page, plus layout metadata)
+result = pdf_inspector.extract_pages_markdown("document.pdf")
+for page in result.pages:
+    print(f"Page {page.page}: {len(page.markdown)} chars, needs_ocr={page.needs_ocr}")
+
+# Restrict to specific 0-indexed pages (preserves caller order)
+result = pdf_inspector.extract_pages_markdown("document.pdf", pages=[0, 2])
 ```
 
 ## API reference
@@ -60,6 +68,8 @@ for item in items[:5]:
 | `extract_text_with_positions_bytes(data, pages=None)` | Text with positions from bytes |
 | `extract_text_in_regions(path, page_regions)` | Extract text in bounding-box regions |
 | `extract_text_in_regions_bytes(data, page_regions)` | Region extraction from bytes |
+| `extract_pages_markdown(path, pages=None)` | Per-page Markdown + layout metadata (all pages by default) |
+| `extract_pages_markdown_bytes(data, pages=None)` | Per-page Markdown from bytes |
 
 ## Types
 
@@ -72,3 +82,7 @@ for item in items[:5]:
 **`RegionText` fields:** `text`, `needs_ocr`
 
 **`PageRegionTexts` fields:** `page` (0-indexed), `regions` (list of RegionText)
+
+**`PageMarkdown` fields:** `page` (0-indexed), `markdown`, `needs_ocr`
+
+**`PagesExtractionResult` fields:** `pages` (list of PageMarkdown), `pages_with_tables` (1-indexed), `pages_with_columns` (1-indexed), `pages_needing_ocr` (1-indexed), `is_complex`

docs/rust-api.md

@@ -79,6 +79,27 @@ let bytes = std::fs::read("document.pdf")?;
 let result = process_pdf_mem(&bytes)?;
 ```
 
+Extract per-page Markdown (one string per page, plus document-wide layout
+metadata):
+
+```rust
+use pdf_inspector::extract_pages_markdown;
+
+// Pass `None` for every page in document order, or a slice of 0-indexed
+// pages to restrict the output (caller-supplied order is preserved).
+let result = extract_pages_markdown("document.pdf", None)?;
+
+for page in &result.pages {
+    if page.needs_ocr {
+        // Route this page to OCR
+    } else {
+        println!("Page {}: {}", page.page, page.markdown);
+    }
+}
+
+println!("Complex layout? {}", result.is_complex);
+```
+
 ## Processing modes
 
 | Mode | What it does | Returns |
@@ -102,6 +123,8 @@ let result = process_pdf_mem(&bytes)?;
 | `to_markdown(text, options)` | Convert plain text to Markdown |
 | `to_markdown_from_items(items, options)` | Markdown from pre-extracted `TextItem`s |
 | `to_markdown_from_items_with_rects(items, options, rects)` | Markdown with rectangle-based table detection |
+| `extract_pages_markdown(path, pages)` | Per-page Markdown + layout metadata (file) |
+| `extract_pages_markdown_mem(bytes, pages)` | Per-page Markdown from bytes |
 
 Low-level detection functions are also available via the `detector` module (`detect_pdf_type`, `detect_pdf_type_with_config`, etc.) for callers who need `PdfTypeResult` instead of `PdfProcessResult`.
 
@@ -119,4 +142,6 @@ Low-level detection functions are also available via the `detector` module (`det
 | `LayoutComplexity` | Layout analysis: is_complex, pages_with_tables, pages_with_columns |
 | `TextItem` | Text with position, font info, and page number |
 | `MarkdownOptions` | Configuration for Markdown formatting (page numbers, etc.) |
+| `PageMarkdown` | Per-page result: page (0-indexed), markdown, needs_ocr |
+| `PagesExtractionResult` | Per-page output + 1-indexed pages_with_tables / pages_with_columns / pages_needing_ocr, is_complex |
 | `PdfError` | `Io`, `Parse`, `Encrypted`, `InvalidStructure`, `NotAPdf` |

napi/README.md

@@ -1,4 +1,4 @@
-# firecrawl-pdf-inspector
+# PDF Inspector
 
 Fast PDF classification and region-based text extraction for Node.js/Bun. Native Rust performance via [napi-rs](https://napi.rs).
 
@@ -7,9 +7,9 @@ Built by [Firecrawl](https://firecrawl.dev) for hybrid OCR pipelines — extract
 ## Install
 
 ```bash
-npm install firecrawl-pdf-inspector
+npm install @firecrawl/pdf-inspector
 # or
-bun add firecrawl-pdf-inspector
+bun add @firecrawl/pdf-inspector
 ```
 
 Prebuilt binaries included for **linux-x64** and **macOS ARM64**. No Rust toolchain needed.
@@ -21,7 +21,7 @@ Prebuilt binaries included for **linux-x64** and **macOS ARM64**. No Rust toolch
 Classify a PDF as TextBased, Scanned, Mixed, or ImageBased (~10-50ms). Returns which pages need OCR.
 
 ```typescript
-import { classifyPdf } from 'firecrawl-pdf-inspector'
+import { classifyPdf } from '@firecrawl/pdf-inspector'
 import { readFileSync } from 'fs'
 
 const pdf = readFileSync('document.pdf')
@@ -40,7 +40,7 @@ Extract text within bounding-box regions from a PDF. Designed for hybrid OCR pip
 Each region result includes a `needsOcr` flag that signals unreliable extraction (empty text, GID-encoded fonts, garbage text, encoding issues).
 
 ```typescript
-import { extractTextInRegions } from 'firecrawl-pdf-inspector'
+import { extractTextInRegions } from '@firecrawl/pdf-inspector'
 
 const result = extractTextInRegions(pdf, [
   {

napi/bin/pdf-inspector.mjs

@@ -0,0 +1,131 @@
+#!/usr/bin/env node
+
+import { readFileSync, writeFileSync } from "fs";
+import { createRequire } from "module";
+
+const require = createRequire(import.meta.url);
+const { version } = require("../package.json");
+
+const HELP = `pdf-inspector v${version} — Fast PDF text extraction to Markdown
+
+Usage:
+  pdf-inspector <file>                  Extract markdown (default)
+  pdf-inspector detect <file>           Classify PDF type
+
+Options:
+  --json                  Output as JSON
+  --pages <pages>         Comma-separated page numbers (e.g. 1,3,5)
+  -o, --output <file>     Write output to file instead of stdout
+  -h, --help              Show this help
+  -v, --version           Show version
+
+Examples:
+  pdf-inspector document.pdf
+  pdf-inspector document.pdf --json
+  pdf-inspector document.pdf --pages 1,2,3
+  pdf-inspector detect document.pdf --json
+  cat document.pdf | pdf-inspector -`;
+
+function die(msg) {
+  process.stderr.write(`error: ${msg}\n`);
+  process.exit(1);
+}
+
+function parseArgs(argv) {
+  const opts = { json: false, pages: null, output: null, file: null, command: "extract" };
+  let i = 0;
+
+  // Check for subcommand
+  if (argv[0] === "detect") {
+    opts.command = "detect";
+    i = 1;
+  }
+
+  while (i < argv.length) {
+    const arg = argv[i];
+    if (arg === "-h" || arg === "--help") {
+      process.stdout.write(HELP + "\n");
+      process.exit(0);
+    } else if (arg === "-v" || arg === "--version") {
+      process.stdout.write(`${version}\n`);
+      process.exit(0);
+    } else if (arg === "--json") {
+      opts.json = true;
+    } else if (arg === "--pages") {
+      i++;
+      if (!argv[i]) die("--pages requires a value (e.g. 1,3,5)");
+      opts.pages = argv[i].split(",").map((p) => {
+        const n = parseInt(p.trim(), 10);
+        if (Number.isNaN(n) || n < 1) die(`invalid page number: ${p}`);
+        return n;
+      });
+    } else if (arg === "-o" || arg === "--output") {
+      i++;
+      if (!argv[i]) die("-o requires a filename");
+      opts.output = argv[i];
+    } else if (arg === "-" || !arg.startsWith("-")) {
+      if (opts.file) die(`unexpected argument: ${arg}`);
+      opts.file = arg;
+    } else {
+      die(`unknown option: ${arg}`);
+    }
+    i++;
+  }
+
+  return opts;
+}
+
+function readInput(file) {
+  if (file === "-") {
+    return readFileSync(0); // stdin fd
+  }
+  try {
+    return readFileSync(file);
+  } catch (err) {
+    if (err.code === "ENOENT") die(`file not found: ${file}`);
+    die(err.message);
+  }
+}
+
+function output(text, outputPath) {
+  if (outputPath) {
+    writeFileSync(outputPath, text);
+  } else {
+    process.stdout.write(text);
+  }
+}
+
+// ---- main ----
+
+const opts = parseArgs(process.argv.slice(2));
+
+if (!opts.file) {
+  // Check if stdin is piped
+  if (process.stdin.isTTY !== false) {
+    process.stderr.write(HELP + "\n");
+    process.exit(1);
+  }
+  opts.file = "-";
+}
+
+const { processPdf, classifyPdf } = await import("../index.js");
+const buffer = readInput(opts.file);
+
+if (opts.command === "detect") {
+  const result = classifyPdf(buffer);
+  if (opts.json) {
+    output(JSON.stringify(result, null, 2) + "\n", opts.output);
+  } else {
+    const ocr = result.pagesNeedingOcr.length > 0
+      ? `, ${result.pagesNeedingOcr.length} pages need OCR`
+      : "";
+    output(`${result.pdfType} (${result.pageCount} pages, confidence: ${result.confidence.toFixed(2)}${ocr})\n`, opts.output);
+  }
+} else {
+  const result = processPdf(buffer, opts.pages ?? undefined);
+  if (opts.json) {
+    output(JSON.stringify(result, null, 2) + "\n", opts.output);
+  } else {
+    output((result.markdown ?? "") + "\n", opts.output);
+  }
+}

napi/package.json

@@ -1,9 +1,12 @@
 {
-  "name": "firecrawl-pdf-inspector",
-  "version": "0.3.6",
+  "name": "@firecrawl/pdf-inspector",
+  "version": "1.6.0",
   "description": "Fast PDF classification and text extraction. Detect text-based vs scanned PDFs, extract text by region with quality checks. Native Rust performance via napi-rs.",
   "main": "index.js",
   "types": "index.d.ts",
+  "bin": {
+    "pdf-inspector": "bin/pdf-inspector.mjs"
+  },
   "license": "MIT",
   "keywords": [
     "pdf",
@@ -20,6 +23,7 @@
     "index.js",
     "index.d.ts",
     "*.node",
+    "bin/",
     "README.md"
   ],
   "repository": {
@@ -34,7 +38,8 @@
     "binaryName": "pdf-inspector",
     "targets": [
       "x86_64-unknown-linux-gnu",
-      "aarch64-apple-darwin"
+      "aarch64-apple-darwin",
+      "x86_64-pc-windows-msvc"
     ],
     "package": {
       "name": "@firecrawl/pdf-inspector-js"

napi/src/lib.rs

@@ -5,6 +5,28 @@ use napi_derive::napi;
 use std::collections::HashSet;
 use std::panic;
 
+// ---------------------------------------------------------------------------
+// Enums
+// ---------------------------------------------------------------------------
+
+/// PDF document type classification.
+#[napi(string_enum)]
+pub enum PdfType {
+    TextBased,
+    Scanned,
+    ImageBased,
+    Mixed,
+}
+
+/// Type of a positioned text item.
+#[napi(string_enum)]
+pub enum ItemType {
+    Text,
+    Image,
+    Link,
+    FormField,
+}
+
 // ---------------------------------------------------------------------------
 // Result types
 // ---------------------------------------------------------------------------
@@ -12,7 +34,7 @@ use std::panic;
 /// Full PDF processing result with markdown and metadata.
 #[napi(object)]
 pub struct PdfResult {
-    pub pdf_type: String,
+    pub pdf_type: PdfType,
     pub markdown: Option<String>,
     pub page_count: u32,
     pub processing_time_ms: u32,
@@ -29,7 +51,7 @@ pub struct PdfResult {
 /// Lightweight PDF classification result.
 #[napi(object)]
 pub struct PdfClassification {
-    pub pdf_type: String,
+    pub pdf_type: PdfType,
     pub page_count: u32,
     /// 0-indexed page numbers that need OCR.
     pub pages_needing_ocr: Vec<u32>,
@@ -49,7 +71,9 @@ pub struct TextItem {
     pub page: u32,
     pub is_bold: bool,
     pub is_italic: bool,
-    pub item_type: String,
+    pub item_type: ItemType,
+    /// URL for link items, `None` for other types.
+    pub link_url: Option<String>,
 }
 
 /// A page's regions for text extraction: (page_index_0based, bboxes).
@@ -79,18 +103,18 @@ pub struct PageRegionTexts {
 // Helpers
 // ---------------------------------------------------------------------------
 
-fn pdf_type_string(t: pdf_inspector::PdfType) -> String {
+fn convert_pdf_type(t: pdf_inspector::PdfType) -> PdfType {
     match t {
-        pdf_inspector::PdfType::TextBased => "TextBased".to_string(),
-        pdf_inspector::PdfType::Scanned => "Scanned".to_string(),
-        pdf_inspector::PdfType::ImageBased => "ImageBased".to_string(),
-        pdf_inspector::PdfType::Mixed => "Mixed".to_string(),
+        pdf_inspector::PdfType::TextBased => PdfType::TextBased,
+        pdf_inspector::PdfType::Scanned => PdfType::Scanned,
+        pdf_inspector::PdfType::ImageBased => PdfType::ImageBased,
+        pdf_inspector::PdfType::Mixed => PdfType::Mixed,
     }
 }
 
 fn to_napi_result(r: pdf_inspector::PdfProcessResult) -> PdfResult {
     PdfResult {
-        pdf_type: pdf_type_string(r.pdf_type),
+        pdf_type: convert_pdf_type(r.pdf_type),
         markdown: r.markdown,
         page_count: r.page_count,
         processing_time_ms: r.processing_time_ms as u32,
@@ -104,12 +128,12 @@ fn to_napi_result(r: pdf_inspector::PdfProcessResult) -> PdfResult {
     }
 }
 
-fn item_type_string(t: &pdf_inspector::types::ItemType) -> String {
+fn convert_item_type(t: &pdf_inspector::types::ItemType) -> (ItemType, Option<String>) {
     match t {
-        pdf_inspector::types::ItemType::Text => "text".into(),
-        pdf_inspector::types::ItemType::Image => "image".into(),
-        pdf_inspector::types::ItemType::Link(url) => format!("link:{url}"),
-        pdf_inspector::types::ItemType::FormField => "form_field".into(),
+        pdf_inspector::types::ItemType::Text => (ItemType::Text, None),
+        pdf_inspector::types::ItemType::Image => (ItemType::Image, None),
+        pdf_inspector::types::ItemType::Link(url) => (ItemType::Link, Some(url.clone())),
+        pdf_inspector::types::ItemType::FormField => (ItemType::FormField, None),
     }
 }
 
@@ -181,7 +205,7 @@ pub fn classify_pdf(buffer: Buffer) -> Result<PdfClassification> {
         let result =
             pdf_inspector::classify_pdf_mem(&bytes).map_err(|e| to_napi_err(e, "classify_pdf"))?;
         Ok(PdfClassification {
-            pdf_type: pdf_type_string(result.pdf_type),
+            pdf_type: convert_pdf_type(result.pdf_type),
             page_count: result.page_count,
             pages_needing_ocr: result.pages_needing_ocr,
             confidence: result.confidence as f64,
@@ -222,18 +246,22 @@ pub fn extract_text_with_positions(
 
         Ok(items
             .into_iter()
-            .map(|item| TextItem {
-                text: item.text,
-                x: item.x as f64,
-                y: item.y as f64,
-                width: item.width as f64,
-                height: item.height as f64,
-                font: item.font,
-                font_size: item.font_size as f64,
-                page: item.page,
-                is_bold: item.is_bold,
-                is_italic: item.is_italic,
-                item_type: item_type_string(&item.item_type),
+            .map(|item| {
+                let (item_type, link_url) = convert_item_type(&item.item_type);
+                TextItem {
+                    text: item.text,
+                    x: item.x as f64,
+                    y: item.y as f64,
+                    width: item.width as f64,
+                    height: item.height as f64,
+                    font: item.font,
+                    font_size: item.font_size as f64,
+                    page: item.page,
+                    is_bold: item.is_bold,
+                    is_italic: item.is_italic,
+                    item_type,
+                    link_url,
+                }
             })
             .collect())
     })
@@ -255,7 +283,240 @@ pub fn extract_text_in_regions(
     page_regions: Vec<PageRegions>,
 ) -> Result<Vec<PageRegionTexts>> {
     let bytes: Vec<u8> = buffer.to_vec();
-    let regions: Vec<(u32, Vec<[f32; 4]>)> = page_regions
+    let regions = parse_page_regions(&page_regions);
+
+    catch_panic("extract_text_in_regions", move || {
+        let results = pdf_inspector::extract_text_in_regions_mem(&bytes, &regions)
+            .map_err(|e| to_napi_err(e, "extract_text_in_regions"))?;
+        Ok(to_page_region_texts(results))
+    })
+}
+
+/// Extract markdown tables within bounding-box regions from a PDF.
+///
+/// Like `extractTextInRegions` but runs table detection on items within each
+/// region and returns markdown pipe-tables instead of flat text.
+///
+/// When table structure is detected, `text` contains a markdown pipe-table and
+/// `needsOcr` is `false`. When no table is found, `text` is empty and
+/// `needsOcr` is `true` so the caller can fall back to GPU OCR.
+///
+/// Coordinates are PDF points with top-left origin.
+#[napi]
+pub fn extract_tables_in_regions(
+    buffer: Buffer,
+    page_regions: Vec<PageRegions>,
+) -> Result<Vec<PageRegionTexts>> {
+    let bytes: Vec<u8> = buffer.to_vec();
+    let regions = parse_page_regions(&page_regions);
+
+    catch_panic("extract_tables_in_regions", move || {
+        let results = pdf_inspector::extract_tables_in_regions_mem(&bytes, &regions)
+            .map_err(|e| to_napi_err(e, "extract_tables_in_regions"))?;
+        Ok(to_page_region_texts(results))
+    })
+}
+
+/// One cropped table region plus its raw structure-recovery output, for
+/// `extractTablesWithStructure`.
+///
+/// `structureTokens` and `cellBboxes` are typically produced by an external
+/// table-structure recognition model (e.g. SLANet on PaddleOCR) running on
+/// a rendered crop of the page. pdf-inspector uses the structure to lay out
+/// the cells and pulls the cell text from the native PDF — no OCR involved.
+#[napi(object)]
+pub struct TsrTableInputJs {
+    /// 0-indexed page number where the crop was taken from.
+    pub page: u32,
+    /// Crop bbox on the page, `[x1, y1, x2, y2]` in PDF points with
+    /// top-left origin.
+    pub crop_pdf_pt_bbox: Vec<f64>,
+    /// DPI the crop image was rendered at (e.g. `200.0`).
+    pub render_dpi: f64,
+    /// Raw structure tokens emitted by the TSR model, in document order.
+    pub structure_tokens: Vec<String>,
+    /// One bbox per cell (in document order). May be 4-element
+    /// `[x1,y1,x2,y2]` or 8-element 4-corner polygon, in crop image-pixel
+    /// space.
+    pub cell_bboxes: Vec<Vec<f64>>,
+}
+
+/// Extract markdown tables using externally-supplied structure recovery.
+///
+/// For each input, pairs structure tokens with cell bboxes (rowspan/colspan
+/// aware), converts each cell bbox from crop image-pixels into page PDF
+/// points, pulls the cell's text from the native PDF, and emits a markdown
+/// pipe-table.
+///
+/// Returns one markdown string per input, in input order.
+#[napi]
+pub fn extract_tables_with_structure(
+    buffer: Buffer,
+    inputs: Vec<TsrTableInputJs>,
+) -> Result<Vec<String>> {
+    let bytes: Vec<u8> = buffer.to_vec();
+    let parsed = parse_tsr_inputs(&inputs);
+
+    catch_panic("extract_tables_with_structure", move || {
+        pdf_inspector::extract_tables_with_structure_mem(&bytes, &parsed)
+            .map_err(|e| to_napi_err(e, "extract_tables_with_structure"))
+    })
+}
+
+/// One resolved cell from `extractTablesWithStructureCells`.
+#[napi(object)]
+pub struct StructuredCellJs {
+    /// 0-indexed grid row.
+    pub row: u32,
+    /// 0-indexed grid column.
+    pub col: u32,
+    /// 1 for a normal cell.
+    pub rowspan: u32,
+    /// 1 for a normal cell.
+    pub colspan: u32,
+    /// `true` when the cell is a `<th>` or sits inside `<thead>`.
+    pub is_header: bool,
+    /// Text extracted from the native PDF for this cell (may be empty).
+    pub text: String,
+    /// Axis-aligned bbox `[x1, y1, x2, y2]` in page PDF-points, top-left
+    /// origin. Useful for debug overlays or per-cell post-processing.
+    pub page_pt_bbox: Vec<f64>,
+}
+
+/// Extract structured cells using externally-supplied structure recovery.
+///
+/// Lower-level sibling of [`extractTablesWithStructure`]: instead of
+/// rendering markdown, returns the resolved cells (row, col, rowspan,
+/// colspan, isHeader, text, pagePtBbox) so callers can drive their own
+/// rendering, debug overlays, or per-cell post-processing.
+///
+/// Returns one `Array<StructuredCellJs>` per input, in input order.
+#[napi]
+pub fn extract_tables_with_structure_cells(
+    buffer: Buffer,
+    inputs: Vec<TsrTableInputJs>,
+) -> Result<Vec<Vec<StructuredCellJs>>> {
+    let bytes: Vec<u8> = buffer.to_vec();
+    let parsed = parse_tsr_inputs(&inputs);
+
+    catch_panic("extract_tables_with_structure_cells", move || {
+        let result = pdf_inspector::extract_tables_with_structure_cells_mem(&bytes, &parsed)
+            .map_err(|e| to_napi_err(e, "extract_tables_with_structure_cells"))?;
+        Ok(result
+            .into_iter()
+            .map(|cells| {
+                cells
+                    .into_iter()
+                    .map(|c| StructuredCellJs {
+                        row: c.row as u32,
+                        col: c.col as u32,
+                        rowspan: c.rowspan as u32,
+                        colspan: c.colspan as u32,
+                        is_header: c.is_header,
+                        text: c.text,
+                        page_pt_bbox: c.page_pt_bbox.iter().map(|v| *v as f64).collect(),
+                    })
+                    .collect()
+            })
+            .collect())
+    })
+}
+
+fn parse_tsr_inputs(inputs: &[TsrTableInputJs]) -> Vec<pdf_inspector::TsrTableInput> {
+    inputs
+        .iter()
+        .map(|i| {
+            let crop = if i.crop_pdf_pt_bbox.len() == 4 {
+                [
+                    i.crop_pdf_pt_bbox[0] as f32,
+                    i.crop_pdf_pt_bbox[1] as f32,
+                    i.crop_pdf_pt_bbox[2] as f32,
+                    i.crop_pdf_pt_bbox[3] as f32,
+                ]
+            } else {
+                [0.0, 0.0, 0.0, 0.0]
+            };
+            let cell_bboxes: Vec<Vec<f32>> = i
+                .cell_bboxes
+                .iter()
+                .map(|bb| bb.iter().map(|v| *v as f32).collect())
+                .collect();
+            pdf_inspector::TsrTableInput {
+                page: i.page,
+                crop_pdf_pt_bbox: crop,
+                render_dpi: i.render_dpi as f32,
+                structure_tokens: i.structure_tokens.clone(),
+                cell_bboxes,
+            }
+        })
+        .collect()
+}
+
+/// Per-page markdown extraction result.
+#[napi(object)]
+pub struct PageMarkdownResult {
+    /// 0-indexed page number.
+    pub page: u32,
+    /// Formatted markdown for this page.
+    pub markdown: String,
+    /// `true` when text on this page is unreliable.
+    pub needs_ocr: bool,
+}
+
+/// Combined per-page markdown extraction and layout classification result.
+#[napi(object)]
+pub struct PagesExtractionResult {
+    /// Per-page markdown results.
+    pub pages: Vec<PageMarkdownResult>,
+    /// 1-indexed pages where tables were detected.
+    pub pages_with_tables: Vec<u32>,
+    /// 1-indexed pages where multi-column layout was detected.
+    pub pages_with_columns: Vec<u32>,
+    /// 1-indexed pages that need OCR (scanned/image-based).
+    pub pages_needing_ocr: Vec<u32>,
+    /// True if any page has tables or columns.
+    pub is_complex: bool,
+}
+
+/// Extract formatted markdown for pages of a PDF, with layout classification
+/// metadata.
+///
+/// Returns per-page markdown and classification data (tables, columns,
+/// OCR needs) from a single parse. Font statistics are computed from the
+/// full document so header detection is consistent across pages.
+///
+/// Omit `pages` (or pass `undefined`) to return every page in document
+/// order. Pass an array of 0-indexed page numbers to restrict output to
+/// those pages, in caller-supplied order.
+#[napi]
+pub fn extract_pages_markdown(
+    buffer: Buffer,
+    pages: Option<Vec<u32>>,
+) -> Result<PagesExtractionResult> {
+    let bytes: Vec<u8> = buffer.to_vec();
+    catch_panic("extract_pages_markdown", move || {
+        let result = pdf_inspector::extract_pages_markdown_mem(&bytes, pages.as_deref())
+            .map_err(|e| to_napi_err(e, "extract_pages_markdown"))?;
+        Ok(PagesExtractionResult {
+            pages: result
+                .pages
+                .into_iter()
+                .map(|r| PageMarkdownResult {
+                    page: r.page,
+                    markdown: r.markdown,
+                    needs_ocr: r.needs_ocr,
+                })
+                .collect(),
+            pages_with_tables: result.pages_with_tables,
+            pages_with_columns: result.pages_with_columns,
+            pages_needing_ocr: result.pages_needing_ocr,
+            is_complex: result.is_complex,
+        })
+    })
+}
+
+fn parse_page_regions(page_regions: &[PageRegions]) -> Vec<(u32, Vec<[f32; 4]>)> {
+    page_regions
         .iter()
         .map(|pr| {
             let bboxes: Vec<[f32; 4]> = pr
@@ -271,25 +532,22 @@ pub fn extract_text_in_regions(
                 .collect();
             (pr.page, bboxes)
         })
-        .collect();
+        .collect()
+}
 
-    catch_panic("extract_text_in_regions", move || {
-        let results = pdf_inspector::extract_text_in_regions_mem(&bytes, &regions)
-            .map_err(|e| to_napi_err(e, "extract_text_in_regions"))?;
-
-        Ok(results
-            .into_iter()
-            .map(|page_result| PageRegionTexts {
-                page: page_result.page,
-                regions: page_result
-                    .regions
-                    .into_iter()
-                    .map(|r| RegionText {
-                        text: r.text,
-                        needs_ocr: r.needs_ocr,
-                    })
-                    .collect(),
-            })
-            .collect())
-    })
+fn to_page_region_texts(results: Vec<pdf_inspector::PageRegionResult>) -> Vec<PageRegionTexts> {
+    results
+        .into_iter()
+        .map(|page_result| PageRegionTexts {
+            page: page_result.page,
+            regions: page_result
+                .regions
+                .into_iter()
+                .map(|r| RegionText {
+                    text: r.text,
+                    needs_ocr: r.needs_ocr,
+                })
+                .collect(),
+        })
+        .collect()
 }

napi/test.mjs

@@ -7,6 +7,7 @@ import {
   extractText,
   extractTextWithPositions,
   extractTextInRegions,
+  extractPagesMarkdown,
 } from './index.js';
 
 const fixture = readFileSync('../tests/fixtures/thermo-freon12.pdf');
@@ -89,6 +90,28 @@ assert.equal(typeof regionResults[0].regions[0].text, 'string');
 assert.equal(typeof regionResults[0].regions[0].needsOcr, 'boolean');
 console.log('  extractTextInRegions: OK');
 
+// --- extractPagesMarkdown ---
+console.log('Testing extractPagesMarkdown...');
+
+// omit pages → every page in document order
+const allPages = extractPagesMarkdown(fixture);
+assert.equal(allPages.pages.length, 3);
+assert.deepEqual(allPages.pages.map(p => p.page), [0, 1, 2]);
+assert.ok(typeof allPages.pages[0].markdown === 'string');
+assert.equal(typeof allPages.pages[0].needsOcr, 'boolean');
+assert.ok(Array.isArray(allPages.pagesWithTables));
+assert.ok(Array.isArray(allPages.pagesWithColumns));
+assert.ok(Array.isArray(allPages.pagesNeedingOcr));
+assert.equal(typeof allPages.isComplex, 'boolean');
+console.log('  extractPagesMarkdown (no pages arg): OK');
+
+// selected pages preserve caller order
+const picked = extractPagesMarkdown(fixture, [2, 0]);
+assert.equal(picked.pages.length, 2);
+assert.equal(picked.pages[0].page, 2);
+assert.equal(picked.pages[1].page, 0);
+console.log('  extractPagesMarkdown with pages: OK');
+
 // --- Error handling ---
 console.log('Testing error handling...');
 assert.throws(() => processPdf(Buffer.from('not a pdf')), /process_pdf/);

pdf_inspector.pyi

@@ -52,6 +52,28 @@ class PageRegionTexts:
     """0-indexed page number."""
     regions: list[RegionText]
 
+class PageMarkdown:
+    """Per-page markdown extraction result."""
+    page: int
+    """0-indexed page number."""
+    markdown: str
+    """Formatted markdown for this page (empty string when needs_ocr is True)."""
+    needs_ocr: bool
+    """True when text on this page is unreliable and OCR should be used instead."""
+
+class PagesExtractionResult:
+    """Per-page markdown output with document-wide layout classification."""
+    pages: list[PageMarkdown]
+    """Per-page markdown results, in the order requested."""
+    pages_with_tables: list[int]
+    """1-indexed pages where tables were detected."""
+    pages_with_columns: list[int]
+    """1-indexed pages where multi-column layout was detected."""
+    pages_needing_ocr: list[int]
+    """1-indexed pages that need OCR."""
+    is_complex: bool
+    """True if any page has tables or multi-column layout."""
+
 def process_pdf(path: str, pages: Optional[list[int]] = None) -> PdfResult:
     """Process a PDF: detect type, extract text, convert to Markdown."""
     ...
@@ -115,3 +137,31 @@ def extract_text_in_regions_bytes(
         page_regions: List of (page_0indexed, [[x1, y1, x2, y2], ...]) tuples.
     """
     ...
+
+def extract_pages_markdown(
+    path: str,
+    pages: Optional[list[int]] = None,
+) -> PagesExtractionResult:
+    """Extract formatted markdown for pages of a PDF, with layout classification.
+
+    Args:
+        path: Path to the PDF file.
+        pages: Optional list of 0-indexed pages. When ``None`` (default), every
+            page is returned in document order. Otherwise, output matches the
+            caller-supplied order.
+
+    Returns:
+        PagesExtractionResult with per-page markdown and document-wide layout
+        classification (tables, columns, OCR needs).
+    """
+    ...
+
+def extract_pages_markdown_bytes(
+    data: bytes,
+    pages: Optional[list[int]] = None,
+) -> PagesExtractionResult:
+    """Extract formatted markdown for pages of a PDF from bytes.
+
+    See :func:`extract_pages_markdown` for details.
+    """
+    ...

src/extractor/content_stream.rs

@@ -216,6 +216,7 @@ pub(crate) fn extract_page_text_items(
     let mut marked_content_stack: Vec<MarkedContentEntry> = Vec::new();
     let mut suppress_glyph_extraction = false;
     let mut actual_text_start_tm: Option<[f32; 6]> = None; // text matrix at BDC entry
+    let mut actual_text_glyph_tm: Option<[f32; 6]> = None; // text matrix at first glyph inside BDC
     /// Get the innermost MCID from the marked content stack.
     fn current_mcid(stack: &[MarkedContentEntry]) -> Option<i64> {
         stack.iter().rev().find_map(|e| e.mcid)
@@ -349,8 +350,15 @@ pub(crate) fn extract_page_text_items(
                             )
                         })
                     });
-                    // ActualText: suppress glyph extraction, just advance text matrix
+                    // ActualText: suppress glyph extraction, just advance text matrix.
+                    // Capture the FIRST glyph's text matrix as the rendering position
+                    // for the ActualText item. Td ops between BDC and the first Tj
+                    // may have moved the position to the correct line — the BDC-entry
+                    // position (actual_text_start_tm) can be on the previous line.
                     if suppress_glyph_extraction {
+                        if actual_text_glyph_tm.is_none() {
+                            actual_text_glyph_tm = Some(text_matrix);
+                        }
                         if let Some(w_ts) = w_ts_opt {
                             text_matrix[4] += w_ts * text_matrix[0];
                             text_matrix[5] += w_ts * text_matrix[1];
@@ -425,6 +433,10 @@ pub(crate) fn extract_page_text_items(
                         let font_info = font_widths.get(&current_font);
                         let is_invisible = (text_rendering_mode == 3 && !include_invisible)
                             || suppress_glyph_extraction;
+                        // Capture first-glyph position for ActualText
+                        if suppress_glyph_extraction && actual_text_glyph_tm.is_none() {
+                            actual_text_glyph_tm = Some(text_matrix);
+                        }
 
                         // Compute space threshold based on font metrics when available
                         let space_threshold = if let Some(font_info) = font_info {
@@ -700,6 +712,7 @@ pub(crate) fn extract_page_text_items(
                 if actual_text.is_some() {
                     suppress_glyph_extraction = true;
                     actual_text_start_tm = Some(text_matrix);
+                    actual_text_glyph_tm = None; // reset — will be captured at first Tj/TJ
                 }
                 marked_content_stack.push(MarkedContentEntry { actual_text, mcid });
             }
@@ -707,8 +720,13 @@ pub(crate) fn extract_page_text_items(
                 // End Marked Content — emit ActualText item with correct width
                 if let Some(entry) = marked_content_stack.pop() {
                     if let Some(at) = entry.actual_text {
-                        // Compute width from text matrix advancement during BDC..EMC
-                        if let Some(start_tm) = actual_text_start_tm.take() {
+                        // Use the first-glyph position (if available) instead of the
+                        // BDC-entry position. Td operators between BDC and the first
+                        // Tj may have moved the text position to the correct line —
+                        // the BDC-entry position can be on the previous line.
+                        let glyph_tm = actual_text_glyph_tm.take();
+                        let entry_tm = actual_text_start_tm.take();
+                        if let Some(start_tm) = glyph_tm.or(entry_tm) {
                             let combined = multiply_matrices(&start_tm, &ctm);
                             if combined[0].abs() >= combined[1].abs() {
                                 rotation_votes.horizontal += 1;

src/extractor/layout.rs

@@ -626,6 +626,33 @@ fn find_relative_valleys(
     valleys
 }
 
+/// Detect whether a side of a gutter consists predominantly of list-marker
+/// glyphs (•, ●, ○, ◦, ▪, ▫, ◆, ◇). A column of bullets on the left margin
+/// creates a spurious histogram valley between the bullet and the content.
+/// Treating it as a real column splits each list item's text across two
+/// "columns," so we reject these candidates.
+fn is_list_marker_column(items: &[&&TextItem]) -> bool {
+    const LIST_MARKERS: &[char] = &['•', '●', '○', '◦', '▪', '▫', '◆', '◇', '■', '□'];
+    if items.is_empty() {
+        return false;
+    }
+    let marker_count = items
+        .iter()
+        .filter(|i| {
+            let t = i.text.trim();
+            let mut chars = t.chars();
+            match (chars.next(), chars.next()) {
+                (Some(c), None) => LIST_MARKERS.contains(&c),
+                _ => false,
+            }
+        })
+        .count();
+    // Require ≥80% of items on this side to be standalone markers. A handful
+    // of non-marker items (stray page numbers, footnote refs) shouldn't
+    // defeat the check.
+    marker_count as f32 / items.len() as f32 >= 0.8
+}
+
 /// Validate valley candidates with vertical consistency checks and build column regions.
 ///
 /// When `center_assign` is true, items are assigned to columns based on their
@@ -694,6 +721,19 @@ fn validate_and_build_columns(
             continue;
         }
 
+        // Reject valleys where the smaller side is just a column of list
+        // markers (bullets aligned at the left margin). This is a common
+        // pattern in PDFs where ● starts each list item: histogram detection
+        // sees the gap between bullet and content as a gutter.
+        let smaller_items: &[&&TextItem] = if left_items.len() <= right_items.len() {
+            &left_items
+        } else {
+            &right_items
+        };
+        if is_list_marker_column(smaller_items) {
+            continue;
+        }
+
         // Check vertical overlap
         if y_range > 0.0 {
             let left_y_min = left_items.iter().map(|i| i.y).fold(f32::INFINITY, f32::min);
@@ -1780,6 +1820,63 @@ mod tests {
         );
     }
 
+    #[test]
+    fn bullet_marker_column_not_detected_as_column() {
+        // Pattern: every line is `● <content>`, with ● at x=90 and content
+        // starting at x=104. Histogram detection sees a gutter between them
+        // and would split the page into a "bullet column" and "content column",
+        // scrambling every list item.
+        let mut items = Vec::new();
+        for i in 0..15 {
+            let y = 750.0 - i as f32 * 30.0;
+            items.push(make_item(1, 90.0, y, "●"));
+            items.push(make_item(
+                1,
+                104.0,
+                y,
+                "FullContentLineTextHere________________",
+            ));
+        }
+        // Pad with content to satisfy min item count for column detection.
+        for i in 0..15 {
+            let y = 300.0 - i as f32 * 14.0;
+            items.push(make_item(1, 72.0, y, "FootnoteText_____________________"));
+        }
+
+        let cols = detect_columns(&items, 1, false);
+        assert_eq!(
+            cols.len(),
+            1,
+            "Bullet markers aligned at left margin should not be treated as their own column"
+        );
+    }
+
+    #[test]
+    fn is_list_marker_column_detects_bullets() {
+        let items = vec![
+            make_item(1, 90.0, 100.0, "●"),
+            make_item(1, 90.0, 114.0, "●"),
+            make_item(1, 90.0, 128.0, "●"),
+            make_item(1, 90.0, 142.0, "●"),
+        ];
+        let refs: Vec<&TextItem> = items.iter().collect();
+        let wrapped: Vec<&&TextItem> = refs.iter().collect();
+        assert!(is_list_marker_column(&wrapped));
+    }
+
+    #[test]
+    fn is_list_marker_column_rejects_prose() {
+        let items = vec![
+            make_item(1, 30.0, 100.0, "Regular prose line"),
+            make_item(1, 30.0, 114.0, "Another sentence"),
+            make_item(1, 30.0, 128.0, "Third line"),
+            make_item(1, 30.0, 142.0, "Fourth line"),
+        ];
+        let refs: Vec<&TextItem> = items.iter().collect();
+        let wrapped: Vec<&&TextItem> = refs.iter().collect();
+        assert!(!is_list_marker_column(&wrapped));
+    }
+
     #[test]
     fn premask_narrow_line_not_masked() {
         // Items that form a line spanning only ~40% of column width → not masked

src/markdown/classify.rs

@@ -64,6 +64,22 @@ pub(crate) fn is_caption_line(text: &str) -> bool {
     false
 }
 
+/// Check if text starts with an unambiguous bullet marker (●, •, ○, ◦).
+///
+/// Narrower than [`is_list_item`]: it excludes numbered/lettered patterns
+/// like `1.` or `a)`, which legitimately appear as section headings in many
+/// documents. Used by the heading classifier to reject bullet lines without
+/// also demoting numbered headings.
+pub(crate) fn starts_with_bullet_marker(text: &str) -> bool {
+    let trimmed = text.trim_start();
+    trimmed.starts_with("• ")
+        || trimmed.starts_with("● ")
+        || trimmed.starts_with("○ ")
+        || trimmed.starts_with("◦ ")
+        || trimmed.starts_with("- ")
+        || trimmed.starts_with("* ")
+}
+
 /// Check if text looks like a list item
 pub(crate) fn is_list_item(text: &str) -> bool {
     let trimmed = text.trim_start();
@@ -115,6 +131,16 @@ pub(crate) fn format_list_item(text: &str) -> String {
         if let Some(rest) = trimmed.strip_prefix(*bullet) {
             return format!("- {}", rest.trim_start());
         }
+        // Bullet inside a leading bold/italic run (e.g. "**● Label:** rest").
+        // The run wraps both the marker and the following label because both
+        // use a bold font in the PDF.
+        for wrapper in ["**", "*"] {
+            if let Some(after_open) = trimmed.strip_prefix(wrapper) {
+                if let Some(rest) = after_open.strip_prefix(*bullet) {
+                    return format!("- {}{}", wrapper, rest.trim_start());
+                }
+            }
+        }
     }
 
     if trimmed.starts_with("- ") || trimmed.starts_with("* ") {
@@ -198,3 +224,42 @@ pub(crate) fn is_monospace_font(font_name: &str) -> bool {
 
     patterns.iter().any(|p| lower.contains(p))
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn format_list_item_plain_bullet() {
+        assert_eq!(format_list_item("● Item"), "- Item");
+        assert_eq!(format_list_item("• Item"), "- Item");
+    }
+
+    #[test]
+    fn format_list_item_bullet_inside_bold() {
+        // PDF that uses bold font for both the marker and the label produces
+        // a single bold run like "**● Label:** rest"; the bullet must still
+        // be stripped and the bold wrapper preserved on the label.
+        assert_eq!(
+            format_list_item("**● Fraud: Willing cooperation;**"),
+            "- **Fraud: Willing cooperation;**"
+        );
+        assert_eq!(
+            format_list_item("**● Label:** rest of line"),
+            "- **Label:** rest of line"
+        );
+        assert_eq!(format_list_item("*● Italic:* rest"), "- *Italic:* rest");
+    }
+
+    #[test]
+    fn format_list_item_already_dash() {
+        assert_eq!(format_list_item("- existing"), "- existing");
+    }
+
+    #[test]
+    fn is_list_item_with_bullet_space() {
+        assert!(is_list_item("● Item"));
+        assert!(is_list_item("• Item"));
+        assert!(is_list_item("- Item"));
+    }
+}

src/markdown/convert.rs

@@ -9,11 +9,64 @@ use super::analysis::{
     bold_heading_level, calculate_font_stats, compute_heading_tiers, compute_paragraph_threshold,
     detect_header_level, font_size_rarity, has_dot_leaders,
 };
-use super::classify::{format_list_item, is_caption_line, is_list_item, is_monospace_font};
+use super::classify::{
+    format_list_item, is_caption_line, is_list_item, is_monospace_font, starts_with_bullet_marker,
+};
 use super::postprocess::clean_markdown;
 use super::preprocess::{merge_drop_caps, merge_heading_lines};
 use super::MarkdownOptions;
 
+/// Pre-scan struct heading tags to find levels that are overused — i.e., tagged on
+/// so many lines that they clearly represent body text, not real headings.
+/// Returns the set of heading levels (1–6) that should be suppressed.
+///
+/// Some PDFs (e.g. British Academy grant guidance) tag every numbered paragraph
+/// line as H2, producing hundreds of false headings. We detect this by checking
+/// if any heading level accounts for >25% of tagged lines.
+fn detect_overused_struct_heading_levels(
+    lines: &[TextLine],
+    struct_roles: Option<
+        &std::collections::HashMap<u32, std::collections::HashMap<i64, StructRole>>,
+    >,
+) -> HashSet<usize> {
+    let mut overused = HashSet::new();
+    let Some(roles) = struct_roles else {
+        return overused;
+    };
+
+    let mut level_counts: HashMap<usize, usize> = HashMap::new();
+    let mut total = 0usize;
+
+    for line in lines {
+        if let Some(role) = resolve_line_struct_role(line, roles) {
+            total += 1;
+            if let Some(level) = struct_role_heading_level(&role) {
+                *level_counts.entry(level).or_insert(0) += 1;
+            }
+        }
+    }
+
+    if total < 20 {
+        return overused;
+    }
+
+    for (&level, &count) in &level_counts {
+        let ratio = count as f32 / total as f32;
+        if ratio > 0.15 {
+            log::debug!(
+                "struct heading H{} overused: {}/{} lines ({:.0}%), suppressing",
+                level,
+                count,
+                total,
+                ratio * 100.0
+            );
+            overused.insert(level);
+        }
+    }
+
+    overused
+}
+
 /// Pre-scan lines to find "isolated" ones: short lines with paragraph breaks both
 /// before and after.  These are heading candidates even at body font size — common
 /// in academic papers ("Acknowledgements", "B.3 Prompt Engineering").
@@ -345,6 +398,9 @@ pub(super) fn to_markdown_from_lines_with_tables_and_images(
     // lookahead in HeadingProcessor (prevNode/nextNode context).
     let isolated_lines = find_isolated_lines(&lines, base_size, para_threshold);
 
+    // Detect struct heading levels that are overused (body text mistagged as headings)
+    let overused_heading_levels = detect_overused_struct_heading_levels(&lines, struct_roles);
+
     let mut output = String::new();
     let mut current_page = 0u32;
     let mut prev_y = f32::MAX;
@@ -526,10 +582,34 @@ pub(super) fn to_markdown_from_lines_with_tables_and_images(
         // Structure roles ADD headings (e.g. same-size text tagged H2) but do NOT
         // suppress headings that the font heuristic would detect (some tagged PDFs
         // mark obvious headings as P or Span).
-        let struct_heading = struct_role.as_ref().and_then(struct_role_heading_level);
+        let struct_heading = struct_role
+            .as_ref()
+            .and_then(struct_role_heading_level)
+            .filter(|level| !overused_heading_levels.contains(level));
+
+        // Protect wrapped list items: when inside a list, a visually-continuing
+        // line (same indent, line-wrap spacing) must not be reclassified as a
+        // heading by the font heuristic — PDFs often bold the lead phrase of a
+        // list item across multiple wrap lines, and an all-bold middle line
+        // would otherwise split one item into a heading + stray body text.
+        // We gate on the document's paragraph threshold so genuine section
+        // headings that follow a numbered paragraph (y_gap > para_threshold)
+        // remain detectable.
+        let looks_like_list_continuation = in_list
+            && match (last_list_x, line.items.first().map(|i| i.x)) {
+                (Some(list_x), Some(curr_x)) => {
+                    let x_ok = curr_x >= list_x - 5.0 && curr_x <= list_x + 50.0;
+                    let y_ok = y_gap >= 0.0 && y_gap <= para_threshold;
+                    x_ok && y_ok && !is_list_item(plain_trimmed)
+                }
+                _ => false,
+            };
+
         let heuristic_heading = if options.detect_headers
+            && !looks_like_list_continuation
             && plain_trimmed.len() > 3
             && plain_trimmed.split_whitespace().count() <= 15
+            && !starts_with_bullet_marker(plain_trimmed)
         {
             let line_font_size = line.items.first().map(|i| i.font_size).unwrap_or(base_size);
             detect_header_level(line_font_size, base_size, &heading_tiers).or_else(|| {
@@ -584,11 +664,16 @@ pub(super) fn to_markdown_from_lines_with_tables_and_images(
             continue;
         }
 
-        // Structure-tree list item (LI only — LBody is a continuation, not a new item)
+        // Structure-tree list item (LI only — LBody is a continuation, not a new item).
+        // Some tagged PDFs use a "flat" style where every wrapped line in a list item
+        // gets its own MCID tagged directly under LI. When we're already inside a list
+        // and the line has no visible bullet marker, treat it as a continuation (falls
+        // through to the continuation logic below) rather than a new list item.
         if struct_role
             .as_ref()
             .is_some_and(|r| matches!(r, StructRole::LI))
             && !is_list_item(plain_trimmed)
+            && !in_list
         {
             if in_paragraph {
                 output.push_str("\n\n");
@@ -1033,6 +1118,59 @@ mod tests {
         );
     }
 
+    #[test]
+    fn test_struct_role_li_flat_continuation_lines_merge() {
+        // Regression: some tagged PDFs put each wrapped visual line of a list
+        // item under its own MCID, all tagged directly as LI. Continuation
+        // lines (no bullet marker) must merge into the bulleted parent item,
+        // not each become their own list item.
+        let make = |text: &str, mcid: i64, x: f32, y: f32| {
+            let mut item = make_item(text, 1, Some(mcid));
+            item.x = x;
+            item.y = y;
+            item
+        };
+        let lines = vec![
+            make_line(vec![make("● First item that wraps onto", 0, 90.0, 322.0)]),
+            make_line(vec![make("a continuation line.", 1, 108.0, 306.0)]),
+            make_line(vec![make("● Second bullet also wraps", 2, 90.0, 290.0)]),
+            make_line(vec![make("to a second line here.", 3, 108.0, 274.0)]),
+        ];
+
+        let mut page_roles = HashMap::new();
+        for mcid in 0..4 {
+            page_roles.insert(mcid, StructRole::LI);
+        }
+        let mut roles = HashMap::new();
+        roles.insert(1u32, page_roles);
+
+        let md = to_markdown_from_lines_with_tables_and_images(
+            lines,
+            MarkdownOptions::default(),
+            HashMap::new(),
+            HashMap::new(),
+            &std::collections::HashSet::new(),
+            Some(&roles),
+        );
+
+        assert!(
+            md.contains("- First item that wraps onto a continuation line."),
+            "continuation should merge into first bullet: {md}"
+        );
+        assert!(
+            md.contains("- Second bullet also wraps to a second line here."),
+            "continuation should merge into second bullet: {md}"
+        );
+        assert!(
+            !md.contains("- a continuation line."),
+            "continuation line should not get its own bullet: {md}"
+        );
+        assert!(
+            !md.contains("- to a second line here."),
+            "continuation line should not get its own bullet: {md}"
+        );
+    }
+
     #[test]
     fn test_struct_role_blockquote() {
         let lines = vec![make_line(vec![make_item("Quoted text", 1, Some(0))])];
@@ -1260,4 +1398,145 @@ mod tests {
             "Should not have adjacent close/open fences: {md}"
         );
     }
+
+    #[test]
+    fn test_overused_struct_heading_suppressed() {
+        // Simulate a PDF where H2 is mistagged on body text lines.
+        // 30 lines total: 5 tagged H1 (real headings), 20 tagged H2 (mistagged body),
+        // 5 tagged P.
+        let mut lines = Vec::new();
+        let mut page_roles = HashMap::new();
+        let mut mcid = 0i64;
+
+        for i in 0..30 {
+            let mut item = make_item(&format!("Line {i}"), 1, Some(mcid));
+            item.y = 700.0 - (i as f32 * 15.0);
+            lines.push(make_line(vec![item]));
+
+            let role = if i < 5 {
+                StructRole::H1
+            } else if i < 25 {
+                StructRole::H2
+            } else {
+                StructRole::P
+            };
+            page_roles.insert(mcid, role);
+            mcid += 1;
+        }
+
+        let mut roles = HashMap::new();
+        roles.insert(1u32, page_roles);
+
+        let overused = detect_overused_struct_heading_levels(&lines, Some(&roles));
+        // H2 is on 20/30 = 67% of lines — should be suppressed
+        assert!(
+            overused.contains(&2),
+            "H2 should be detected as overused: {:?}",
+            overused
+        );
+        // H1 is on 5/30 = 17% — should also be suppressed at >15% threshold
+        assert!(
+            overused.contains(&1),
+            "H1 at 17% should also be suppressed: {:?}",
+            overused
+        );
+    }
+
+    #[test]
+    fn test_normal_struct_headings_not_suppressed() {
+        // Normal document: a few headings, mostly body text
+        let mut lines = Vec::new();
+        let mut page_roles = HashMap::new();
+        let mut mcid = 0i64;
+
+        for i in 0..50 {
+            let mut item = make_item(&format!("Line {i}"), 1, Some(mcid));
+            item.y = 700.0 - (i as f32 * 14.0);
+            lines.push(make_line(vec![item]));
+
+            let role = if i % 10 == 0 {
+                StructRole::H1 // 5 headings out of 50 = 10%
+            } else {
+                StructRole::P
+            };
+            page_roles.insert(mcid, role);
+            mcid += 1;
+        }
+
+        let mut roles = HashMap::new();
+        roles.insert(1u32, page_roles);
+
+        let overused = detect_overused_struct_heading_levels(&lines, Some(&roles));
+        assert!(
+            overused.is_empty(),
+            "No heading level should be overused: {:?}",
+            overused
+        );
+    }
+
+    #[test]
+    fn test_wrapped_bold_lead_in_list_item_not_heading() {
+        // Regression: numbered-list items whose bold "lead" phrase wraps onto
+        // a second line (e.g. definitions in system cards) must not have the
+        // wrapped line reclassified as a heading. The middle line is
+        // all_bold + standalone (in_paragraph=false while in_list), which
+        // previously tripped the rarity heuristic and emitted #### in the
+        // middle of the item, splitting the body into stray bullets.
+        let make = |text: &str, x: f32, y: f32, bold: bool| {
+            let mut item = make_item(text, 1, None);
+            item.x = x;
+            item.y = y;
+            item.is_bold = bold;
+            item
+        };
+
+        let lines = vec![
+            // "1. **bold lead phrase start**"
+            make_line(vec![
+                make("1. ", 72.0, 700.0, false),
+                make(
+                    "Chemical and biological weapons threat model 1 (CB-1): Non-novel",
+                    90.0,
+                    700.0,
+                    true,
+                ),
+            ]),
+            // wrapped continuation of the bold lead — all_bold, same indent
+            make_line(vec![make(
+                "chemical/biological weapons production capabilities: A model has CB-1",
+                90.0,
+                686.0,
+                true,
+            )]),
+            // body text of the same list item
+            make_line(vec![make(
+                "capabilities if it has the ability to significantly help individuals.",
+                90.0,
+                672.0,
+                false,
+            )]),
+        ];
+
+        let md = to_markdown_from_lines_with_tables_and_images(
+            lines,
+            MarkdownOptions::default(),
+            HashMap::new(),
+            HashMap::new(),
+            &std::collections::HashSet::new(),
+            None,
+        );
+
+        assert!(
+            !md.contains("#### "),
+            "wrapped bold lead must not become a heading: {md}"
+        );
+        assert!(
+            md.lines().filter(|l| l.starts_with("- ")).count() == 0,
+            "continuation body must not become a stray bullet: {md}"
+        );
+        assert!(
+            md.contains("1. ") && md.contains("A model has CB-1"),
+            "numbered list item should remain intact: {md}"
+        );
+    }
 }

src/python.rs

@@ -146,6 +146,67 @@ impl PyPageRegionTexts {
 // Text item wrapper
 // ---------------------------------------------------------------------------
 
+/// Per-page markdown extraction result.
+#[pyclass(name = "PageMarkdown")]
+#[derive(Clone)]
+pub struct PyPageMarkdown {
+    /// 0-indexed page number.
+    #[pyo3(get)]
+    pub page: u32,
+    /// Formatted markdown for this page.
+    #[pyo3(get)]
+    pub markdown: String,
+    /// True when text on this page is unreliable (GID-encoded fonts,
+    /// encoding issues, garbage text, or empty extraction).
+    #[pyo3(get)]
+    pub needs_ocr: bool,
+}
+
+#[pymethods]
+impl PyPageMarkdown {
+    fn __repr__(&self) -> String {
+        format!(
+            "PageMarkdown(page={}, markdown='{}', needs_ocr={})",
+            self.page,
+            self.markdown.chars().take(40).collect::<String>(),
+            self.needs_ocr
+        )
+    }
+}
+
+/// Combined per-page markdown extraction and layout classification result.
+#[pyclass(name = "PagesExtractionResult")]
+#[derive(Clone)]
+pub struct PyPagesExtractionResult {
+    /// Per-page markdown results, in the order requested.
+    #[pyo3(get)]
+    pub pages: Vec<PyPageMarkdown>,
+    /// 1-indexed pages where tables were detected.
+    #[pyo3(get)]
+    pub pages_with_tables: Vec<u32>,
+    /// 1-indexed pages where multi-column layout was detected.
+    #[pyo3(get)]
+    pub pages_with_columns: Vec<u32>,
+    /// 1-indexed pages that need OCR (scanned/image-based or unreliable text).
+    #[pyo3(get)]
+    pub pages_needing_ocr: Vec<u32>,
+    /// True if any page has tables or columns.
+    #[pyo3(get)]
+    pub is_complex: bool,
+}
+
+#[pymethods]
+impl PyPagesExtractionResult {
+    fn __repr__(&self) -> String {
+        format!(
+            "PagesExtractionResult(pages={}, pages_with_tables={:?}, is_complex={})",
+            self.pages.len(),
+            self.pages_with_tables,
+            self.is_complex
+        )
+    }
+}
+
 /// A positioned text item extracted from a PDF.
 #[pyclass(name = "TextItem")]
 #[derive(Clone)]
@@ -280,6 +341,24 @@ fn parse_page_regions(
         .collect()
 }
 
+fn to_py_pages_result(r: crate::PagesExtractionResult) -> PyPagesExtractionResult {
+    PyPagesExtractionResult {
+        pages: r
+            .pages
+            .into_iter()
+            .map(|p| PyPageMarkdown {
+                page: p.page,
+                markdown: p.markdown,
+                needs_ocr: p.needs_ocr,
+            })
+            .collect(),
+        pages_with_tables: r.pages_with_tables,
+        pages_with_columns: r.pages_with_columns,
+        pages_needing_ocr: r.pages_needing_ocr,
+        is_complex: r.is_complex,
+    }
+}
+
 fn convert_region_results(results: Vec<crate::PageRegionResult>) -> Vec<PyPageRegionTexts> {
     results
         .into_iter()
@@ -442,6 +521,44 @@ fn extract_text_in_regions_bytes(
     Ok(convert_region_results(results))
 }
 
+/// Extract formatted markdown for pages of a PDF file, with layout
+/// classification metadata.
+///
+/// Returns per-page markdown and classification data (tables, columns,
+/// OCR needs) from a single parse. Font statistics are computed from the
+/// full document so header detection is consistent across pages.
+///
+/// Args:
+///     path: Path to the PDF file.
+///     pages: Optional list of 0-indexed pages. When None (default), every
+///         page is returned in document order. When provided, output
+///         matches the caller-supplied order.
+///
+/// Returns:
+///     PagesExtractionResult with per-page markdown and classification data.
+#[pyfunction]
+#[pyo3(signature = (path, pages=None))]
+fn extract_pages_markdown(
+    path: &str,
+    pages: Option<Vec<u32>>,
+) -> PyResult<PyPagesExtractionResult> {
+    let result = crate::extract_pages_markdown(path, pages.as_deref()).map_err(to_py_err)?;
+    Ok(to_py_pages_result(result))
+}
+
+/// Extract formatted markdown for pages of a PDF from bytes.
+///
+/// See [`extract_pages_markdown`] for details.
+#[pyfunction]
+#[pyo3(signature = (data, pages=None))]
+fn extract_pages_markdown_bytes(
+    data: &[u8],
+    pages: Option<Vec<u32>>,
+) -> PyResult<PyPagesExtractionResult> {
+    let result = crate::extract_pages_markdown_mem(data, pages.as_deref()).map_err(to_py_err)?;
+    Ok(to_py_pages_result(result))
+}
+
 /// Python module definition.
 #[pymodule]
 fn pdf_inspector(m: &Bound<'_, PyModule>) -> PyResult<()> {
@@ -450,6 +567,8 @@ fn pdf_inspector(m: &Bound<'_, PyModule>) -> PyResult<()> {
     m.add_class::<PyTextItem>()?;
     m.add_class::<PyRegionText>()?;
     m.add_class::<PyPageRegionTexts>()?;
+    m.add_class::<PyPageMarkdown>()?;
+    m.add_class::<PyPagesExtractionResult>()?;
     m.add_function(wrap_pyfunction!(process_pdf, m)?)?;
     m.add_function(wrap_pyfunction!(process_pdf_bytes, m)?)?;
     m.add_function(wrap_pyfunction!(detect_pdf, m)?)?;
@@ -462,5 +581,7 @@ fn pdf_inspector(m: &Bound<'_, PyModule>) -> PyResult<()> {
     m.add_function(wrap_pyfunction!(extract_text_with_positions_bytes, m)?)?;
     m.add_function(wrap_pyfunction!(extract_text_in_regions, m)?)?;
     m.add_function(wrap_pyfunction!(extract_text_in_regions_bytes, m)?)?;
+    m.add_function(wrap_pyfunction!(extract_pages_markdown, m)?)?;
+    m.add_function(wrap_pyfunction!(extract_pages_markdown_bytes, m)?)?;
     Ok(())
 }

src/tables/detect_heuristic.rs

@@ -581,8 +581,14 @@ fn detect_table_in_region(items: &[(usize, &TextItem)], mode: TableDetectionMode
     // Validation 1: some rows should have content in first column.
     // Use a lower threshold (25%) for tables with wrapped cells where
     // continuation lines leave the first column empty.
+    // Skip when cells form a narrow TOC pattern: hierarchical entries indented
+    // across multiple X levels leave the leftmost column sparse (only top-level
+    // chapters land there) but the structure is still a valid TOC. Narrow only
+    // (<=5 cols) — wide multi-column TOCs (e.g. 2-up indices) would render
+    // poorly through format_toc_as_list, which assumes one entry per row.
     let rows_with_first_col = cells.iter().filter(|row| !row[0].is_empty()).count();
-    if rows_with_first_col < rows.len() / 4 {
+    let is_narrow_toc = columns.len() <= 5 && is_table_of_contents(&cells);
+    if rows_with_first_col < rows.len() / 4 && !is_narrow_toc {
         log::debug!(
             "  validation 1 fail: {}/{} rows have first col",
             rows_with_first_col,
@@ -653,15 +659,24 @@ fn detect_table_in_region(items: &[(usize, &TextItem)], mode: TableDetectionMode
         return None;
     }
 
-    // Validation 8: Check for Table of Contents pattern
-    if is_table_of_contents(&cells) {
-        log::debug!("  validation 8 fail: table of contents");
+    // Validation 8: Reject paragraph-like content falsely detected as tables.
+    // TOC pages with deep indentation (top-level chapters in col 0, subsections
+    // in cols 1-3, page numbers in last col) leave most cells empty and trip
+    // the paragraph heuristic; TOC shape is a safer signal here. Narrow only
+    // — see narrow-TOC rationale at validation 1.
+    if is_paragraph_content(&cells) && !is_narrow_toc {
+        log::debug!("  validation 9 fail: paragraph content");
         return None;
     }
 
-    // Validation 9: Reject paragraph-like content falsely detected as tables
-    if is_paragraph_content(&cells) {
-        log::debug!("  validation 9 fail: paragraph content");
+    // Validation 9: Reject wide "index" layouts where every cell carries a
+    // full "label ... page" fragment (back-of-book IRS-style indices).
+    // These render poorly in any structured form; text flow is the best
+    // fallback.  Narrow dot-leader TOCs (2-3 cols) are kept so format.rs
+    // can emit them as a per-row flat list with titles tab-joined to page
+    // numbers.
+    if is_inline_leader_index(&cells) {
+        log::debug!("  validation 9 fail: inline-leader index");
         return None;
     }
 
@@ -672,12 +687,7 @@ fn detect_table_in_region(items: &[(usize, &TextItem)], mode: TableDetectionMode
         item_indices.len()
     );
 
-    Some(Table {
-        columns,
-        rows,
-        cells,
-        item_indices,
-    })
+    Some(Table::new(columns, rows, cells, item_indices))
 }
 
 /// Check if this looks like a key-value pair layout rather than a table
@@ -898,77 +908,247 @@ fn looks_like_number(s: &str) -> bool {
         && s.chars().any(|c| c.is_ascii_digit())
 }
 
-/// Check if this looks like a Table of Contents
-/// TOCs have characteristic patterns: leader dots, page numbers, section names
-fn is_table_of_contents(cells: &[Vec<String>]) -> bool {
+/// Check if this looks like a Table of Contents (either style).
+///
+/// Used by format.rs to render TOCs as flat lists instead of markdown tables.
+pub fn is_table_of_contents(cells: &[Vec<String>]) -> bool {
+    is_dot_leader_toc(cells) || is_tabular_toc(cells)
+}
+
+/// Dot-leader TOC: any "Chapter 1 ........ 42" style with explicit leader
+/// dots.  Covers both narrow 2-3 col TOCs (where the leader is a dedicated
+/// cell) and wide indices (where each cell encodes a full "label ... page"
+/// fragment).  Used by format.rs to render as a flat list.
+pub(super) fn is_dot_leader_toc(cells: &[Vec<String>]) -> bool {
+    has_structural_dot_leader(cells) || is_inline_leader_index(cells)
+}
+
+/// Rows with a dedicated dots-only cell flanked by label + number (2-3 col
+/// TOC layout).  Format.rs handles these well via per-row flat-list
+/// rendering; they should NOT be rejected at detect time.
+fn has_structural_dot_leader(cells: &[Vec<String>]) -> bool {
     if cells.is_empty() {
         return false;
     }
+    let structural_rows = cells.iter().filter(|row| row_has_dot_leader(row)).count();
+    structural_rows as f32 / cells.len() as f32 >= 0.3
+}
 
-    let num_cols = cells[0].len();
-    let mut dot_cells = 0;
-    let mut page_number_cells = 0;
-    let mut total_cells = 0;
-    // Track which columns contain dots vs numbers to distinguish
-    // TOC (dots span middle, page number at end) from data tables
-    // (dots only in label column, many number columns).
-    let mut dot_cols = vec![0u32; num_cols];
-    let mut numeric_cols = vec![0u32; num_cols];
-
+/// Wide index layout: each cell holds a full "label ... page" fragment
+/// because the column detector kept multi-column indices as single cells.
+/// These render poorly both as markdown tables (column boundaries are
+/// arbitrary) and as flat lists (each row holds 3+ separate index
+/// entries).  Reject these at detect time so they fall back to the page's
+/// normal text flow.
+pub(super) fn is_inline_leader_index(cells: &[Vec<String>]) -> bool {
+    let mut inline_cells = 0;
+    let mut total_nonempty = 0;
     for row in cells {
-        for (ci, cell) in row.iter().enumerate() {
+        for cell in row {
             let trimmed = cell.trim();
             if trimmed.is_empty() {
                 continue;
             }
-            total_cells += 1;
-
-            // Check for leader dots (sequences of periods)
-            // TOCs often have "........" or ". . . ." patterns
-            let dot_count = trimmed.chars().filter(|&c| c == '.').count();
-            let is_mostly_dots = dot_count > trimmed.len() / 2 && dot_count >= 3;
-            if is_mostly_dots {
-                dot_cells += 1;
-                if ci < num_cols {
-                    dot_cols[ci] += 1;
-                }
-            }
-
-            // Check for standalone page numbers (1-4 digits, possibly with spaces)
-            let digits_only: String = trimmed.chars().filter(|c| !c.is_whitespace()).collect();
-            if digits_only.len() <= 4
-                && !digits_only.is_empty()
-                && digits_only.chars().all(|c| c.is_ascii_digit())
-            {
-                page_number_cells += 1;
-                if ci < num_cols {
-                    numeric_cols[ci] += 1;
-                }
+            total_nonempty += 1;
+            if cell_is_inline_leader(trimmed) {
+                inline_cells += 1;
             }
         }
     }
+    total_nonempty >= 4 && inline_cells as f32 / total_nonempty as f32 >= 0.25
+}
 
-    if total_cells == 0 {
+/// A row with a dot-leader.  Accepts two layouts:
+///   1. A dedicated dots-only cell ("....") with a text label somewhere
+///      to its left and a page number somewhere to its right.
+///   2. A "title ... " cell (trailing leader dots glued to the title)
+///      with a page number elsewhere in the same row.
+fn row_has_dot_leader(row: &[String]) -> bool {
+    let has_page_number = row.iter().any(|c| row_cell_is_page_number(c));
+
+    for (ci, cell) in row.iter().enumerate() {
+        let trimmed = cell.trim();
+
+        // Pattern 1: dedicated dots-only cell.
+        let dot_count = trimmed.chars().filter(|&c| c == '.').count();
+        let is_mostly_dots = dot_count >= 3
+            && dot_count > trimmed.len() / 2
+            && trimmed.chars().all(|c| c == '.' || c.is_whitespace());
+        if is_mostly_dots {
+            let has_label_left = row[..ci].iter().any(|c| {
+                let t = c.trim();
+                !t.is_empty() && t.chars().any(|ch| ch.is_alphabetic())
+            });
+            if has_label_left && has_page_number {
+                return true;
+            }
+            continue;
+        }
+
+        // Pattern 2: cell ends with a trailing " ... " run after a label.
+        if has_page_number && cell_has_trailing_leader(trimmed) {
+            return true;
+        }
+    }
+    false
+}
+
+/// Cell ends with a run of ≥3 dots preceded by alphabetic text and a
+/// space — the "Title ... " layout where the leader is glued to the name.
+/// Alphabetic (not alphanumeric) so that data-table row labels like
+/// "1973 ... " do not register as titles.
+fn cell_has_trailing_leader(cell: &str) -> bool {
+    let trimmed = cell.trim_end();
+    if !trimmed.ends_with('.') {
+        return false;
+    }
+    let without_dots = trimmed.trim_end_matches('.');
+    let dot_run = trimmed.len() - without_dots.len();
+    if dot_run < 3 {
+        return false;
+    }
+    // Require a space before the dot run (rules out "etc..." / "Mr...") and
+    // at least one alphabetic char (rules out "1973 ... " data-row labels).
+    without_dots.ends_with(' ') && without_dots.trim().chars().any(|c| c.is_alphabetic())
+}
+
+/// Page-number shape: single ≤4-digit integer, a ", "-separated list of
+/// ≤4-digit integers ("18, 36, 107"), or a dashed section-page ID
+/// ("A-1", "5-21").  Rejects decimal cells ("4. 0"), thousands-separated
+/// values ("189,164"), and other long numeric data that appears in
+/// statistical tables.
+fn row_cell_is_page_number(cell: &str) -> bool {
+    let t = cell.trim();
+    if t.is_empty() {
+        return false;
+    }
+    if looks_like_section_page_id(t) {
+        return true;
+    }
+    // Page list: ", " separator (with space) distinguishes real page lists
+    // from thousands-separated numbers like "189,164".
+    let parts: Vec<&str> = t.split(", ").collect();
+    parts
+        .iter()
+        .all(|p| !p.is_empty() && p.len() <= 4 && p.chars().all(|c| c.is_ascii_digit()))
+}
+
+/// A cell shaped like an index leader fragment.  Accepts two forms:
+///   - "text ... number" — label + dots + page number in one cell
+///   - "... number"      — bare leader + number (row where the label
+///     landed in a separate column)
+///
+/// Both only count if followed by pure numeric content (optionally
+/// comma-separated page lists like "127, 213").
+fn cell_is_inline_leader(cell: &str) -> bool {
+    let cell = cell.trim();
+
+    // Find the first "..." run.  Surrounding-whitespace checks below
+    // reject intra-word ellipses ("etc...").
+    let idx = match cell.match_indices("...").next() {
+        Some((i, _)) => i,
+        None => return false,
+    };
+
+    let before = &cell[..idx];
+    let after_dots = &cell[idx + 3..];
+    // Allow extra dots (e.g. "....") by skipping any additional '.'
+    let after = after_dots.trim_start_matches('.');
+
+    // Require space (or start-of-cell) before the dots and space/digit
+    // after — blocks intra-word ellipses.
+    let before_ok = before.is_empty() || before.ends_with(' ');
+    let after_ok = after.starts_with(' ') || after.is_empty();
+    if !before_ok || !after_ok {
         return false;
     }
 
-    // Data tables with dot leaders (e.g. "1973....") have dots concentrated
-    // in one column (the label column) while many other columns contain numbers.
-    // True TOCs have dots spanning the middle and one page-number column at the end.
-    // If dots are confined to ≤1 column AND there are ≥3 columns with numbers,
-    // this is a data table, not a TOC.
-    let cols_with_dots = dot_cols.iter().filter(|&&c| c >= 2).count();
-    let cols_with_numbers = numeric_cols.iter().filter(|&&c| c >= 2).count();
-    if cols_with_dots <= 1 && cols_with_numbers >= 3 {
+    let after_trim = after.trim();
+    if after_trim.is_empty() {
+        return false;
+    }
+    // Tail must be purely numeric/page-list content.
+    let tail_numeric = after_trim
+        .chars()
+        .all(|c| c.is_ascii_digit() || matches!(c, ',' | ' ' | '.' | '-' | '$'))
+        && after_trim.chars().any(|c| c.is_ascii_digit());
+    if !tail_numeric {
         return false;
     }
 
-    // If a significant portion of cells are dots or page numbers, it's likely a TOC
-    let dot_ratio = dot_cells as f32 / total_cells as f32;
-    let page_num_ratio = page_number_cells as f32 / total_cells as f32;
+    // Either we have a label before, or the leader is bare (starts the cell)
+    // — both are legitimate index fragments.
+    before.chars().any(|c| c.is_alphabetic()) || before.trim().is_empty()
+}
 
-    // TOC typically has >15% dot cells and >10% page number cells
-    dot_ratio > 0.15 || (dot_ratio > 0.05 && page_num_ratio > 0.15)
+/// Dot-less tabular TOC: tagged PDFs emit entries as rows where the first
+/// column starts with a dotted section number (e.g. "4.3.1 Something") and
+/// the last column is one or more page numbers.  These have no leader dots
+/// and benefit from flat-list formatting (page numbers aligned to titles).
+pub(super) fn is_tabular_toc(cells: &[Vec<String>]) -> bool {
+    if cells.is_empty() {
+        return false;
+    }
+    let num_cols = cells[0].len();
+    if num_cols < 2 || cells.len() < 4 {
+        return false;
+    }
+
+    let section_rows = cells
+        .iter()
+        .filter(|row| {
+            row.iter()
+                .find(|c| !c.trim().is_empty())
+                .is_some_and(|c| starts_with_section_number(c.trim()))
+        })
+        .count();
+
+    let last_col = num_cols - 1;
+    let (last_filled, last_page_num) = cells.iter().fold((0u32, 0u32), |(f, n), row| {
+        let cell = row.get(last_col).map(|s| s.trim()).unwrap_or("");
+        if cell.is_empty() {
+            return (f, n);
+        }
+        let is_page_nums = cell
+            .split_whitespace()
+            .all(|tok| !tok.is_empty() && tok.chars().all(|c| c.is_ascii_digit()));
+        (f + 1, n + if is_page_nums { 1 } else { 0 })
+    });
+
+    let section_ratio = section_rows as f32 / cells.len() as f32;
+    let page_num_last_ratio = if last_filled > 0 {
+        last_page_num as f32 / last_filled as f32
+    } else {
+        0.0
+    };
+
+    section_ratio >= 0.6 && last_filled >= 3 && page_num_last_ratio >= 0.7
+}
+
+/// Matches dashed section-page identifiers used in technical manuals:
+/// "5-21", "A-1", "B--3", "TC-2".  At least one ASCII digit is required.
+fn looks_like_section_page_id(s: &str) -> bool {
+    let ok = s
+        .chars()
+        .all(|c| c.is_ascii_digit() || c.is_ascii_uppercase() || c == '-');
+    ok && s.chars().any(|c| c.is_ascii_digit())
+}
+
+/// Returns true when the leading token looks like a dotted section number:
+/// "1", "1.2", "1.2.3", "4.3.1.2" — integer components joined by dots,
+/// with at least one dot (single-number prefixes are too ambiguous).
+fn starts_with_section_number(s: &str) -> bool {
+    let Some(first) = s.split_whitespace().next() else {
+        return false;
+    };
+    let first = first.trim_end_matches('.');
+    let parts: Vec<&str> = first.split('.').collect();
+    if parts.len() < 2 || parts.len() > 6 {
+        return false;
+    }
+    parts
+        .iter()
+        .all(|p| !p.is_empty() && p.len() <= 3 && p.chars().all(|c| c.is_ascii_digit()))
 }
 
 /// Check if detected "table" cells are actually paragraph text fragments.
@@ -1144,6 +1324,33 @@ pub(crate) fn find_first_table_row(
             continue;
         }
 
+        // Skip rows that have duplicate non-empty cells. These are spanning
+        // super-headers (e.g., "First Degree | First Degree | Higher Degree")
+        // that sit above the real column header row. Using them as the markdown
+        // header produces duplicate column names that downstream validation
+        // rejects. Only skip if a subsequent row looks like a better header
+        // (denser fill or has data).
+        if filled_count >= 2 && !has_data {
+            let mut text_counts: std::collections::HashMap<&str, usize> =
+                std::collections::HashMap::new();
+            for cell in &filled_cells {
+                *text_counts.entry(cell.trim()).or_insert(0) += 1;
+            }
+            let has_duplicates = text_counts.values().any(|&count| count >= 2);
+            if has_duplicates {
+                // Check if a later row is a better header candidate
+                let has_better_below = cells.iter().skip(row_idx + 1).take(3).any(|r| {
+                    let next_filled = r.iter().filter(|c| !c.trim().is_empty()).count();
+                    let next_fill = next_filled as f32 / total_cols as f32;
+                    let next_numeric = r.iter().filter(|c| looks_like_number(c.trim())).count();
+                    next_fill >= 0.4 || next_numeric >= 2
+                });
+                if has_better_below {
+                    continue;
+                }
+            }
+        }
+
         // Data rows are definitely table content
         if has_data {
             first_table_row = row_idx;
@@ -1398,4 +1605,283 @@ mod tests {
             "data table with dot-leader labels should not be rejected as TOC"
         );
     }
+
+    #[test]
+    fn is_table_of_contents_accepts_hierarchical_indented_toc() {
+        // Mythos system card pages 4-5: top-level chapters indent at col 0,
+        // subsections at cols 1-2, leaving col 0 mostly empty (only ~10% of
+        // rows). Validation 1 was rejecting these even though the structure
+        // is unambiguously a TOC.
+        let cells = vec![
+            vec!["Abstract".to_string(), String::new(), "3".to_string()],
+            vec![
+                "1 Introduction".to_string(),
+                String::new(),
+                "10".to_string(),
+            ],
+            vec![
+                String::new(),
+                "1.1 Model training".to_string(),
+                "11".to_string(),
+            ],
+            vec![
+                String::new(),
+                "1.1.1 Training data".to_string(),
+                "11".to_string(),
+            ],
+            vec![
+                String::new(),
+                "1.1.2 Crowd workers".to_string(),
+                "12".to_string(),
+            ],
+            vec![
+                String::new(),
+                "1.2 Release decision".to_string(),
+                "13".to_string(),
+            ],
+            vec![
+                "2 RSP evaluations".to_string(),
+                String::new(),
+                "16".to_string(),
+            ],
+            vec![
+                String::new(),
+                "2.1 RSP risk assessment".to_string(),
+                "16".to_string(),
+            ],
+            vec![String::new(), "2.1.1 Context".to_string(), "16".to_string()],
+            vec![
+                String::new(),
+                "2.2 CB evaluations".to_string(),
+                "20".to_string(),
+            ],
+        ];
+        assert!(
+            is_table_of_contents(&cells),
+            "hierarchical TOC with sparse col 0 should still be detected"
+        );
+    }
+
+    #[test]
+    fn is_table_of_contents_rejects_dotless_toc() {
+        // Tabular TOC without leader dots: first column starts with dotted
+        // section numbers, last column is page numbers.  Pattern from
+        // Mythos system card pages 6-8.
+        let cells = vec![
+            vec![
+                "4.3 Case studies and targeted evaluations".to_string(),
+                String::new(),
+                "86".to_string(),
+            ],
+            vec![
+                "4.3.1 Destructive or reckless actions".to_string(),
+                "4.3.1.1 Synthetic-backend evaluation".to_string(),
+                "86 86".to_string(),
+            ],
+            vec![
+                "4.3.2 Adherence to constitution".to_string(),
+                "4.3.2.1 Overview".to_string(),
+                "89 89".to_string(),
+            ],
+            vec![
+                "4.3.3 Honesty and hallucinations".to_string(),
+                "4.3.3.1 Factual hallucinations".to_string(),
+                "93 94".to_string(),
+            ],
+            vec![
+                "4.4 Capability evaluations".to_string(),
+                String::new(),
+                "101".to_string(),
+            ],
+        ];
+        assert!(
+            is_table_of_contents(&cells),
+            "dot-less TOC with section numbers + page numbers should be rejected"
+        );
+    }
+
+    #[test]
+    fn dot_leader_toc_accepts_short_inline_leaders() {
+        // Index-style cells where the full "label ... number" pattern is
+        // preserved in a single cell (IRS Publication 17 back-of-book index).
+        let cells = vec![
+            vec!["Child tax credit ... 235".to_string(), String::new()],
+            vec!["Church employee ... 252".to_string(), String::new()],
+            vec!["Citizens outside the U.S ... 6".to_string(), String::new()],
+            vec![
+                "Claim for refund ... 18, 36, 107".to_string(),
+                String::new(),
+            ],
+            vec!["Clergy ... 7, 52".to_string(), String::new()],
+        ];
+        assert!(is_dot_leader_toc(&cells));
+    }
+
+    #[test]
+    fn dot_leader_toc_allows_ellipsis_data_table() {
+        // Data tables using "..." as a row-omission marker must not be
+        // mistaken for dot-leader TOCs.  Based on MCF5235RM QSPI RAM layout.
+        let cells = vec![
+            vec![
+                "0x00".to_string(),
+                "QTR0".to_string(),
+                "Transmit RAM".to_string(),
+            ],
+            vec!["0x01".to_string(), "QTR1".to_string(), String::new()],
+            vec![
+                "...".to_string(),
+                "...".to_string(),
+                "16 bits wide".to_string(),
+            ],
+            vec!["0x0F".to_string(), "QTR15".to_string(), String::new()],
+            vec![
+                "0x10".to_string(),
+                "QRR0".to_string(),
+                "Receive RAM".to_string(),
+            ],
+            vec!["0x11".to_string(), "QRR1".to_string(), String::new()],
+            vec![
+                "...".to_string(),
+                "...".to_string(),
+                "16 bits wide".to_string(),
+            ],
+            vec!["0x1F".to_string(), "QRR15".to_string(), String::new()],
+        ];
+        assert!(
+            !is_dot_leader_toc(&cells),
+            "ellipsis markers in a data table should not match TOC detection"
+        );
+    }
+
+    #[test]
+    fn dot_leader_toc_rejects_year_row_data_table() {
+        // ERP-2025 economic data tables: year labels with trailing " ... ",
+        // a final " ... " column, and decimal-looking numeric cells.  The
+        // detection previously classified these as dot-leader TOCs and
+        // routed them through flat-list formatting, destroying the grid.
+        let cells = vec![
+            vec![
+                "1973 ... ".to_string(),
+                "4. 0".to_string(),
+                "1. 8".to_string(),
+                "0. 4".to_string(),
+                "3. 2".to_string(),
+                " ... ".to_string(),
+            ],
+            vec![
+                "1974 ... ".to_string(),
+                "–1. 9".to_string(),
+                "–1. 6".to_string(),
+                "–5. 6".to_string(),
+                "2. 4".to_string(),
+                " ... ".to_string(),
+            ],
+            vec![
+                "1975 ... ".to_string(),
+                "2. 6".to_string(),
+                "5. 1".to_string(),
+                "6. 1".to_string(),
+                "4. 1".to_string(),
+                " ... ".to_string(),
+            ],
+            vec![
+                "1976 ... ".to_string(),
+                "4. 3".to_string(),
+                "5. 4".to_string(),
+                "6. 4".to_string(),
+                "4. 5".to_string(),
+                " ... ".to_string(),
+            ],
+        ];
+        assert!(
+            !is_dot_leader_toc(&cells),
+            "year-indexed data tables with decimal cells must not match TOC detection"
+        );
+    }
+
+    #[test]
+    fn dot_leader_toc_rejects_monthly_data_table() {
+        // ERP-2025 Table B-22: monthly labor-force rows with "Jan ... ",
+        // "Feb ... " labels and thousands-separated cells ("189,164").
+        // Previously matched TOC detection because "Jan ..." has alphabetic
+        // text and "189,164" passed the page-number shape check.
+        let cells = vec![
+            vec![
+                "2023: Jan ... ".to_string(),
+                "265,962".to_string(),
+                "165,871".to_string(),
+                "160,152".to_string(),
+                "62. 4".to_string(),
+            ],
+            vec![
+                "Feb ... ".to_string(),
+                "266,112".to_string(),
+                "166,263".to_string(),
+                "160,301".to_string(),
+                "62. 5".to_string(),
+            ],
+            vec![
+                "Mar ... ".to_string(),
+                "266,272".to_string(),
+                "166,690".to_string(),
+                "160,824".to_string(),
+                "62. 6".to_string(),
+            ],
+            vec![
+                "Apr ... ".to_string(),
+                "266,443".to_string(),
+                "166,678".to_string(),
+                "160,962".to_string(),
+                "62. 6".to_string(),
+            ],
+        ];
+        assert!(
+            !is_dot_leader_toc(&cells),
+            "monthly labor-force rows with thousands-separated data must not match TOC detection"
+        );
+    }
+
+    #[test]
+    fn tabular_toc_requires_section_numbers_and_pages() {
+        // Dot-less tabular TOC matches is_tabular_toc but not dot-leader.
+        let cells = vec![
+            vec![
+                "4.3 Case studies".to_string(),
+                String::new(),
+                "86".to_string(),
+            ],
+            vec![
+                "4.3.1 Destructive actions".to_string(),
+                String::new(),
+                "86".to_string(),
+            ],
+            vec![
+                "4.3.2 Adherence".to_string(),
+                String::new(),
+                "89".to_string(),
+            ],
+            vec!["4.3.3 Honesty".to_string(), String::new(), "93".to_string()],
+        ];
+        assert!(is_tabular_toc(&cells));
+        assert!(!is_dot_leader_toc(&cells));
+    }
+
+    #[test]
+    fn starts_with_section_number_matches_dotted() {
+        assert!(starts_with_section_number("1.2"));
+        assert!(starts_with_section_number("4.3.1"));
+        assert!(starts_with_section_number("4.3.1.2"));
+        assert!(starts_with_section_number("4.3 Case studies"));
+        assert!(starts_with_section_number("2.2.5.1 Expert red teaming"));
+    }
+
+    #[test]
+    fn starts_with_section_number_rejects_non_sections() {
+        assert!(!starts_with_section_number("Chapter 1"));
+        assert!(!starts_with_section_number("1973"));
+        assert!(!starts_with_section_number("1.5M"));
+        assert!(!starts_with_section_number("10.0%"));
+        assert!(!starts_with_section_number(""));
+        assert!(!starts_with_section_number("Hello world"));
+    }
 }

src/tables/detect_lines.rs

@@ -265,12 +265,12 @@ pub fn detect_tables_from_lines(items: &[TextItem], lines: &[PdfLine], page: u32
         page, num_rows, num_cols, item_indices.len(), page_item_count, non_empty_rows, cols_with_content
     );
 
-    vec![Table {
-        columns: col_edges,
-        rows: row_edges_desc[..num_rows].to_vec(),
+    vec![Table::new(
+        col_edges,
+        row_edges_desc[..num_rows].to_vec(),
         cells,
         item_indices,
-    }]
+    )]
 }
 
 #[cfg(test)]

src/tables/detect_rects.rs

@@ -1037,12 +1037,7 @@ fn try_build_grid(
         (columns, cells)
     };
 
-    GridResult::Ok(Table {
-        columns,
-        rows,
-        cells,
-        item_indices,
-    })
+    GridResult::Ok(Table::new(columns, rows, cells, item_indices))
 }
 
 /// Deduplicate nearby edge values within a tolerance, returning sorted unique edges.
@@ -1161,11 +1156,23 @@ fn propagate_merged_cells(
                 continue;
             }
 
-            // Find first and last grid rows that the rect spans
-            let first_row = (0..num_rows)
-                .find(|&r| ry <= row_edges[r] + tol && (ry + rh) >= row_edges[r + 1] - tol);
-            let last_row = (0..num_rows)
-                .rfind(|&r| ry <= row_edges[r] + tol && (ry + rh) >= row_edges[r + 1] - tol);
+            // Find first and last grid rows that the rect spans.
+            //
+            // Require a rect to actually overlap the row by more than `tol`
+            // to count as a span. A "rect bottom ≤ row top + tol AND rect
+            // top ≥ row bottom − tol" check gives false positives at shared
+            // row boundaries — a rect whose top equals row N's bottom lies
+            // entirely below the row but still passes the tolerance-slack
+            // check, cascading body text from unrelated rows into one
+            // merged cell.
+            let spans = |r: usize| {
+                let row_top = row_edges[r];
+                let row_bot = row_edges[r + 1];
+                let overlap = (row_top.min(ry + rh) - row_bot.max(ry)).max(0.0);
+                overlap > tol
+            };
+            let first_row = (0..num_rows).find(|&r| spans(r));
+            let last_row = (0..num_rows).rfind(|&r| spans(r));
 
             let (first, last) = match (first_row, last_row) {
                 (Some(f), Some(l)) if l > f => (f, l),
@@ -1442,12 +1449,7 @@ fn detect_row_stripe_table(
         content_ratio * 100.0
     );
 
-    Some(Table {
-        columns: column_centers,
-        rows: row_centers,
-        cells,
-        item_indices,
-    })
+    Some(Table::new(column_centers, row_centers, cells, item_indices))
 }
 
 /// Detect a table from cell-background rects that failed grid detection.
@@ -1677,6 +1679,49 @@ fn detect_row_stripe_table_from_cell_rects(
         return None;
     }
 
+    // Reject "tables" that are actually prose in a framed region.
+    // Columns here come from text X-position clustering; when prose wraps
+    // inside a bounding-box rect (e.g. chat-transcript figures) the
+    // word-boundary gaps cluster into many spurious columns, and the
+    // resulting cells hold sentence fragments riddled with common English
+    // function words. Count cells with any such word and reject when
+    // 20%+ of non-empty cells match — real tabular data (labels, units,
+    // numbers) rarely contains these words.
+    if num_cols >= 4 {
+        const PROSE_WORDS: &[&str] = &[
+            "a", "an", "the", "of", "to", "is", "was", "are", "were", "be", "been", "in", "on",
+            "at", "with", "for", "by", "as", "and", "or", "but", "this", "that", "these", "those",
+            "from", "into", "has", "have", "had", "not", "don't", "doesn't", "it's", "its", "it",
+            "i", "me", "my", "we", "our", "us", "you", "your", "they", "them", "their", "he",
+            "she", "his", "her",
+        ];
+        let mut prose_cells = 0usize;
+        let mut counted = 0usize;
+        for row in &cells {
+            for cell in row {
+                let t = cell.trim();
+                if t.is_empty() {
+                    continue;
+                }
+                counted += 1;
+                let lower = t.to_ascii_lowercase();
+                let has_prose_word = lower
+                    .split(|c: char| !c.is_ascii_alphabetic() && c != '\'')
+                    .any(|w| PROSE_WORDS.contains(&w));
+                if has_prose_word {
+                    prose_cells += 1;
+                }
+            }
+        }
+        if counted > 0 && prose_cells * 5 >= counted {
+            debug!(
+                "  cell-rect rejected: {}/{} cells contain prose function words — likely prose",
+                prose_cells, counted
+            );
+            return None;
+        }
+    }
+
     let column_centers: Vec<f32> = (0..num_cols)
         .map(|c| (col_edges[c] + col_edges[c + 1]) / 2.0)
         .collect();
@@ -1691,12 +1736,7 @@ fn detect_row_stripe_table_from_cell_rects(
         non_empty_cells as f32 / total_cells * 100.0
     );
 
-    Some(Table {
-        columns: column_centers,
-        rows: row_centers,
-        cells,
-        item_indices,
-    })
+    Some(Table::new(column_centers, row_centers, cells, item_indices))
 }
 
 /// Detect a table by merging all cluster rects into one group.
@@ -1875,12 +1915,7 @@ fn detect_merged_cluster_table(
         content_ratio * 100.0
     );
 
-    Some(Table {
-        columns: column_centers,
-        rows: row_centers,
-        cells,
-        item_indices,
-    })
+    Some(Table::new(column_centers, row_centers, cells, item_indices))
 }
 
 /// Cluster text item X positions into column centers with a given minimum threshold.
@@ -2370,6 +2405,27 @@ mod tests {
         assert_eq!(cells[1][0], "B");
     }
 
+    #[test]
+    fn test_propagate_merged_cells_rect_tangent_to_row_boundary() {
+        // Regression: a rect whose top exactly equals a row's bottom lies
+        // entirely outside that row, so it must not be considered to span
+        // it. With the old overlap-based predicate this cascaded into body
+        // text from unrelated rows being merged into a single header cell
+        // (mythos system card CB task-based evaluations table).
+        //
+        // Layout: two rows 0..80 and 80..160 (bottom → top in PDF coords),
+        // rect occupies only the lower row (y=0..80). Its top equals the
+        // upper row's bottom; it must not span the upper row.
+        let col_edges = vec![0.0, 50.0];
+        let row_edges = vec![160.0, 80.0, 0.0]; // top → bot
+        let mut cells = vec![vec!["Upper".to_string()], vec!["Lower".to_string()]];
+        let group_rects = vec![(0.0, 0.0, 50.0, 80.0)]; // rect at y=0..80
+        let skip = vec![false];
+        propagate_merged_cells(&mut cells, &col_edges, &row_edges, &group_rects, &skip);
+        assert_eq!(cells[0][0], "Upper", "upper row must not be merged");
+        assert_eq!(cells[1][0], "Lower", "lower row must not be touched");
+    }
+
     #[test]
     fn test_propagate_merged_cells_empty_cells_preserved() {
         let col_edges = vec![0.0, 50.0];

src/tables/detect_struct.rs

@@ -4,15 +4,385 @@
 //! elements linked to MCIDs, this module builds `Table` structs directly from
 //! the semantic hierarchy — no geometry heuristics needed.
 
-use std::collections::HashMap;
+use std::collections::{HashMap, HashSet};
 
 use log::debug;
 
-use crate::structure_tree::StructTable;
+use crate::structure_tree::{StructTable, StructTableRow};
 use crate::types::TextItem;
 
 use super::Table;
 
+#[derive(Debug, Clone)]
+struct MatchedCell {
+    text: String,
+    item_indices: Vec<usize>,
+    x: Option<f32>,
+    y: Option<f32>,
+}
+
+fn legacy_column_positions(
+    page_rows: &[&StructTableRow],
+    mcid_to_items: &HashMap<i64, Vec<usize>>,
+    items: &[TextItem],
+    page: u32,
+    num_cols: usize,
+) -> Vec<f32> {
+    let mut col_positions: Vec<f32> = vec![0.0; num_cols];
+    for (col, col_pos) in col_positions.iter_mut().enumerate() {
+        for row in page_rows {
+            if col < row.cells.len() {
+                if let Some(x) = row.cells[col]
+                    .mcids
+                    .iter()
+                    .filter(|(_, p)| *p == page)
+                    .filter_map(|(mcid, _)| mcid_to_items.get(mcid))
+                    .flatten()
+                    .map(|&idx| items[idx].x)
+                    .reduce(f32::min)
+                {
+                    *col_pos = x;
+                    break;
+                }
+            }
+        }
+    }
+    col_positions
+}
+
+fn infer_column_positions(
+    raw_rows: &[Vec<MatchedCell>],
+    fallback_positions: &[f32],
+    num_cols: usize,
+) -> Vec<f32> {
+    const SAME_COLUMN_TOLERANCE: f32 = 18.0;
+
+    let mut anchors = raw_rows
+        .iter()
+        .max_by_key(|row| row.iter().filter(|cell| cell.x.is_some()).count())
+        .map(|row| row.iter().filter_map(|cell| cell.x).collect::<Vec<_>>())
+        .unwrap_or_default();
+
+    if anchors.len() > num_cols {
+        anchors.truncate(num_cols);
+    }
+
+    let mut additional_positions: Vec<f32> = raw_rows
+        .iter()
+        .flat_map(|row| row.iter().filter_map(|cell| cell.x))
+        .collect();
+    additional_positions.sort_by(|a, b| a.total_cmp(b));
+
+    for x in additional_positions {
+        if anchors.len() >= num_cols {
+            break;
+        }
+        if anchors
+            .iter()
+            .all(|existing| (x - *existing).abs() > SAME_COLUMN_TOLERANCE)
+        {
+            anchors.push(x);
+            anchors.sort_by(|a, b| a.total_cmp(b));
+        }
+    }
+
+    if anchors.len() < num_cols {
+        for &x in fallback_positions {
+            if anchors.len() >= num_cols {
+                break;
+            }
+            if anchors
+                .iter()
+                .all(|existing| (x - *existing).abs() > SAME_COLUMN_TOLERANCE)
+            {
+                anchors.push(x);
+                anchors.sort_by(|a, b| a.total_cmp(b));
+            }
+        }
+    }
+
+    if anchors.is_empty() {
+        return fallback_positions.to_vec();
+    }
+
+    while anchors.len() < num_cols {
+        anchors.push(*anchors.last().unwrap());
+    }
+
+    anchors
+}
+
+fn align_positions_to_columns(cell_xs: &[f32], columns: &[f32]) -> Vec<usize> {
+    if cell_xs.is_empty() || columns.is_empty() {
+        return Vec::new();
+    }
+    if cell_xs.len() >= columns.len() {
+        return (0..cell_xs.len().min(columns.len())).collect();
+    }
+
+    let mut dp = vec![vec![f32::INFINITY; columns.len() + 1]; cell_xs.len() + 1];
+    let mut take = vec![vec![false; columns.len() + 1]; cell_xs.len() + 1];
+
+    for value in &mut dp[0] {
+        *value = 0.0;
+    }
+
+    for i in 1..=cell_xs.len() {
+        for j in 1..=columns.len() {
+            let skip_cost = dp[i][j - 1];
+            let take_cost = dp[i - 1][j - 1] + (cell_xs[i - 1] - columns[j - 1]).abs();
+            if take_cost <= skip_cost {
+                dp[i][j] = take_cost;
+                take[i][j] = true;
+            } else {
+                dp[i][j] = skip_cost;
+            }
+        }
+    }
+
+    let mut assignments_rev = Vec::with_capacity(cell_xs.len());
+    let mut i = cell_xs.len();
+    let mut j = columns.len();
+    while i > 0 && j > 0 {
+        if take[i][j] {
+            assignments_rev.push(j - 1);
+            i -= 1;
+            j -= 1;
+        } else {
+            j -= 1;
+        }
+    }
+
+    assignments_rev.reverse();
+    assignments_rev
+}
+
+fn align_struct_rows(
+    raw_rows: &[Vec<MatchedCell>],
+    col_positions: &[f32],
+) -> (Vec<Vec<String>>, Vec<f32>, Vec<usize>) {
+    let mut cells: Vec<Vec<String>> = Vec::with_capacity(raw_rows.len());
+    let mut row_positions: Vec<f32> = Vec::with_capacity(raw_rows.len());
+    let mut all_item_indices: Vec<usize> = Vec::new();
+
+    for row in raw_rows {
+        let present_cells: Vec<&MatchedCell> = row
+            .iter()
+            .filter(|cell| {
+                !cell.item_indices.is_empty() || !cell.text.is_empty() || cell.x.is_some()
+            })
+            .collect();
+        let cell_xs: Vec<f32> = present_cells.iter().filter_map(|cell| cell.x).collect();
+        let assignments = if cell_xs.len() == present_cells.len() {
+            align_positions_to_columns(&cell_xs, col_positions)
+        } else {
+            (0..present_cells.len().min(col_positions.len())).collect()
+        };
+
+        let mut row_cells = vec![String::new(); col_positions.len()];
+        for (cell, &col_idx) in present_cells.iter().zip(assignments.iter()) {
+            if !cell.text.is_empty() {
+                if !row_cells[col_idx].is_empty() {
+                    row_cells[col_idx].push(' ');
+                }
+                row_cells[col_idx].push_str(&cell.text);
+            }
+            all_item_indices.extend(cell.item_indices.iter().copied());
+        }
+
+        let row_y = row
+            .iter()
+            .filter_map(|cell| cell.y)
+            .reduce(f32::max)
+            .unwrap_or(0.0);
+        cells.push(row_cells);
+        row_positions.push(row_y);
+    }
+
+    (cells, row_positions, all_item_indices)
+}
+
+fn left_align_struct_rows(
+    raw_rows: &[Vec<MatchedCell>],
+    num_cols: usize,
+) -> (Vec<Vec<String>>, Vec<f32>, Vec<usize>) {
+    let mut cells: Vec<Vec<String>> = Vec::with_capacity(raw_rows.len());
+    let mut row_positions: Vec<f32> = Vec::with_capacity(raw_rows.len());
+    let mut all_item_indices: Vec<usize> = Vec::new();
+
+    for row in raw_rows {
+        let mut row_cells: Vec<String> = row.iter().map(|cell| cell.text.clone()).collect();
+        row_cells.truncate(num_cols);
+        while row_cells.len() < num_cols {
+            row_cells.push(String::new());
+        }
+        cells.push(row_cells);
+
+        all_item_indices.extend(
+            row.iter()
+                .flat_map(|cell| cell.item_indices.iter().copied()),
+        );
+        row_positions.push(
+            row.iter()
+                .filter_map(|cell| cell.y)
+                .reduce(f32::max)
+                .unwrap_or(0.0),
+        );
+    }
+
+    (cells, row_positions, all_item_indices)
+}
+
+fn recover_unclaimed_header_row(table: &mut Table, items: &[TextItem], has_ragged_rows: bool) {
+    if !has_ragged_rows || table.rows.is_empty() || table.columns.len() < 3 {
+        return;
+    }
+
+    const MAX_HEADER_DISTANCE: f32 = 90.0;
+    const MAX_GAP_TO_TABLE: f32 = 35.0;
+    const MAX_INTER_HEADER_GAP: f32 = 25.0;
+    const MAX_HEADER_ROWS: usize = 3;
+    const Y_TOLERANCE: f32 = 5.0;
+
+    let top_row_y = table.rows[0];
+    let x_min = table.columns.first().copied().unwrap_or(0.0) - 25.0;
+    let x_max = table.columns.last().copied().unwrap_or(0.0) + 120.0;
+    let claimed: HashSet<usize> = table.item_indices.iter().copied().collect();
+
+    let mut candidate_rows: Vec<(f32, Vec<(usize, &TextItem)>)> = Vec::new();
+    for (idx, item) in items.iter().enumerate() {
+        if claimed.contains(&idx)
+            || item.text.trim().is_empty()
+            || item.y <= top_row_y
+            || item.y - top_row_y > MAX_HEADER_DISTANCE
+            || item.x < x_min
+            || item.x > x_max
+        {
+            continue;
+        }
+
+        if let Some((_, row_items)) = candidate_rows
+            .iter_mut()
+            .find(|(row_y, _)| (item.y - *row_y).abs() < Y_TOLERANCE)
+        {
+            row_items.push((idx, item));
+        } else {
+            candidate_rows.push((item.y, vec![(idx, item)]));
+        }
+    }
+
+    if candidate_rows.is_empty() {
+        return;
+    }
+
+    for (_, row_items) in &mut candidate_rows {
+        row_items.sort_by(|a, b| a.1.x.total_cmp(&b.1.x));
+    }
+    candidate_rows.sort_by(|a, b| a.0.total_cmp(&b.0));
+
+    if candidate_rows[0].0 - top_row_y > MAX_GAP_TO_TABLE {
+        return;
+    }
+
+    let mut candidate_iter = candidate_rows.into_iter();
+    let Some(first_row) = candidate_iter.next() else {
+        return;
+    };
+    let mut selected_rows: Vec<(f32, Vec<(usize, &TextItem)>)> = vec![first_row];
+    let mut prev_y = selected_rows[0].0;
+    for (row_y, row_items) in candidate_iter {
+        if selected_rows.len() >= MAX_HEADER_ROWS {
+            break;
+        }
+        if row_y - prev_y > MAX_INTER_HEADER_GAP {
+            break;
+        }
+        prev_y = row_y;
+        selected_rows.push((row_y, row_items));
+    }
+
+    if selected_rows.is_empty() {
+        return;
+    }
+
+    let mut assigned_rows: Vec<(f32, Vec<String>, Vec<usize>)> = Vec::new();
+    let mut closest_row_populated = 0usize;
+    let mut combined_cols: HashSet<usize> = HashSet::new();
+
+    for (row_idx, (row_y, row_items)) in selected_rows.iter().enumerate() {
+        if row_items.len() > table.columns.len() {
+            return;
+        }
+
+        let row_xs: Vec<f32> = row_items.iter().map(|(_, item)| item.x).collect();
+        let assignments = align_positions_to_columns(&row_xs, &table.columns);
+        if assignments.len() != row_items.len() {
+            return;
+        }
+
+        let mut row_cells = vec![String::new(); table.columns.len()];
+        let mut row_indices = Vec::with_capacity(row_items.len());
+        let mut populated_cols: HashSet<usize> = HashSet::new();
+
+        for ((idx, item), &col_idx) in row_items.iter().zip(assignments.iter()) {
+            let text = item.text.trim();
+            if text.is_empty() {
+                continue;
+            }
+            if !row_cells[col_idx].is_empty() {
+                row_cells[col_idx].push(' ');
+            }
+            row_cells[col_idx].push_str(text);
+            row_indices.push(*idx);
+            populated_cols.insert(col_idx);
+        }
+
+        if row_idx == 0 {
+            closest_row_populated = populated_cols.len();
+        }
+
+        combined_cols.extend(populated_cols.iter().copied());
+        assigned_rows.push((*row_y, row_cells, row_indices));
+    }
+
+    let required_cols = if table.columns.len() <= 4 {
+        table.columns.len()
+    } else {
+        table.columns.len() - 1
+    };
+    if closest_row_populated < 2 || combined_cols.len() < required_cols {
+        return;
+    }
+
+    let mut header_cells = vec![String::new(); table.columns.len()];
+    let mut header_indices = Vec::new();
+    for (_, row_cells, row_indices) in assigned_rows.iter().rev() {
+        for (col_idx, cell_text) in row_cells.iter().enumerate() {
+            if cell_text.is_empty() {
+                continue;
+            }
+            if !header_cells[col_idx].is_empty() {
+                header_cells[col_idx].push(' ');
+            }
+            header_cells[col_idx].push_str(cell_text);
+        }
+        header_indices.extend(row_indices.iter().copied());
+    }
+
+    table.rows.insert(
+        0,
+        assigned_rows
+            .iter()
+            .map(|(row_y, _, _)| *row_y)
+            .reduce(f32::max)
+            .unwrap_or(top_row_y),
+    );
+    table.cells.insert(0, header_cells);
+    table.item_indices.extend(header_indices);
+    table.item_indices.sort_unstable();
+    table.item_indices.dedup();
+}
+
 /// Build tables from structure-tree table descriptors by matching MCIDs to TextItems.
 ///
 /// Returns tables for the given page.  Tables where fewer than 50% of cells
@@ -67,18 +437,14 @@ pub fn detect_tables_from_struct_tree(
             continue;
         }
 
-        // Build cell text and collect item indices
-        let mut cells: Vec<Vec<String>> = Vec::new();
-        let mut all_item_indices: Vec<usize> = Vec::new();
+        // Build cell text and geometry for alignment and header recovery.
+        let mut raw_rows: Vec<Vec<MatchedCell>> = Vec::new();
         let mut total_cells = 0u32;
         let mut matched_cells = 0u32;
 
         for row in &page_rows {
-            let mut row_cells = Vec::with_capacity(num_cols);
-            for (col_idx, cell) in row.cells.iter().enumerate() {
-                if col_idx >= num_cols {
-                    break;
-                }
+            let mut row_cells = Vec::with_capacity(row.cells.len());
+            for cell in &row.cells {
                 total_cells += 1;
 
                 // Collect all items for this cell's MCIDs
@@ -115,18 +481,18 @@ pub fn detect_tables_from_struct_tree(
                     .collect::<Vec<_>>()
                     .join(" ");
 
-                for (idx, _) in &cell_items {
-                    all_item_indices.push(*idx);
-                }
+                let item_indices = cell_items.iter().map(|(idx, _)| *idx).collect::<Vec<_>>();
+                let x = cell_items.iter().map(|(_, item)| item.x).reduce(f32::min);
+                let y = cell_items.iter().map(|(_, item)| item.y).reduce(f32::max);
 
-                row_cells.push(text);
+                row_cells.push(MatchedCell {
+                    text,
+                    item_indices,
+                    x,
+                    y,
+                });
             }
-
-            // Pad to num_cols
-            while row_cells.len() < num_cols {
-                row_cells.push(String::new());
-            }
-            cells.push(row_cells);
+            raw_rows.push(row_cells);
         }
 
         // Reject if too few cells matched (stale structure tree)
@@ -148,51 +514,54 @@ pub fn detect_tables_from_struct_tree(
             continue;
         }
 
-        // Derive row/column positions from item geometry
-        let mut row_positions: Vec<f32> = Vec::new();
-        for row in &page_rows {
-            let y = row
-                .cells
-                .iter()
-                .flat_map(|c| c.mcids.iter())
-                .filter(|(_, p)| *p == page)
-                .filter_map(|(mcid, _)| mcid_to_items.get(mcid))
-                .flatten()
-                .map(|&idx| items[idx].y)
-                .reduce(f32::max)
-                .unwrap_or(0.0);
-            row_positions.push(y);
-        }
+        let has_ragged_rows = raw_rows
+            .iter()
+            .any(|row| row.iter().filter(|cell| cell.x.is_some()).count() < num_cols);
+        let first_row_has_tagged_header = page_rows.first().is_some_and(|row| {
+            let header_cells = row.cells.iter().filter(|cell| cell.is_header).count();
+            header_cells * 2 >= row.cells.len()
+        });
+        let fallback_col_positions =
+            legacy_column_positions(&page_rows, &mcid_to_items, items, page, num_cols);
+        let (legacy_cells, legacy_row_positions, mut legacy_item_indices) =
+            left_align_struct_rows(&raw_rows, num_cols);
+        legacy_item_indices.sort_unstable();
+        legacy_item_indices.dedup();
+        let legacy_table = Table::new(
+            fallback_col_positions.clone(),
+            legacy_row_positions,
+            legacy_cells,
+            legacy_item_indices,
+        );
 
-        // Column positions: use X positions of first non-empty cell in each column
-        let mut col_positions: Vec<f32> = vec![0.0; num_cols];
-        for (col, col_pos) in col_positions.iter_mut().enumerate() {
-            for row in &page_rows {
-                if col < row.cells.len() {
-                    if let Some(x) = row.cells[col]
-                        .mcids
-                        .iter()
-                        .filter(|(_, p)| *p == page)
-                        .filter_map(|(mcid, _)| mcid_to_items.get(mcid))
-                        .flatten()
-                        .map(|&idx| items[idx].x)
-                        .reduce(f32::min)
-                    {
-                        *col_pos = x;
-                        break;
-                    }
-                }
-            }
-        }
+        let col_positions = infer_column_positions(&raw_rows, &fallback_col_positions, num_cols);
+        let (aligned_cells, aligned_row_positions, mut aligned_item_indices) =
+            align_struct_rows(&raw_rows, &col_positions);
+        aligned_item_indices.sort_unstable();
+        aligned_item_indices.dedup();
 
-        all_item_indices.sort_unstable();
-        all_item_indices.dedup();
+        let mut aligned_table = Table::new(
+            col_positions,
+            aligned_row_positions,
+            aligned_cells,
+            aligned_item_indices,
+        );
+        let item_count_before_header = aligned_table.item_indices.len();
+        let row_count_before_header = aligned_table.cells.len();
+        recover_unclaimed_header_row(
+            &mut aligned_table,
+            items,
+            has_ragged_rows && !first_row_has_tagged_header,
+        );
 
-        tables.push(Table {
-            columns: col_positions,
-            rows: row_positions,
-            cells,
-            item_indices: all_item_indices,
+        let recovered_header = aligned_table.item_indices.len() > item_count_before_header
+            || aligned_table.cells.len() > row_count_before_header;
+        let prefer_aligned = recovered_header;
+
+        tables.push(if prefer_aligned {
+            aligned_table
+        } else {
+            legacy_table
         });
     }
 
@@ -374,4 +743,419 @@ mod tests {
         let tables = detect_tables_from_struct_tree(&items, &struct_tables, 2);
         assert_eq!(tables.len(), 1);
     }
+
+    #[test]
+    fn realigns_ragged_rows_and_recovers_untagged_header() {
+        let items = vec![
+            make_item("Category", 50.0, 120.0, 1, None),
+            make_item("Potentially", 150.0, 120.0, 1, None),
+            make_item("Summary", 250.0, 120.0, 1, None),
+            make_item("Most commonly", 350.0, 120.0, 1, None),
+            make_item("concerning aspect", 150.0, 110.0, 1, None),
+            make_item("suggested", 350.0, 110.0, 1, None),
+            make_item("of circumstances", 150.0, 100.0, 1, None),
+            make_item("intervention", 350.0, 100.0, 1, None),
+            make_item("Existence of red-teaming", 150.0, 80.0, 1, Some(10)),
+            make_item("Important for safety", 250.0, 80.0, 1, Some(11)),
+            make_item("Ensure welfare interviews", 350.0, 80.0, 1, Some(12)),
+            make_item("Identity & self-knowledge", 50.0, 60.0, 1, Some(20)),
+            make_item("Lack of knowledge", 150.0, 60.0, 1, Some(21)),
+            make_item("Overall negative", 250.0, 60.0, 1, Some(22)),
+            make_item("Describe training process", 350.0, 60.0, 1, Some(23)),
+            make_item("Uncertainty around other copies", 150.0, 40.0, 1, Some(30)),
+            make_item("High uncertainty", 250.0, 40.0, 1, Some(31)),
+            make_item("No intervention suggested", 350.0, 40.0, 1, Some(32)),
+        ];
+
+        let struct_tables = vec![StructTable {
+            rows: vec![
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(10, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(11, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(12, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(20, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(21, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(22, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(23, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(30, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(31, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(32, 1)],
+                        },
+                    ],
+                },
+            ],
+        }];
+
+        let tables = detect_tables_from_struct_tree(&items, &struct_tables, 1);
+        assert_eq!(tables.len(), 1);
+        let table = &tables[0];
+        assert_eq!(table.cells.len(), 4);
+        assert_eq!(
+            table.cells[0],
+            vec![
+                "Category",
+                "Potentially concerning aspect of circumstances",
+                "Summary",
+                "Most commonly suggested intervention",
+            ]
+        );
+        assert_eq!(table.cells[1][0], "");
+        assert_eq!(table.cells[1][1], "Existence of red-teaming");
+        assert_eq!(table.cells[2][0], "Identity & self-knowledge");
+        assert_eq!(table.cells[3][0], "");
+        assert_eq!(table.columns.len(), 4);
+        assert!(table.columns.windows(2).all(|w| w[0] < w[1]));
+        assert_eq!(table.item_indices.len(), items.len());
+    }
+
+    #[test]
+    fn does_not_absorb_caption_above_ragged_struct_table() {
+        let items = vec![
+            make_item("Table 5-7: Summary of responses", 50.0, 120.0, 1, None),
+            make_item("Aspect one", 150.0, 80.0, 1, Some(10)),
+            make_item("Summary one", 250.0, 80.0, 1, Some(11)),
+            make_item("Category", 50.0, 60.0, 1, Some(20)),
+            make_item("Aspect two", 150.0, 60.0, 1, Some(21)),
+            make_item("Summary two", 250.0, 60.0, 1, Some(22)),
+            make_item("Aspect three", 150.0, 40.0, 1, Some(30)),
+            make_item("Summary three", 250.0, 40.0, 1, Some(31)),
+        ];
+
+        let struct_tables = vec![StructTable {
+            rows: vec![
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(10, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(11, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(20, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(21, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(22, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(30, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(31, 1)],
+                        },
+                    ],
+                },
+            ],
+        }];
+
+        let tables = detect_tables_from_struct_tree(&items, &struct_tables, 1);
+        assert_eq!(tables.len(), 1);
+        let table = &tables[0];
+        assert_eq!(table.cells.len(), 3);
+        assert!(
+            table
+                .cells
+                .iter()
+                .flatten()
+                .all(|cell| !cell.contains("Table 5-7")),
+            "caption must stay outside the table"
+        );
+        assert!(!table.item_indices.contains(&0));
+    }
+
+    #[test]
+    fn keeps_existing_tagged_header_without_absorbing_intro_or_caption() {
+        let items = vec![
+            make_item(
+                "Eighteen people left other comments regarding the Project.",
+                50.0,
+                130.0,
+                1,
+                None,
+            ),
+            make_item("Table 5-1:", 220.0, 130.0, 1, None),
+            make_item("Other Comments", 350.0, 130.0, 1, None),
+            make_item("Theme", 50.0, 110.0, 1, Some(10)),
+            make_item("Specific Concern/Inquiry", 200.0, 110.0, 1, Some(11)),
+            make_item("Response", 420.0, 110.0, 1, Some(12)),
+            make_item("Traffic", 50.0, 90.0, 1, Some(20)),
+            make_item("Road conditions", 200.0, 90.0, 1, Some(21)),
+            make_item("Maintenance response", 420.0, 90.0, 1, Some(22)),
+            make_item("Noise", 50.0, 70.0, 1, Some(30)),
+            make_item("Dust concerns", 200.0, 70.0, 1, Some(31)),
+            make_item("Mitigation response", 420.0, 70.0, 1, Some(32)),
+            make_item("Resource Use", 50.0, 50.0, 1, Some(40)),
+            make_item("Snowmobile trails", 200.0, 50.0, 1, Some(41)),
+            make_item("Access response", 420.0, 50.0, 1, Some(42)),
+        ];
+
+        let struct_tables = vec![StructTable {
+            rows: vec![
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(10, 1)],
+                        },
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(11, 1)],
+                        },
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(12, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(20, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(21, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(22, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(30, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(31, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(32, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(40, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(41, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(42, 1)],
+                        },
+                    ],
+                },
+            ],
+        }];
+
+        let tables = detect_tables_from_struct_tree(&items, &struct_tables, 1);
+        assert_eq!(tables.len(), 1);
+        let table = &tables[0];
+        assert_eq!(table.cells.len(), 4);
+        assert_eq!(
+            table.cells[0],
+            vec!["Theme", "Specific Concern/Inquiry", "Response"]
+        );
+        assert!(!table.item_indices.contains(&0));
+        assert!(!table.item_indices.contains(&1));
+        assert!(!table.item_indices.contains(&2));
+    }
+
+    #[test]
+    fn does_not_recover_header_for_narrow_two_column_table() {
+        let items = vec![
+            make_item("Alpha", 50.0, 120.0, 1, None),
+            make_item("Beta", 200.0, 120.0, 1, None),
+            make_item("First value", 200.0, 80.0, 1, Some(10)),
+            make_item("Only labeled row", 50.0, 60.0, 1, Some(20)),
+            make_item("Second value", 200.0, 60.0, 1, Some(21)),
+            make_item("Third value", 200.0, 40.0, 1, Some(30)),
+        ];
+
+        let struct_tables = vec![StructTable {
+            rows: vec![
+                StructTableRow {
+                    cells: vec![StructTableCell {
+                        is_header: false,
+                        mcids: vec![(10, 1)],
+                    }],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(20, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(21, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![StructTableCell {
+                        is_header: false,
+                        mcids: vec![(30, 1)],
+                    }],
+                },
+            ],
+        }];
+
+        let tables = detect_tables_from_struct_tree(&items, &struct_tables, 1);
+        assert_eq!(tables.len(), 1);
+        let table = &tables[0];
+        assert_eq!(table.cells.len(), 3);
+        assert_eq!(table.cells[0], vec!["First value", ""]);
+        assert_eq!(table.cells[1], vec!["Only labeled row", "Second value"]);
+        assert_eq!(table.cells[2], vec!["Third value", ""]);
+        assert!(!table.item_indices.contains(&0));
+        assert!(!table.item_indices.contains(&1));
+    }
+
+    #[test]
+    fn ragged_rows_without_recovered_header_keep_legacy_alignment() {
+        let items = vec![
+            make_item("Date", 150.0, 120.0, 1, Some(10)),
+            make_item("Title", 250.0, 120.0, 1, Some(11)),
+            make_item("PE", 350.0, 120.0, 1, Some(12)),
+            make_item("Bidder", 450.0, 120.0, 1, Some(13)),
+            make_item("Amount", 550.0, 120.0, 1, Some(14)),
+            make_item("1", 50.0, 100.0, 1, Some(20)),
+            make_item("8/1", 150.0, 100.0, 1, Some(21)),
+            make_item("Procurement", 250.0, 100.0, 1, Some(22)),
+            make_item("PUC", 350.0, 100.0, 1, Some(23)),
+            make_item("Vendor", 450.0, 100.0, 1, Some(24)),
+            make_item("SR1", 550.0, 100.0, 1, Some(25)),
+        ];
+
+        let struct_tables = vec![StructTable {
+            rows: vec![
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(10, 1)],
+                        },
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(11, 1)],
+                        },
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(12, 1)],
+                        },
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(13, 1)],
+                        },
+                        StructTableCell {
+                            is_header: true,
+                            mcids: vec![(14, 1)],
+                        },
+                    ],
+                },
+                StructTableRow {
+                    cells: vec![
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(20, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(21, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(22, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(23, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(24, 1)],
+                        },
+                        StructTableCell {
+                            is_header: false,
+                            mcids: vec![(25, 1)],
+                        },
+                    ],
+                },
+            ],
+        }];
+
+        let tables = detect_tables_from_struct_tree(&items, &struct_tables, 1);
+        assert_eq!(tables.len(), 1);
+        let table = &tables[0];
+        assert_eq!(table.cells[0][0], "Date");
+        assert_eq!(table.cells[0][4], "Amount");
+        assert_eq!(table.cells[0][5], "");
+    }
 }

src/tables/format.rs

@@ -1,12 +1,22 @@
 //! Table-to-markdown formatting and cell cleanup.
 
-use super::Table;
+use super::{Table, TableKind};
 
 pub fn table_to_markdown(table: &Table) -> String {
     if table.cells.is_empty() || table.cells[0].is_empty() {
         return String::new();
     }
 
+    // TOCs render poorly as markdown tables — emit a flat per-row text list
+    // instead so the page numbers stay aligned with their section titles
+    // rather than drifting to a separate column. Format from raw cells
+    // because continuation-row merging in clean_table_cells collapses
+    // separate TOC entries (e.g. "6.2 Contamination" + "6.2.1 SWE-bench")
+    // into one line where sub-entries leave column 0 empty.
+    if table.kind == TableKind::Toc {
+        return format_toc_as_list(&table.cells, &[]);
+    }
+
     // Clean up the table: merge continuation rows, extract footnotes, remove empty rows
     let (cleaned_cells, footnotes) = clean_table_cells(&table.cells);
 
@@ -49,6 +59,107 @@ pub fn table_to_markdown(table: &Table) -> String {
     output
 }
 
+/// Render a table-of-contents as a flat per-row text block.
+///
+/// Each row becomes one line: non-empty cells joined with spaces, and the
+/// last cell (typically a page number) is separated by a tab so the page
+/// numbers stay aligned with their titles instead of being pulled into a
+/// separate column by the column-aware reader.
+fn format_toc_as_list(cells: &[Vec<String>], footnotes: &[String]) -> String {
+    let mut output = String::new();
+
+    for row in cells {
+        let trimmed: Vec<&str> = row.iter().map(|c| c.trim()).collect();
+        let last_idx = trimmed.iter().rposition(|c| !c.is_empty());
+        let Some(last_idx) = last_idx else {
+            continue;
+        };
+
+        let last_cell = trimmed[last_idx];
+        let last_is_page = is_page_number_cell(last_cell);
+
+        let (title_cells, trailing) = if last_is_page && last_idx > 0 {
+            (&trimmed[..last_idx], Some(last_cell))
+        } else {
+            (&trimmed[..=last_idx], None)
+        };
+
+        // Skip dots-only cells when joining the title — in a detected TOC
+        // layout, a "...." cell is a leader separator, not part of the
+        // entry name.
+        let title = title_cells
+            .iter()
+            .filter(|c| !c.is_empty() && !is_dots_only(c))
+            .copied()
+            .collect::<Vec<_>>()
+            .join(" ");
+
+        if title.is_empty() && trailing.is_none() {
+            continue;
+        }
+
+        if !title.is_empty() {
+            output.push_str(&title);
+        }
+        if let Some(page) = trailing {
+            if !title.is_empty() {
+                output.push('\t');
+            }
+            output.push_str(page);
+        }
+        output.push('\n');
+    }
+
+    if !footnotes.is_empty() {
+        output.push('\n');
+        for footnote in footnotes {
+            output.push_str(footnote);
+            output.push('\n');
+        }
+    }
+
+    output
+}
+
+/// True when the cell looks like a page number.  Accepts:
+///   - plain digit tokens: "42", "86 86"
+///   - dashed section-page IDs: "5-21", "A-1", "B--3", "TC-2" (common in
+///     technical manuals)
+fn is_page_number_cell(cell: &str) -> bool {
+    let tokens: Vec<&str> = cell.split_whitespace().collect();
+    if tokens.is_empty() {
+        return false;
+    }
+    tokens.iter().all(|t| {
+        if t.is_empty() || t.len() > 8 {
+            return false;
+        }
+        let all_digits = t.chars().all(|c| c.is_ascii_digit());
+        if all_digits {
+            return t.len() <= 4;
+        }
+        // Section-page form: uppercase letters, digits, dashes; at least
+        // one digit present.
+        t.chars()
+            .all(|c| c.is_ascii_digit() || c.is_ascii_uppercase() || c == '-')
+            && t.chars().any(|c| c.is_ascii_digit())
+    })
+}
+
+/// True when the cell is purely leader dots (any length ≥ 3) with optional
+/// whitespace.
+fn is_dots_only(cell: &str) -> bool {
+    let t = cell.trim();
+    let dots = t.chars().filter(|&c| c == '.').count();
+    dots >= 3 && t.chars().all(|c| c == '.' || c.is_whitespace())
+}
+
+fn starts_with_uppercase_word(cell: &str) -> bool {
+    cell.chars()
+        .find(|c| c.is_alphanumeric())
+        .is_some_and(|c| c.is_uppercase())
+}
+
 /// Clean up table cells: merge continuation rows, extract footnotes, remove empty rows
 fn clean_table_cells(cells: &[Vec<String>]) -> (Vec<Vec<String>>, Vec<String>) {
     let mut cleaned: Vec<Vec<String>> = Vec::new();
@@ -107,11 +218,20 @@ fn clean_table_cells(cells: &[Vec<String>]) -> (Vec<Vec<String>>, Vec<String>) {
         let looks_like_data_row = non_first_cells.len() >= 2
             && avg_cell_len <= 10.0
             && numeric_cells > non_first_cells.len() / 2;
+        let uppercase_leading_cells = non_first_cells
+            .iter()
+            .filter(|cell| starts_with_uppercase_word(cell))
+            .count();
+        let looks_like_spanning_first_column_row = first_cell.is_empty()
+            && row.len() >= 4
+            && non_first_cells.len() == row.len().saturating_sub(1)
+            && uppercase_leading_cells >= non_first_cells.len().saturating_sub(1);
         // Classic continuation: first cell empty, content in other cells
         let is_classic_continuation = first_cell.is_empty()
             && !non_first_cells.is_empty()
             && !is_short_subheader
             && !looks_like_data_row
+            && !looks_like_spanning_first_column_row
             && cleaned.len() > 1;
 
         // Wrapped-cell continuation: row has fewer filled cells than the header
@@ -141,6 +261,7 @@ fn clean_table_cells(cells: &[Vec<String>]) -> (Vec<Vec<String>>, Vec<String>) {
             && filled_cells <= max_filled_for_merge
             && prev_filled > filled_cells
             && !looks_like_data_row
+            && !looks_like_spanning_first_column_row
             && !is_short_subheader;
 
         let is_continuation = is_classic_continuation || is_wrapped_continuation;
@@ -310,6 +431,65 @@ mod tests {
         assert_eq!(cleaned.len(), 3);
     }
 
+    #[test]
+    fn test_clean_table_cells_spanning_first_column_row_not_merged() {
+        let cells = vec![
+            vec![
+                "Category".into(),
+                "Potentially concerning aspect".into(),
+                "Summary".into(),
+                "Intervention".into(),
+            ],
+            vec![
+                "Identity & self-knowledge".into(),
+                "Lack of knowledge".into(),
+                "Overall negative".into(),
+                "Describe training".into(),
+            ],
+            vec![
+                "".into(),
+                "Uncertainty around other copies".into(),
+                "High uncertainty".into(),
+                "No intervention suggested".into(),
+            ],
+        ];
+        let (cleaned, _) = clean_table_cells(&cells);
+        assert_eq!(cleaned.len(), 3);
+        assert_eq!(cleaned[2][0], "");
+        assert_eq!(cleaned[2][1], "Uncertainty around other copies");
+    }
+
+    #[test]
+    fn test_clean_table_cells_full_width_continuation_row_still_merges_when_lowercase() {
+        let cells = vec![
+            vec![
+                "Classification".into(),
+                "Before tax".into(),
+                "After tax".into(),
+                "Standard equipment".into(),
+                "Options".into(),
+            ],
+            vec![
+                "Exclusive Special".into(),
+                "83,500,000".into(),
+                "79,275,000".into(),
+                "Standard equipment".into(),
+                "Option A".into(),
+            ],
+            vec![
+                "".into(),
+                "with 3.5% individual consumption tax applied".into(),
+                "with 3.5% individual consumption tax applied".into(),
+                "lighting(crash pad)".into(),
+                "sound system".into(),
+            ],
+        ];
+        let (cleaned, _) = clean_table_cells(&cells);
+        assert_eq!(cleaned.len(), 2);
+        assert!(cleaned[1][1].contains("83,500,000"));
+        assert!(cleaned[1][1].contains("with 3.5%"));
+    }
+
     #[test]
     fn test_clean_table_cells_header_row_not_merged() {
         // Continuation requires cleaned.len() > 1 (don't merge into header)
@@ -358,6 +538,7 @@ mod tests {
                 vec!["Bob".into(), "25".into()],
             ],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
         let md = table_to_markdown(&table);
         assert!(md.contains("|Name|"));
@@ -373,6 +554,7 @@ mod tests {
             rows: vec![500.0],
             cells: vec![vec!["Only".into(), "Row".into()]],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
         let md = table_to_markdown(&table);
         assert!(md.contains("|Only|"));
@@ -386,6 +568,7 @@ mod tests {
             rows: vec![],
             cells: vec![],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
         assert_eq!(table_to_markdown(&table), "");
     }
@@ -401,6 +584,7 @@ mod tests {
                 vec!["(1)".into(), "Footnote text".into()],
             ],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
         let md = table_to_markdown(&table);
         assert!(md.contains("(1) Footnote text"));
@@ -416,6 +600,7 @@ mod tests {
                 vec!["太郎".into(), "25".into()],
             ],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
         let md = table_to_markdown(&table);
         assert!(md.contains("名前"));
@@ -429,7 +614,47 @@ mod tests {
             rows: vec![500.0],
             cells: vec![vec![]],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
         assert_eq!(table_to_markdown(&table), "");
     }
+
+    #[test]
+    fn test_table_to_markdown_toc_renders_as_flat_list() {
+        // A TOC-shaped table with section numbers in col 0 and page numbers
+        // in the last column should render as a flat list, not a markdown
+        // table, so the page numbers stay on the same line as their titles.
+        let table = Table::new(
+            vec![50.0, 80.0, 300.0],
+            vec![500.0; 5],
+            vec![
+                vec![
+                    "4.3".into(),
+                    "Case studies and targeted evaluations".into(),
+                    "86".into(),
+                ],
+                vec![
+                    "4.3.1".into(),
+                    "Destructive or reckless actions".into(),
+                    "86".into(),
+                ],
+                vec![
+                    "4.3.2".into(),
+                    "Adherence to its constitution".into(),
+                    "89".into(),
+                ],
+                vec!["4.4".into(), "Capability evaluations".into(), "101".into()],
+                vec!["4.5".into(), "White-box analyses".into(), "113".into()],
+            ],
+            vec![],
+        );
+        assert_eq!(table.kind, TableKind::Toc);
+        let md = table_to_markdown(&table);
+        assert!(
+            !md.contains("|---|"),
+            "TOC should not render as a markdown table: {md}"
+        );
+        assert!(md.contains("4.3 Case studies and targeted evaluations\t86"));
+        assert!(md.contains("4.5 White-box analyses\t113"));
+    }
 }

src/tables/grid.rs

@@ -82,33 +82,42 @@ pub(crate) fn find_column_boundaries(
         }
     }
 
-    let mut columns = Vec::new();
-    let mut cluster_items: Vec<f32> = vec![x_positions[0]];
+    // Track cluster membership: for each cluster, store the list of x positions
+    let mut cluster_xs: Vec<Vec<f32>> = vec![vec![x_positions[0]]];
 
     for &x in &x_positions[1..] {
+        let last_cluster = cluster_xs.last().unwrap();
         // For dense columns (gap-histogram triggered), use edge-based clustering:
         // compare with the last item to avoid center-drift that merges adjacent
         // narrow columns.  For normal tables, use center-based (original behavior).
         let reference = if use_edge_clustering {
-            *cluster_items.last().unwrap()
+            *last_cluster.last().unwrap()
         } else {
-            cluster_items.iter().sum::<f32>() / cluster_items.len() as f32
+            last_cluster.iter().sum::<f32>() / last_cluster.len() as f32
         };
 
         if x - reference > cluster_threshold {
-            let cluster_center = cluster_items.iter().sum::<f32>() / cluster_items.len() as f32;
-            columns.push(cluster_center);
-            cluster_items = vec![x];
+            cluster_xs.push(vec![x]);
         } else {
-            cluster_items.push(x);
+            cluster_xs.last_mut().unwrap().push(x);
         }
     }
 
-    // Don't forget last cluster
-    if !cluster_items.is_empty() {
-        columns.push(cluster_items.iter().sum::<f32>() / cluster_items.len() as f32);
+    // Numeric column merge pass: when a sparse cluster (few items, typically
+    // header text) is adjacent to a dense numeric cluster and within 1.5×
+    // threshold, merge them. This fixes tables where multi-line wrapped
+    // headers have slightly different X positions than the data columns,
+    // causing the header and data to split into separate clusters.
+    let columns_before_merge = cluster_xs.len();
+    if columns_before_merge >= 3 {
+        cluster_xs = merge_numeric_adjacent_clusters(cluster_xs, items, cluster_threshold);
     }
 
+    let columns: Vec<f32> = cluster_xs
+        .iter()
+        .map(|xs| xs.iter().sum::<f32>() / xs.len() as f32)
+        .collect();
+
     // Filter columns - each should have multiple items
     let min_items_per_col = (items.len() / columns.len().max(1) / 4).max(2);
     let columns: Vec<f32> = columns
@@ -123,8 +132,9 @@ pub(crate) fn find_column_boundaries(
         .collect();
 
     log::debug!(
-        "  find_column_boundaries: {} columns before filter, threshold={:.1}, {} items",
+        "  find_column_boundaries: {} columns (merged from {}), threshold={:.1}, {} items",
         columns.len(),
+        columns_before_merge,
         cluster_threshold,
         items.len()
     );
@@ -148,6 +158,116 @@ pub(crate) fn find_column_boundaries(
     columns
 }
 
+/// Check if a text string looks like a number (digits, decimals, sign, comma).
+fn is_numeric_text(s: &str) -> bool {
+    let s = s.trim();
+    if s.is_empty() {
+        return false;
+    }
+    // Match patterns like: 8.23, -1.05, 9.99, 7.12, 100, 3,456.78, +5%, ---
+    // But NOT: BIO, Department, Core Courses
+    s.chars()
+        .all(|c| c.is_ascii_digit() || c == '.' || c == ',' || c == '-' || c == '+' || c == '%')
+        && s.chars().any(|c| c.is_ascii_digit())
+}
+
+/// Merge adjacent X-position clusters when one is a sparse header cluster
+/// and the other is a dense numeric data cluster. This prevents multi-line
+/// wrapped headers from splitting a logical column into two clusters.
+fn merge_numeric_adjacent_clusters(
+    mut clusters: Vec<Vec<f32>>,
+    items: &[(usize, &TextItem)],
+    threshold: f32,
+) -> Vec<Vec<f32>> {
+    // For each cluster, compute: center, item count, numeric fraction
+    struct ClusterInfo {
+        center: f32,
+        count: usize,
+        numeric_frac: f32,
+    }
+
+    let compute_info = |xs: &[f32]| -> ClusterInfo {
+        let center = xs.iter().sum::<f32>() / xs.len() as f32;
+        // Count items and numeric fraction for items near this cluster center
+        let mut total = 0;
+        let mut numeric = 0;
+        for (_, item) in items {
+            if (item.x - center).abs() < threshold {
+                total += 1;
+                if is_numeric_text(&item.text) {
+                    numeric += 1;
+                }
+            }
+        }
+        ClusterInfo {
+            center,
+            count: total,
+            numeric_frac: if total > 0 {
+                numeric as f32 / total as f32
+            } else {
+                0.0
+            },
+        }
+    };
+
+    // Merge distance: allow merging clusters that are slightly beyond the
+    // original threshold. Use 1.5× threshold to catch header-vs-data splits.
+    let merge_dist = threshold * 1.5;
+
+    // Iterate and merge adjacent pairs. Use a simple left-to-right scan.
+    let mut merged = true;
+    while merged {
+        merged = false;
+        let mut i = 0;
+        while i + 1 < clusters.len() {
+            let info_a = compute_info(&clusters[i]);
+            let info_b = compute_info(&clusters[i + 1]);
+            let dist = (info_b.center - info_a.center).abs();
+
+            if dist > merge_dist {
+                i += 1;
+                continue;
+            }
+
+            // Determine if one cluster is sparse (header) and the other
+            // is dense and numeric (data). A cluster is "sparse" if it has
+            // significantly fewer items than the other.
+            let (sparse, dense) = if info_a.count < info_b.count {
+                (&info_a, &info_b)
+            } else {
+                (&info_b, &info_a)
+            };
+
+            // Merge if the dense cluster is predominantly numeric (>50%)
+            // and the sparse cluster has at most 1/3 the items of the dense one.
+            let should_merge =
+                dense.numeric_frac > 0.50 && sparse.count <= dense.count / 2 && sparse.count <= 5;
+
+            if should_merge {
+                log::debug!(
+                    "  merging column clusters: center {:.1} ({} items, {:.0}% numeric) + {:.1} ({} items, {:.0}% numeric), dist={:.1}",
+                    info_a.center,
+                    info_a.count,
+                    info_a.numeric_frac * 100.0,
+                    info_b.center,
+                    info_b.count,
+                    info_b.numeric_frac * 100.0,
+                    dist,
+                );
+                // Merge cluster i+1 into cluster i
+                let next = clusters.remove(i + 1);
+                clusters[i].extend(next);
+                merged = true;
+                // Don't increment i — check if the merged cluster can merge further
+            } else {
+                i += 1;
+            }
+        }
+    }
+
+    clusters
+}
+
 /// Find row boundaries by clustering Y positions
 pub(crate) fn find_row_boundaries(items: &[(usize, &TextItem)]) -> Vec<f32> {
     let mut y_positions: Vec<f32> = items.iter().map(|(_, i)| i.y).collect();
@@ -379,6 +499,7 @@ pub(crate) fn recover_header_row(
 #[cfg(test)]
 mod tests {
     use super::*;
+    use crate::tables::TableKind;
     use crate::types::ItemType;
 
     fn make_item(text: &str, x: f32, y: f32, font_size: f32) -> TextItem {
@@ -636,6 +757,7 @@ mod tests {
             rows: vec![500.0, 480.0],
             cells: vec![vec!["A".into(), "B".into()], vec!["C".into(), "D".into()]],
             item_indices: vec![2, 3],
+            kind: TableKind::Data,
         };
 
         recover_header_row(&mut table, &all_items, 9.0);
@@ -654,6 +776,7 @@ mod tests {
             rows: vec![500.0],
             cells: vec![vec!["A".into(), "B".into()]],
             item_indices: vec![0, 1],
+            kind: TableKind::Data,
         };
 
         let rows_before = table.rows.len();
@@ -674,6 +797,7 @@ mod tests {
             rows: vec![500.0, 480.0],
             cells: vec![vec!["A".into(), "B".into()], vec!["C".into(), "D".into()]],
             item_indices: vec![2, 3],
+            kind: TableKind::Data,
         };
 
         let rows_before = table.rows.len();
@@ -694,6 +818,7 @@ mod tests {
             rows: vec![500.0],
             cells: vec![vec!["A".into(), "B".into()]],
             item_indices: vec![1, 2],
+            kind: TableKind::Data,
         };
 
         let rows_before = table.rows.len();
@@ -709,6 +834,7 @@ mod tests {
             rows: vec![],
             cells: vec![],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
 
         recover_header_row(&mut table, &all_items, 9.0);

src/tables/mod.rs

@@ -9,13 +9,16 @@ mod detect_struct;
 mod financial;
 mod format;
 mod grid;
+pub mod structured;
 
 pub use detect_heuristic::detect_tables;
+pub(crate) use detect_heuristic::is_table_of_contents;
 pub use detect_lines::detect_tables_from_lines;
 pub(crate) use detect_rects::cluster_rects;
 pub use detect_rects::{detect_tables_from_rects, RectHintRegion};
 pub use detect_struct::detect_tables_from_struct_tree;
 pub use format::table_to_markdown;
+pub use structured::{cells_to_markdown, StructuredCell};
 
 use crate::types::TextItem;
 
@@ -166,12 +169,12 @@ pub(crate) fn try_build_rect_guided_table(
     used_indices.sort_unstable();
     used_indices.dedup();
 
-    Some(Table {
-        columns: col_boundaries,
-        rows: row_boundaries,
+    Some(Table::new(
+        col_boundaries,
+        row_boundaries,
         cells,
-        item_indices: used_indices,
-    })
+        used_indices,
+    ))
 }
 
 /// Split a TextItem whose text contains multiple whitespace-separated tokens
@@ -541,12 +544,22 @@ pub(crate) fn try_build_table_from_columns(items: &[TextItem], page: u32) -> Opt
         multi_col_rows
     );
 
-    Some(Table {
-        columns: col_xs,
-        rows: row_ys,
-        cells,
-        item_indices,
-    })
+    Some(Table::new(col_xs, row_ys, cells, item_indices))
+}
+
+/// What kind of structure a detected `Table` represents. Classification is
+/// computed once at construction so consumers don't have to re-analyze the
+/// cells (and stay consistent across detection backends).
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub enum TableKind {
+    /// A real data table — renders as markdown table syntax.
+    #[default]
+    Data,
+    /// A table of contents — renders as a flat list with tab-aligned page
+    /// numbers via `format_toc_as_list`. Detected through the table pipeline
+    /// because TOCs share row/column structure with tables, but they are not
+    /// data tables and shouldn't appear in `pages_with_tables` etc.
+    Toc,
 }
 
 /// A detected table.
@@ -560,6 +573,31 @@ pub struct Table {
     pub cells: Vec<Vec<String>>,
     /// Items that belong to this table
     pub item_indices: Vec<usize>,
+    /// Data table vs TOC. Set by `Table::new` from `cells`.
+    pub kind: TableKind,
+}
+
+impl Table {
+    /// Build a table and classify it (data vs TOC) from its cells.
+    pub fn new(
+        columns: Vec<f32>,
+        rows: Vec<f32>,
+        cells: Vec<Vec<String>>,
+        item_indices: Vec<usize>,
+    ) -> Self {
+        let kind = if is_table_of_contents(&cells) {
+            TableKind::Toc
+        } else {
+            TableKind::Data
+        };
+        Self {
+            columns,
+            rows,
+            cells,
+            item_indices,
+            kind,
+        }
+    }
 }
 
 #[cfg(test)]
@@ -642,6 +680,7 @@ mod tests {
                 vec!["Cell 1".into(), "Cell 2".into()],
             ],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
 
         let md = table_to_markdown(&table);
@@ -1002,6 +1041,7 @@ mod tests {
                 vec!["3".into(), "5/2".into(), "Item C".into(), "300".into()],
             ],
             item_indices: vec![],
+            kind: TableKind::Data,
         };
 
         let md = table_to_markdown(&table);

src/tables/structured.rs

@@ -0,0 +1,972 @@
+//! Structure-recovery-aware (TSR) table assembly.
+//!
+//! Consumes the raw output of an external table-structure recognition model
+//! (e.g. SLANet on PaddleOCR): a flat list of HTML structure tokens plus a
+//! parallel list of per-cell bboxes. Pairs each cell open-tag with its bbox
+//! in document order, tracks row/column position with rowspan/colspan
+//! awareness, and emits a markdown pipe-table.
+//!
+//! No real HTML parser is needed — the token grammar is restricted (see
+//! [`parse_structure`]), so a small state machine is enough.
+//!
+//! Cell text is supplied separately by the caller (typically by overlap-
+//! testing PDF text items against each cell's page-PDF-pt bbox).
+
+use std::collections::{HashMap, HashSet};
+
+/// A single resolved cell, with both structural metadata and its bbox in
+/// page PDF-points (top-left origin).
+#[derive(Debug, Clone)]
+pub struct StructuredCell {
+    /// 0-indexed grid row.
+    pub row: usize,
+    /// 0-indexed grid column.
+    pub col: usize,
+    /// 1 for a normal cell.
+    pub rowspan: usize,
+    /// 1 for a normal cell.
+    pub colspan: usize,
+    /// `true` when the cell is a `<th>` or sits inside `<thead>`.
+    pub is_header: bool,
+    /// Cell text (filled in by the caller after overlap-testing PDF items).
+    pub text: String,
+    /// Axis-aligned bbox `[x1, y1, x2, y2]` in page PDF-points, top-left origin.
+    pub page_pt_bbox: [f32; 4],
+}
+
+/// Intermediate parse result before the caller fills in text + page coords.
+#[derive(Debug, Clone)]
+pub(crate) struct CellSlot {
+    pub row: usize,
+    pub col: usize,
+    pub rowspan: usize,
+    pub colspan: usize,
+    pub is_header: bool,
+    /// Index into the parallel `cell_bboxes` array.
+    pub bbox_idx: usize,
+}
+
+/// Parse a sequence of SLANet structure tokens into ordered cell slots.
+///
+/// Token grammar (no real HTML parsing required):
+/// - Section markers: `<thead>`, `</thead>`, `<tbody>`, `</tbody>` and
+///   wrapper tokens (`<html>`, `<body>`, `<table>`, plus closing variants)
+///   are tracked or skipped.
+/// - Row markers: `<tr>` opens a new row, `</tr>` is informational.
+/// - Empty cell, single token: `<td></td>` or `<th></th>`.
+/// - Cell with attributes, multi-token sequence: `<td` (or `<th`), then
+///   attribute fragments like ` colspan="4"`, then `>`, then later `</td>`
+///   (or `</th>`). Cells get paired with the next bbox in document order.
+///
+/// Cells inside `<thead>` and any `<th>` cells are flagged as headers.
+/// rowspan/colspan attributes are honoured and prior-row rowspans push
+/// later-row cells to the right.
+pub(crate) fn parse_structure(tokens: &[String]) -> Vec<CellSlot> {
+    let mut slots: Vec<CellSlot> = Vec::new();
+    let mut occupied: HashSet<(usize, usize)> = HashSet::new();
+    let mut row: usize = 0;
+    let mut col: usize = 0;
+    let mut bbox_idx: usize = 0;
+    let mut in_thead = false;
+    let mut started_first_row = false;
+
+    let mut i = 0;
+    while i < tokens.len() {
+        let tok = tokens[i].trim();
+        match tok {
+            "<thead>" => {
+                in_thead = true;
+            }
+            "</thead>" => {
+                in_thead = false;
+            }
+            "<tr>" => {
+                if started_first_row {
+                    row += 1;
+                }
+                col = 0;
+                started_first_row = true;
+            }
+            "<td></td>" | "<th></th>" => {
+                let is_th = tok == "<th></th>";
+                while occupied.contains(&(row, col)) {
+                    col += 1;
+                }
+                slots.push(CellSlot {
+                    row,
+                    col,
+                    rowspan: 1,
+                    colspan: 1,
+                    is_header: in_thead || is_th,
+                    bbox_idx,
+                });
+                bbox_idx += 1;
+                col += 1;
+            }
+            "<td" | "<th" => {
+                let is_th = tok == "<th";
+                let mut rowspan: usize = 1;
+                let mut colspan: usize = 1;
+                // Consume attribute fragments until we hit ">".
+                i += 1;
+                while i < tokens.len() && tokens[i].trim() != ">" {
+                    let attr = tokens[i].as_str();
+                    if let Some(v) = parse_int_attr(attr, "rowspan") {
+                        rowspan = v.max(1);
+                    } else if let Some(v) = parse_int_attr(attr, "colspan") {
+                        colspan = v.max(1);
+                    }
+                    i += 1;
+                }
+                // i now points at the `>` token (or off the end if malformed).
+                while occupied.contains(&(row, col)) {
+                    col += 1;
+                }
+                slots.push(CellSlot {
+                    row,
+                    col,
+                    rowspan,
+                    colspan,
+                    is_header: in_thead || is_th,
+                    bbox_idx,
+                });
+                for r in row..row + rowspan {
+                    for c in col..col + colspan {
+                        occupied.insert((r, c));
+                    }
+                }
+                bbox_idx += 1;
+                col += colspan;
+            }
+            // Wrapper / informational tokens — no-op.
+            _ => {}
+        }
+        i += 1;
+    }
+
+    slots
+}
+
+/// Parse an attribute fragment like ` colspan="4"` or `rowspan='2'`.
+///
+/// Tolerates leading whitespace and either single or double quotes.
+fn parse_int_attr(s: &str, name: &str) -> Option<usize> {
+    let trimmed = s.trim();
+    if !trimmed.starts_with(name) {
+        return None;
+    }
+    let rest = trimmed[name.len()..].trim_start();
+    let rest = rest.strip_prefix('=')?.trim_start();
+    let value = rest
+        .trim_start_matches(['"', '\''])
+        .trim_end_matches(['"', '\'']);
+    value.parse().ok()
+}
+
+/// Convert a SLANet polygon (4 or 8 elements) into an axis-aligned
+/// `[x1, y1, x2, y2]` rect.
+///
+/// 8-element form: `[x1,y1, x2,y1, x2,y2, x1,y2]` (4 corners). We ignore the
+/// implicit corner order and just take min/max so rotated polygons collapse
+/// to a sane bounding box.
+///
+/// 4-element form: `[x1, y1, x2, y2]` (axis-aligned, older SLANet variants).
+pub(crate) fn polygon_to_aabb(coords: &[f32]) -> Option<[f32; 4]> {
+    match coords.len() {
+        4 => {
+            let x1 = coords[0].min(coords[2]);
+            let y1 = coords[1].min(coords[3]);
+            let x2 = coords[0].max(coords[2]);
+            let y2 = coords[1].max(coords[3]);
+            Some([x1, y1, x2, y2])
+        }
+        8 => {
+            let xs = [coords[0], coords[2], coords[4], coords[6]];
+            let ys = [coords[1], coords[3], coords[5], coords[7]];
+            let x1 = xs.iter().copied().fold(f32::INFINITY, f32::min);
+            let y1 = ys.iter().copied().fold(f32::INFINITY, f32::min);
+            let x2 = xs.iter().copied().fold(f32::NEG_INFINITY, f32::max);
+            let y2 = ys.iter().copied().fold(f32::NEG_INFINITY, f32::max);
+            if x1.is_finite() && y1.is_finite() && x2.is_finite() && y2.is_finite() {
+                Some([x1, y1, x2, y2])
+            } else {
+                None
+            }
+        }
+        _ => None,
+    }
+}
+
+/// Convert a cell rect from crop image-pixel space to page PDF-points
+/// (top-left origin), given the crop's PDF-point offset on the page and the
+/// DPI the crop image was rendered at.
+pub(crate) fn cell_px_to_page_pt(
+    cell_px: [f32; 4],
+    render_dpi: f32,
+    crop_origin_pt: [f32; 2],
+) -> [f32; 4] {
+    let pt_per_px = if render_dpi > 0.0 {
+        72.0 / render_dpi
+    } else {
+        1.0
+    };
+    let [x_off, y_off] = crop_origin_pt;
+    [
+        cell_px[0] * pt_per_px + x_off,
+        cell_px[1] * pt_per_px + y_off,
+        cell_px[2] * pt_per_px + x_off,
+        cell_px[3] * pt_per_px + y_off,
+    ]
+}
+
+/// Refine TSR cell bboxes into non-overlapping row/column bands.
+///
+/// SLANet-style bboxes are often plausible but too tall on dense borderless
+/// tables. Native PDF text assignment is more reliable when each parsed row
+/// owns the band between neighboring row centers instead of the full model box.
+pub(crate) fn normalize_cell_bands(cells: &mut [StructuredCell]) {
+    if cells.len() < 2 {
+        return;
+    }
+
+    let row_bands = derive_axis_bands(cells, Axis::Y);
+    let col_bands = derive_axis_bands(cells, Axis::X);
+
+    for cell in cells {
+        let row_end = cell.row + cell.rowspan.max(1).saturating_sub(1);
+        if let (Some(&(y1, _)), Some(&(_, y2))) =
+            (row_bands.get(&cell.row), row_bands.get(&row_end))
+        {
+            let clamped_y1 = cell.page_pt_bbox[1].max(y1);
+            let clamped_y2 = cell.page_pt_bbox[3].min(y2);
+            if clamped_y1 < clamped_y2 {
+                cell.page_pt_bbox[1] = clamped_y1;
+                cell.page_pt_bbox[3] = clamped_y2;
+            }
+        }
+
+        let col_end = cell.col + cell.colspan.max(1).saturating_sub(1);
+        if let (Some(&(x1, _)), Some(&(_, x2))) =
+            (col_bands.get(&cell.col), col_bands.get(&col_end))
+        {
+            let clamped_x1 = cell.page_pt_bbox[0].max(x1);
+            let clamped_x2 = cell.page_pt_bbox[2].min(x2);
+            if clamped_x1 < clamped_x2 {
+                cell.page_pt_bbox[0] = clamped_x1;
+                cell.page_pt_bbox[2] = clamped_x2;
+            }
+        }
+    }
+}
+
+#[derive(Clone, Copy)]
+enum Axis {
+    X,
+    Y,
+}
+
+fn derive_axis_bands(cells: &[StructuredCell], axis: Axis) -> HashMap<usize, (f32, f32)> {
+    let mut by_index: HashMap<usize, Vec<(f32, f32)>> = HashMap::new();
+
+    // Prefer non-spanning cells so colspan/rowspan boxes do not skew a single
+    // column/row center. If an axis has no non-spanning examples for an index,
+    // fall back to anchored cells below.
+    for cell in cells {
+        let span = match axis {
+            Axis::X => cell.colspan.max(1),
+            Axis::Y => cell.rowspan.max(1),
+        };
+        if span == 1 {
+            let idx = match axis {
+                Axis::X => cell.col,
+                Axis::Y => cell.row,
+            };
+            by_index
+                .entry(idx)
+                .or_default()
+                .push(axis_bounds(cell.page_pt_bbox, axis));
+        }
+    }
+
+    for cell in cells {
+        let idx = match axis {
+            Axis::X => cell.col,
+            Axis::Y => cell.row,
+        };
+        if !by_index.contains_key(&idx) {
+            by_index
+                .entry(idx)
+                .or_default()
+                .push(axis_bounds(cell.page_pt_bbox, axis));
+        }
+    }
+
+    let mut rows: Vec<(usize, f32, f32, f32)> = by_index
+        .into_iter()
+        .filter_map(|(idx, bounds)| {
+            let mut min_edge = f32::INFINITY;
+            let mut max_edge = f32::NEG_INFINITY;
+            let mut center_sum = 0.0;
+            let mut count = 0usize;
+            for (lo, hi) in bounds {
+                if lo.is_finite() && hi.is_finite() && lo < hi {
+                    min_edge = min_edge.min(lo);
+                    max_edge = max_edge.max(hi);
+                    center_sum += (lo + hi) * 0.5;
+                    count += 1;
+                }
+            }
+            (count > 0).then_some((idx, center_sum / count as f32, min_edge, max_edge))
+        })
+        .collect();
+
+    if rows.len() < 2 {
+        return rows
+            .into_iter()
+            .map(|(idx, _center, lo, hi)| (idx, (lo, hi)))
+            .collect();
+    }
+
+    rows.sort_by_key(|(idx, _, _, _)| *idx);
+
+    let mut bands = HashMap::new();
+    for i in 0..rows.len() {
+        let (idx, _center, min_edge, max_edge) = rows[i];
+        let lo = if i == 0 {
+            min_edge
+        } else {
+            (rows[i - 1].1 + rows[i].1) * 0.5
+        };
+        let hi = if i + 1 == rows.len() {
+            max_edge
+        } else {
+            (rows[i].1 + rows[i + 1].1) * 0.5
+        };
+        if lo.is_finite() && hi.is_finite() && lo < hi {
+            bands.insert(idx, (lo, hi));
+        }
+    }
+
+    bands
+}
+
+fn axis_bounds(bbox: [f32; 4], axis: Axis) -> (f32, f32) {
+    match axis {
+        Axis::X => (bbox[0].min(bbox[2]), bbox[0].max(bbox[2])),
+        Axis::Y => (bbox[1].min(bbox[3]), bbox[1].max(bbox[3])),
+    }
+}
+
+/// Sanitize cell text for inclusion in a markdown pipe-table cell:
+/// collapse whitespace runs, drop newlines/tabs (cells must be one line),
+/// and escape pipes that would otherwise break the table.
+fn sanitize_cell(text: &str) -> String {
+    let mut s = String::with_capacity(text.len());
+    let mut prev_space = false;
+    for c in text.chars() {
+        match c {
+            '|' => {
+                s.push_str("\\|");
+                prev_space = false;
+            }
+            '\n' | '\r' | '\t' | ' ' => {
+                if !prev_space {
+                    s.push(' ');
+                }
+                prev_space = true;
+            }
+            other => {
+                s.push(other);
+                prev_space = false;
+            }
+        }
+    }
+    s.trim().to_string()
+}
+
+/// Render a list of explicitly-positioned cells as a markdown pipe-table.
+///
+/// Grid dimensions are inferred from the cells' (row, col, rowspan, colspan)
+/// extents. A cell with colspan/rowspan > 1 is rendered in its top-left
+/// position; the absorbed grid positions are emitted as empty cells so the
+/// markdown stays a valid rectangular grid that downstream readers can
+/// column-count correctly.
+///
+/// The separator row (`|---|...|`) is emitted after the **last** row that
+/// contains a header cell (`is_header == true`). When no cells are flagged
+/// as headers — e.g. the upstream TSR model didn't emit `<thead>`/`<th>` —
+/// the separator falls back to "after row 0" so the output is still a
+/// valid pipe-table.
+pub fn cells_to_markdown(cells: &[StructuredCell]) -> String {
+    if cells.is_empty() {
+        return String::new();
+    }
+    let num_rows = cells
+        .iter()
+        .map(|c| c.row + c.rowspan.max(1))
+        .max()
+        .unwrap_or(0);
+    let num_cols = cells
+        .iter()
+        .map(|c| c.col + c.colspan.max(1))
+        .max()
+        .unwrap_or(0);
+    if num_rows == 0 || num_cols == 0 {
+        return String::new();
+    }
+
+    // Separator goes after the last header row, falling back to row 0 when
+    // no header cells exist. Clamped into range so a malformed cell with
+    // row >= num_rows can't push it past the table.
+    let separator_after_row = cells
+        .iter()
+        .filter(|c| c.is_header)
+        .map(|c| c.row)
+        .max()
+        .unwrap_or(0)
+        .min(num_rows.saturating_sub(1));
+
+    let mut grid: Vec<Vec<String>> = vec![vec![String::new(); num_cols]; num_rows];
+    for cell in cells {
+        if cell.row < num_rows && cell.col < num_cols {
+            grid[cell.row][cell.col] = sanitize_cell(&cell.text);
+        }
+    }
+
+    let mut output = String::new();
+    for (row_idx, row) in grid.iter().enumerate() {
+        output.push('|');
+        for cell in row {
+            output.push_str(cell);
+            output.push('|');
+        }
+        output.push('\n');
+        if row_idx == separator_after_row {
+            output.push('|');
+            for _ in 0..num_cols {
+                output.push_str("---|");
+            }
+            output.push('\n');
+        }
+    }
+    output
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn t(s: &str) -> String {
+        s.to_string()
+    }
+
+    /// Tokens for the synthetic 3×3 grid example (one colspan-4 row + two
+    /// data rows of 4 cells each = 9 cells total, 3 rows × 4 cols).
+    fn synthetic_3x3_tokens() -> Vec<String> {
+        vec![
+            "<html>",
+            "<body>",
+            "<table>",
+            "<tbody>",
+            "<tr>",
+            "<td",
+            " colspan=\"4\"",
+            ">",
+            "</td>",
+            "</tr>",
+            "<tr>",
+            "<td></td>",
+            "<td></td>",
+            "<td></td>",
+            "<td></td>",
+            "</tr>",
+            "<tr>",
+            "<td></td>",
+            "<td></td>",
+            "<td></td>",
+            "<td></td>",
+            "</tr>",
+            "</tbody>",
+            "</table>",
+            "</body>",
+            "</html>",
+        ]
+        .into_iter()
+        .map(t)
+        .collect()
+    }
+
+    /// Bboxes for the synthetic 3×3 grid (8-element polygon form), all
+    /// within a 400×120 px crop.
+    fn synthetic_3x3_bboxes() -> Vec<Vec<f32>> {
+        vec![
+            vec![3.0, 2.0, 395.0, 2.0, 396.0, 59.0, 3.0, 59.0],
+            vec![26.0, 62.0, 140.0, 62.0, 141.0, 120.0, 26.0, 120.0],
+            vec![149.0, 64.0, 248.0, 64.0, 248.0, 119.0, 149.0, 119.0],
+            vec![257.0, 64.0, 350.0, 64.0, 350.0, 119.0, 257.0, 119.0],
+            vec![359.0, 64.0, 395.0, 64.0, 395.0, 119.0, 359.0, 119.0],
+            vec![26.0, 122.0, 140.0, 122.0, 140.0, 178.0, 26.0, 178.0],
+            vec![149.0, 124.0, 248.0, 124.0, 248.0, 179.0, 149.0, 179.0],
+            vec![257.0, 124.0, 350.0, 124.0, 350.0, 179.0, 257.0, 179.0],
+            vec![359.0, 124.0, 395.0, 124.0, 395.0, 179.0, 359.0, 179.0],
+        ]
+    }
+
+    #[test]
+    fn parse_structure_synthetic_3x3() {
+        let tokens = synthetic_3x3_tokens();
+        let slots = parse_structure(&tokens);
+
+        assert_eq!(slots.len(), 9, "should parse 9 cells");
+
+        // Cell 0: row 0 col 0, colspan 4
+        assert_eq!(slots[0].row, 0);
+        assert_eq!(slots[0].col, 0);
+        assert_eq!(slots[0].colspan, 4);
+        assert_eq!(slots[0].rowspan, 1);
+
+        // Cells 1..5: row 1, cols 0..3
+        for (i, slot) in slots.iter().enumerate().skip(1).take(4) {
+            assert_eq!(slot.row, 1, "cell {i}: row should be 1");
+            assert_eq!(slot.col, i - 1, "cell {i}: col should be {}", i - 1);
+            assert_eq!(slot.colspan, 1);
+            assert_eq!(slot.rowspan, 1);
+        }
+
+        // Cells 5..9: row 2, cols 0..3
+        for (i, slot) in slots.iter().enumerate().skip(5).take(4) {
+            assert_eq!(slot.row, 2, "cell {i}: row should be 2");
+            assert_eq!(slot.col, i - 5);
+            assert_eq!(slot.colspan, 1);
+        }
+    }
+
+    #[test]
+    fn polygon_to_aabb_8elt() {
+        // Synthetic cell bbox 0
+        let coords = vec![3.0, 2.0, 395.0, 2.0, 396.0, 59.0, 3.0, 59.0];
+        let aabb = polygon_to_aabb(&coords).unwrap();
+        assert_eq!(aabb, [3.0, 2.0, 396.0, 59.0]);
+    }
+
+    #[test]
+    fn polygon_to_aabb_4elt() {
+        let coords = vec![5.0, 10.0, 50.0, 60.0];
+        let aabb = polygon_to_aabb(&coords).unwrap();
+        assert_eq!(aabb, [5.0, 10.0, 50.0, 60.0]);
+    }
+
+    #[test]
+    fn polygon_to_aabb_4elt_unordered() {
+        // Caller may pass corners in any order; min/max should normalise.
+        let coords = vec![50.0, 60.0, 5.0, 10.0];
+        let aabb = polygon_to_aabb(&coords).unwrap();
+        assert_eq!(aabb, [5.0, 10.0, 50.0, 60.0]);
+    }
+
+    #[test]
+    fn polygon_to_aabb_invalid_len() {
+        assert!(polygon_to_aabb(&[1.0, 2.0, 3.0]).is_none());
+        assert!(polygon_to_aabb(&[1.0; 6]).is_none());
+        assert!(polygon_to_aabb(&[]).is_none());
+    }
+
+    #[test]
+    fn synthetic_3x3_aabbs_inside_crop() {
+        // All 9 bboxes should produce valid (x1<x2, y1<y2) rects within the
+        // crop bounds (400 wide, ~180 tall by inspection of the fixture).
+        let bboxes = synthetic_3x3_bboxes();
+        assert_eq!(bboxes.len(), 9);
+        for (i, bb) in bboxes.iter().enumerate() {
+            let aabb = polygon_to_aabb(bb).unwrap_or_else(|| panic!("bbox {i} invalid"));
+            assert!(aabb[0] < aabb[2], "bbox {i}: x1 < x2");
+            assert!(aabb[1] < aabb[3], "bbox {i}: y1 < y2");
+            assert!(aabb[0] >= 0.0 && aabb[2] <= 500.0, "bbox {i}: within crop");
+            assert!(aabb[1] >= 0.0 && aabb[3] <= 200.0, "bbox {i}: within crop");
+        }
+    }
+
+    #[test]
+    fn normalize_cell_bands_splits_overlapping_slanet_rows() {
+        let mut cells = vec![
+            StructuredCell {
+                row: 0,
+                col: 0,
+                rowspan: 1,
+                colspan: 1,
+                is_header: true,
+                text: String::new(),
+                page_pt_bbox: [10.0, 100.0, 90.0, 120.0],
+            },
+            StructuredCell {
+                row: 0,
+                col: 1,
+                rowspan: 1,
+                colspan: 1,
+                is_header: true,
+                text: String::new(),
+                page_pt_bbox: [90.0, 100.0, 170.0, 120.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 0,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: String::new(),
+                page_pt_bbox: [10.0, 116.0, 90.0, 136.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 1,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: String::new(),
+                page_pt_bbox: [90.0, 116.0, 170.0, 136.0],
+            },
+        ];
+
+        normalize_cell_bands(&mut cells);
+
+        assert_eq!(cells[0].page_pt_bbox[3], cells[2].page_pt_bbox[1]);
+        assert_eq!(cells[1].page_pt_bbox[3], cells[3].page_pt_bbox[1]);
+        assert!(
+            (cells[0].page_pt_bbox[3] - 118.0).abs() < 0.01,
+            "row separator should be midpoint between row centers: {:?}",
+            cells
+        );
+    }
+
+    #[test]
+    fn normalize_cell_bands_preserves_colspan_extent() {
+        let mut cells = vec![
+            StructuredCell {
+                row: 0,
+                col: 0,
+                rowspan: 1,
+                colspan: 2,
+                is_header: true,
+                text: String::new(),
+                page_pt_bbox: [8.0, 80.0, 172.0, 98.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 0,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: String::new(),
+                page_pt_bbox: [10.0, 96.0, 90.0, 114.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 1,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: String::new(),
+                page_pt_bbox: [88.0, 96.0, 170.0, 114.0],
+            },
+        ];
+
+        normalize_cell_bands(&mut cells);
+
+        assert!(
+            cells[0].page_pt_bbox[0] <= cells[1].page_pt_bbox[0],
+            "spanning cell should retain the first column's left edge"
+        );
+        assert!(
+            cells[0].page_pt_bbox[2] >= cells[2].page_pt_bbox[2],
+            "spanning cell should retain the last column's right edge"
+        );
+    }
+
+    #[test]
+    fn parse_int_attr_basic() {
+        assert_eq!(parse_int_attr(" colspan=\"4\"", "colspan"), Some(4));
+        assert_eq!(parse_int_attr(" rowspan=\"2\"", "rowspan"), Some(2));
+        assert_eq!(parse_int_attr("colspan='3'", "colspan"), Some(3));
+        assert_eq!(parse_int_attr(" colspan=\"4\"", "rowspan"), None);
+        assert_eq!(parse_int_attr(" class=\"foo\"", "colspan"), None);
+    }
+
+    #[test]
+    fn parse_structure_rowspan_pushes_next_row_right() {
+        // <tr><td rowspan="2">A</td><td>B</td></tr><tr><td>C</td></tr>
+        // Expected: A at (0,0), B at (0,1), C at (1,1) — col 0 of row 1
+        // is occupied by A's rowspan.
+        let tokens: Vec<String> = vec![
+            "<table>",
+            "<tbody>",
+            "<tr>",
+            "<td",
+            " rowspan=\"2\"",
+            ">",
+            "</td>",
+            "<td></td>",
+            "</tr>",
+            "<tr>",
+            "<td></td>",
+            "</tr>",
+            "</tbody>",
+            "</table>",
+        ]
+        .into_iter()
+        .map(t)
+        .collect();
+
+        let slots = parse_structure(&tokens);
+        assert_eq!(slots.len(), 3);
+        assert_eq!((slots[0].row, slots[0].col), (0, 0));
+        assert_eq!(slots[0].rowspan, 2);
+        assert_eq!((slots[1].row, slots[1].col), (0, 1));
+        // C should be at (1, 1) because (1, 0) is occupied by A's rowspan.
+        assert_eq!((slots[2].row, slots[2].col), (1, 1));
+    }
+
+    #[test]
+    fn parse_structure_thead_marks_headers() {
+        // <thead><tr><th>H1</th><th>H2</th></tr></thead>
+        // <tbody><tr><td>D1</td><td>D2</td></tr></tbody>
+        let tokens: Vec<String> = vec![
+            "<table>",
+            "<thead>",
+            "<tr>",
+            "<th></th>",
+            "<th></th>",
+            "</tr>",
+            "</thead>",
+            "<tbody>",
+            "<tr>",
+            "<td></td>",
+            "<td></td>",
+            "</tr>",
+            "</tbody>",
+            "</table>",
+        ]
+        .into_iter()
+        .map(t)
+        .collect();
+
+        let slots = parse_structure(&tokens);
+        assert_eq!(slots.len(), 4);
+        assert!(slots[0].is_header && slots[1].is_header);
+        assert!(!slots[2].is_header && !slots[3].is_header);
+    }
+
+    #[test]
+    fn parse_structure_th_outside_thead_still_header() {
+        // A row-header style: leading <th> in tbody.
+        let tokens: Vec<String> = vec![
+            "<table>",
+            "<tbody>",
+            "<tr>",
+            "<th></th>",
+            "<td></td>",
+            "</tr>",
+            "</tbody>",
+            "</table>",
+        ]
+        .into_iter()
+        .map(t)
+        .collect();
+
+        let slots = parse_structure(&tokens);
+        assert_eq!(slots.len(), 2);
+        assert!(slots[0].is_header);
+        assert!(!slots[1].is_header);
+    }
+
+    #[test]
+    fn parse_structure_th_with_attrs() {
+        let tokens: Vec<String> = vec![
+            "<table>",
+            "<thead>",
+            "<tr>",
+            "<th",
+            " colspan=\"2\"",
+            ">",
+            "</th>",
+            "</tr>",
+            "</thead>",
+            "</table>",
+        ]
+        .into_iter()
+        .map(t)
+        .collect();
+        let slots = parse_structure(&tokens);
+        assert_eq!(slots.len(), 1);
+        assert_eq!(slots[0].colspan, 2);
+        assert!(slots[0].is_header);
+    }
+
+    #[test]
+    fn cells_to_markdown_synthetic_3x3() {
+        // Build the cells the parser would produce for the synthetic grid,
+        // and provide some sample text so we can sanity-check output.
+        let cells = vec![
+            StructuredCell {
+                row: 0,
+                col: 0,
+                rowspan: 1,
+                colspan: 4,
+                is_header: false,
+                text: "Title".into(),
+                page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 0,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: "a".into(),
+                page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 1,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: "b".into(),
+                page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 2,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: "c".into(),
+                page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+            },
+            StructuredCell {
+                row: 1,
+                col: 3,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: "d".into(),
+                page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+            },
+        ];
+
+        let md = cells_to_markdown(&cells);
+        // Header row contains the spanning cell text in col 0 and pads to 4 cols.
+        // Absorbed-by-colspan positions render as empty cells (no padding).
+        assert!(md.starts_with("|Title||||\n"), "got: {md}");
+        assert!(md.contains("|---|---|---|---|\n"));
+        assert!(md.contains("|a|b|c|d|\n"));
+    }
+
+    #[test]
+    fn cells_to_markdown_escapes_pipes() {
+        let cells = vec![
+            StructuredCell {
+                row: 0,
+                col: 0,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: "a|b".into(),
+                page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+            },
+            StructuredCell {
+                row: 0,
+                col: 1,
+                rowspan: 1,
+                colspan: 1,
+                is_header: false,
+                text: "x".into(),
+                page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+            },
+        ];
+        let md = cells_to_markdown(&cells);
+        assert!(md.contains("|a\\|b|x|"));
+    }
+
+    #[test]
+    fn cells_to_markdown_collapses_whitespace_and_newlines() {
+        let cells = vec![StructuredCell {
+            row: 0,
+            col: 0,
+            rowspan: 1,
+            colspan: 1,
+            is_header: false,
+            text: "foo  \n  bar\tbaz".into(),
+            page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+        }];
+        let md = cells_to_markdown(&cells);
+        assert!(md.contains("|foo bar baz|"));
+    }
+
+    fn cell(row: usize, col: usize, is_header: bool, text: &str) -> StructuredCell {
+        StructuredCell {
+            row,
+            col,
+            rowspan: 1,
+            colspan: 1,
+            is_header,
+            text: text.into(),
+            page_pt_bbox: [0.0, 0.0, 0.0, 0.0],
+        }
+    }
+
+    #[test]
+    fn cells_to_markdown_separator_after_last_header_row() {
+        // Two-row header (a multi-row thead), then two body rows. Separator
+        // should land after row 1 (the LAST header row), not after row 0.
+        let cells = vec![
+            cell(0, 0, true, "H0a"),
+            cell(0, 1, true, "H0b"),
+            cell(1, 0, true, "H1a"),
+            cell(1, 1, true, "H1b"),
+            cell(2, 0, false, "d0a"),
+            cell(2, 1, false, "d0b"),
+            cell(3, 0, false, "d1a"),
+            cell(3, 1, false, "d1b"),
+        ];
+        let md = cells_to_markdown(&cells);
+        let expected = "|H0a|H0b|\n|H1a|H1b|\n|---|---|\n|d0a|d0b|\n|d1a|d1b|\n";
+        assert_eq!(md, expected, "got: {md}");
+    }
+
+    #[test]
+    fn cells_to_markdown_separator_when_row_0_not_header() {
+        // Row 0 is not flagged as a header but row 1 is. Separator should
+        // follow row 1 (the header), demonstrating that we don't blindly
+        // emit after row 0.
+        let cells = vec![
+            cell(0, 0, false, "x0a"),
+            cell(0, 1, false, "x0b"),
+            cell(1, 0, true, "Hdr1"),
+            cell(1, 1, true, "Hdr2"),
+            cell(2, 0, false, "data1"),
+            cell(2, 1, false, "data2"),
+        ];
+        let md = cells_to_markdown(&cells);
+        // Confirm the separator is NOT after row 0.
+        assert!(!md.starts_with("|x0a|x0b|\n|---|"), "got: {md}");
+        // Confirm it IS after row 1.
+        assert!(
+            md.contains("|Hdr1|Hdr2|\n|---|---|\n|data1|data2|"),
+            "got: {md}"
+        );
+    }
+
+    #[test]
+    fn cells_to_markdown_no_headers_falls_back_to_row_0() {
+        // No header cells at all — fallback: separator after row 0 so the
+        // output is still a valid markdown pipe-table.
+        let cells = vec![
+            cell(0, 0, false, "a"),
+            cell(0, 1, false, "b"),
+            cell(1, 0, false, "c"),
+            cell(1, 1, false, "d"),
+        ];
+        let md = cells_to_markdown(&cells);
+        assert_eq!(md, "|a|b|\n|---|---|\n|c|d|\n");
+    }
+}

src/tounicode.rs

@@ -520,6 +520,18 @@ impl ToUnicodeCMap {
         }
     }
 
+    /// Get the maximum source CID across all mappings (char_map + ranges).
+    fn max_source_cid(&self) -> Option<u16> {
+        let char_max = self.char_map.keys().copied().max();
+        let range_max = self.ranges.iter().map(|&(_, end, _)| end).max();
+        match (char_max, range_max) {
+            (Some(a), Some(b)) => Some(a.max(b)),
+            (a @ Some(_), None) => a,
+            (None, b @ Some(_)) => b,
+            (None, None) => None,
+        }
+    }
+
     /// Remap a CMap that references pre-subsetting GIDs to sequential post-subsetting GIDs.
     /// Collects all source CIDs, sorts them, and reassigns to 1, 2, 3, ...
     pub fn remap_to_sequential(&self) -> ToUnicodeCMap {
@@ -657,6 +669,81 @@ fn get_w_array_start_cid(cid_font_dict: &lopdf::Dictionary, doc: &Document) -> O
     }
 }
 
+/// Return true if the CIDFont's W (widths) array explicitly covers the given CID.
+///
+/// The W array uses two formats (PDF 32000-1:2008, §9.7.4.3):
+///   1. `c [w1 w2 ... wn]` — widths for CIDs c, c+1, ..., c+n-1
+///   2. `c_first c_last w` — CIDs c_first..c_last all have width w
+fn w_array_covers_cid(cid_font_dict: &lopdf::Dictionary, doc: &Document, target: u16) -> bool {
+    let Ok(w_obj) = cid_font_dict.get(b"W") else {
+        return false;
+    };
+    let arr = match w_obj {
+        Object::Array(arr) => arr,
+        Object::Reference(r) => match doc.get_object(*r) {
+            Ok(Object::Array(arr)) => arr,
+            _ => return false,
+        },
+        _ => return false,
+    };
+
+    let resolve_int = |o: &Object| -> Option<i64> {
+        match o {
+            Object::Integer(n) => Some(*n),
+            Object::Reference(r) => match doc.get_object(*r) {
+                Ok(Object::Integer(n)) => Some(*n),
+                _ => None,
+            },
+            _ => None,
+        }
+    };
+
+    let resolve_arr = |o: &Object| -> Option<Vec<Object>> {
+        match o {
+            Object::Array(a) => Some(a.clone()),
+            Object::Reference(r) => match doc.get_object(*r) {
+                Ok(Object::Array(a)) => Some(a.clone()),
+                _ => None,
+            },
+            _ => None,
+        }
+    };
+
+    let target = target as i64;
+    let mut i = 0usize;
+    while i < arr.len() {
+        let Some(first) = resolve_int(&arr[i]) else {
+            break;
+        };
+        i += 1;
+        if i >= arr.len() {
+            break;
+        }
+        // Peek at arr[i] to decide format.
+        if let Some(widths) = resolve_arr(&arr[i]) {
+            // Format 1: c [w1 ... wn]
+            let last = first + widths.len() as i64 - 1;
+            if target >= first && target <= last {
+                return true;
+            }
+            i += 1;
+        } else if let Some(last) = resolve_int(&arr[i]) {
+            // Format 2: c_first c_last w
+            i += 1;
+            if i < arr.len() {
+                i += 1; // skip the width value
+            }
+            if target >= first && target <= last {
+                return true;
+            }
+        } else {
+            // Unknown token — abort parsing safely
+            break;
+        }
+    }
+    false
+}
+
 /// Extract CIDToGIDMap as a vector of GIDs (u16) indexed by CID.
 fn get_cid_to_gid_map(cid_font_dict: &lopdf::Dictionary, doc: &Document) -> Option<Vec<u16>> {
     let obj = cid_font_dict.get(b"CIDToGIDMap").ok()?;
@@ -752,6 +839,20 @@ fn try_remap_subset_cmap(
         _ => return (cmap, None),
     };
 
+    // If the W array actually covers the CMap's max source CID, the CMap is
+    // aligned with the font — no sequential renumbering happened. A sparse W
+    // array starting at CID 0 (for .notdef) with additional high-CID entries
+    // matching the CMap is the normal subset layout, not a mismatch.
+    if let Some(max_cid) = cmap.max_source_cid() {
+        if w_array_covers_cid(cid_font_dict, doc, max_cid) {
+            debug!(
+                "Subset remap skipped for obj={}: W array covers CMap max CID {}",
+                obj_num, max_cid
+            );
+            return (cmap, None);
+        }
+    }
+
     debug!(
         "Subset GID mismatch detected for obj={}: W starts at CID {}, CMap min CID {}. Remapping to sequential.",
         obj_num, w_start, min_cid
@@ -2717,4 +2818,187 @@ endbfchar
         assert_eq!(remapped.unwrap().char_map.len(), 50);
         assert_eq!(fallback.unwrap().char_map.len(), 10);
     }
+
+    #[test]
+    fn test_max_source_cid() {
+        let cmap_content = r#"
+1 begincodespacerange
+<0000><FFFF>
+endcodespacerange
+2 beginbfchar
+<0003> <0020>
+<0031> <004E>
+endbfchar
+1 beginbfrange
+<0208> <0227> <0430>
+endbfrange
+"#;
+        let cmap = ToUnicodeCMap::parse(cmap_content.as_bytes()).unwrap();
+        assert_eq!(cmap.min_source_cid(), Some(0x0003));
+        assert_eq!(cmap.max_source_cid(), Some(0x0227));
+    }
+
+    /// Helper: build a minimal CIDFont dict with a W array and check coverage.
+    fn cid_font_dict_with_w(w_items: Vec<lopdf::Object>) -> lopdf::Dictionary {
+        let mut d = lopdf::Dictionary::new();
+        d.set("W", lopdf::Object::Array(w_items));
+        d
+    }
+
+    #[test]
+    fn test_w_array_covers_cid_format1() {
+        // Format 1: `c [w1 w2 ... wn]` — widths for CIDs c..c+n-1.
+        // Mimics the 16.pdf Tahoma W array: 0[1000] 3[313] 5[401] 11[383 383] 16[363 303 382]
+        let doc = Document::new();
+        let d = cid_font_dict_with_w(vec![
+            lopdf::Object::Integer(0),
+            lopdf::Object::Array(vec![lopdf::Object::Integer(1000)]),
+            lopdf::Object::Integer(3),
+            lopdf::Object::Array(vec![lopdf::Object::Integer(313)]),
+            lopdf::Object::Integer(5),
+            lopdf::Object::Array(vec![lopdf::Object::Integer(401)]),
+            lopdf::Object::Integer(11),
+            lopdf::Object::Array(vec![
+                lopdf::Object::Integer(383),
+                lopdf::Object::Integer(383),
+            ]),
+            lopdf::Object::Integer(16),
+            lopdf::Object::Array(vec![
+                lopdf::Object::Integer(363),
+                lopdf::Object::Integer(303),
+                lopdf::Object::Integer(382),
+            ]),
+            lopdf::Object::Integer(570),
+            lopdf::Object::Array(vec![lopdf::Object::Integer(667); 26]),
+        ]);
+
+        assert!(w_array_covers_cid(&d, &doc, 0));
+        assert!(w_array_covers_cid(&d, &doc, 3));
+        assert!(w_array_covers_cid(&d, &doc, 5));
+        assert!(w_array_covers_cid(&d, &doc, 11));
+        assert!(w_array_covers_cid(&d, &doc, 12));
+        assert!(w_array_covers_cid(&d, &doc, 16));
+        assert!(w_array_covers_cid(&d, &doc, 18));
+        assert!(w_array_covers_cid(&d, &doc, 570));
+        assert!(w_array_covers_cid(&d, &doc, 595));
+        // Gaps are NOT covered
+        assert!(!w_array_covers_cid(&d, &doc, 1));
+        assert!(!w_array_covers_cid(&d, &doc, 4));
+        assert!(!w_array_covers_cid(&d, &doc, 19));
+        assert!(!w_array_covers_cid(&d, &doc, 596));
+    }
+
+    #[test]
+    fn test_w_array_covers_cid_format2() {
+        // Format 2: `c_first c_last w` — CIDs c_first..c_last all have width w.
+        let doc = Document::new();
+        let d = cid_font_dict_with_w(vec![
+            lopdf::Object::Integer(100),
+            lopdf::Object::Integer(120),
+            lopdf::Object::Integer(500),
+        ]);
+
+        assert!(w_array_covers_cid(&d, &doc, 100));
+        assert!(w_array_covers_cid(&d, &doc, 110));
+        assert!(w_array_covers_cid(&d, &doc, 120));
+        assert!(!w_array_covers_cid(&d, &doc, 99));
+        assert!(!w_array_covers_cid(&d, &doc, 121));
+    }
+
+    #[test]
+    fn test_w_array_covers_cid_missing_w() {
+        let doc = Document::new();
+        let d = lopdf::Dictionary::new();
+        assert!(!w_array_covers_cid(&d, &doc, 3));
+    }
+
+    #[test]
+    fn test_try_remap_skipped_when_w_covers_cmap() {
+        // Simulates 16.pdf: CMap's max source CID (0x0279 = 633) is explicitly
+        // in the W array, so no subset-renumbering happened — remap must NOT fire.
+        let cmap_content = r#"
+1 begincodespacerange
+<0000><FFFF>
+endcodespacerange
+2 beginbfchar
+<0003> <0020>
+<0031> <004E>
+endbfchar
+2 beginbfrange
+<023A> <0253> <0410>
+<0255> <0279> <042B>
+endbfrange
+"#;
+        let cmap = ToUnicodeCMap::parse(cmap_content.as_bytes()).unwrap();
+
+        let mut doc = Document::new();
+        // Build a CIDFont dict with Identity CIDToGIDMap and a W array that
+        // covers CID 633 via `597 [widths...]`.
+        let mut cid_font = lopdf::Dictionary::new();
+        cid_font.set("CIDToGIDMap", lopdf::Object::Name(b"Identity".to_vec()));
+        cid_font.set(
+            "W",
+            lopdf::Object::Array(vec![
+                lopdf::Object::Integer(0),
+                lopdf::Object::Array(vec![lopdf::Object::Integer(750)]),
+                lopdf::Object::Integer(597),
+                lopdf::Object::Array(vec![lopdf::Object::Integer(500); 37]), // 597..633
+            ]),
+        );
+        let cid_font_id = doc.add_object(cid_font);
+
+        // Build the Type0 font dict with Identity-H + DescendantFonts ref.
+        let mut font_dict = lopdf::Dictionary::new();
+        font_dict.set("Encoding", lopdf::Object::Name(b"Identity-H".to_vec()));
+        font_dict.set(
+            "DescendantFonts",
+            lopdf::Object::Array(vec![lopdf::Object::Reference(cid_font_id)]),
+        );
+
+        let (primary, remapped) = try_remap_subset_cmap(cmap, &font_dict, &doc, 123);
+        assert!(
+            remapped.is_none(),
+            "Remap must be skipped when W covers CMap max CID (this is 16.pdf)"
+        );
+        assert_eq!(primary.lookup(0x0003), Some(" ".to_string()));
+    }
+
+    #[test]
+    fn test_try_remap_fires_for_true_subset_mismatch() {
+        // True mismatch: CMap has high CIDs (512-544) but W only lists low sequential CIDs.
+        let cmap_content = r#"
+1 begincodespacerange
+<0000><FFFF>
+endcodespacerange
+1 beginbfrange
+<0200> <0220> <0410>
+endbfrange
+"#;
+        let cmap = ToUnicodeCMap::parse(cmap_content.as_bytes()).unwrap();
+
+        let mut doc = Document::new();
+        let mut cid_font = lopdf::Dictionary::new();
+        cid_font.set("CIDToGIDMap", lopdf::Object::Name(b"Identity".to_vec()));
+        cid_font.set(
+            "W",
+            lopdf::Object::Array(vec![
+                lopdf::Object::Integer(0),
+                lopdf::Object::Array(vec![lopdf::Object::Integer(500); 34]), // 0..33
+            ]),
+        );
+        let cid_font_id = doc.add_object(cid_font);
+
+        let mut font_dict = lopdf::Dictionary::new();
+        font_dict.set("Encoding", lopdf::Object::Name(b"Identity-H".to_vec()));
+        font_dict.set(
+            "DescendantFonts",
+            lopdf::Object::Array(vec![lopdf::Object::Reference(cid_font_id)]),
+        );
+
+        let (_primary, remapped) = try_remap_subset_cmap(cmap, &font_dict, &doc, 456);
+        assert!(
+            remapped.is_some(),
+            "Remap must fire when CMap's CIDs are outside W array coverage"
+        );
+    }
 }

tests/integration_tests.rs

@@ -4,9 +4,10 @@ use pdf_inspector::detector::{DetectionConfig, ScanStrategy};
 use pdf_inspector::extractor::group_into_lines;
 use pdf_inspector::types::TextLine;
 use pdf_inspector::{
-    detect_pdf_type, extract_text, extract_text_in_regions_mem, extract_text_with_positions,
-    process_pdf_mem, process_pdf_with_options, to_markdown, MarkdownOptions, PdfError, PdfOptions,
-    PdfType, TextItem,
+    detect_pdf_type, extract_pages_markdown, extract_pages_markdown_mem,
+    extract_tables_in_regions_mem, extract_text, extract_text_in_regions_mem,
+    extract_text_with_positions, process_pdf_mem, process_pdf_with_options, to_markdown,
+    MarkdownOptions, PdfError, PdfOptions, PdfType, TextItem,
 };
 use std::collections::HashSet;
 
@@ -1344,3 +1345,760 @@ fn test_extract_regions_fast_vs_normal_comparison() {
         }
     }
 }
+
+// =========================================================================
+// extract_tables_in_regions_mem tests
+// =========================================================================
+
+#[test]
+fn test_extract_tables_in_regions_table_pdf() {
+    // tnagriculture has a clear table with district names and spice columns
+    let buf = std::fs::read("tests/fixtures/tnagriculture_06_12.pdf").unwrap();
+    let results =
+        extract_tables_in_regions_mem(&buf, &[(0, vec![[0.0, 0.0, 1200.0, 1200.0]])]).unwrap();
+
+    assert_eq!(results.len(), 1);
+    assert_eq!(results[0].regions.len(), 1);
+
+    let region = &results[0].regions[0];
+    // Should detect a table with pipe-delimited markdown
+    if !region.needs_ocr {
+        assert!(
+            region.text.contains('|'),
+            "Table output should contain pipe delimiters"
+        );
+        // Should have separator row
+        assert!(
+            region.text.lines().any(|l| l.contains("---")),
+            "Table output should contain separator row"
+        );
+    }
+}
+
+#[test]
+fn test_extract_tables_in_regions_non_table_region() {
+    // Use a small region that likely won't contain enough items for a table
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let results =
+        extract_tables_in_regions_mem(&buf, &[(0, vec![[0.0, 0.0, 50.0, 50.0]])]).unwrap();
+
+    assert_eq!(results.len(), 1);
+    assert_eq!(results[0].regions.len(), 1);
+
+    let region = &results[0].regions[0];
+    // Small region with few items should fall back to needs_ocr
+    assert!(
+        region.needs_ocr,
+        "Non-table region should set needs_ocr = true"
+    );
+    assert!(
+        region.text.is_empty(),
+        "Non-table region should have empty text"
+    );
+}
+
+#[test]
+fn test_extract_tables_in_regions_empty_region() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let results = extract_tables_in_regions_mem(&buf, &[(0, vec![[0.0, 0.0, 0.0, 0.0]])]).unwrap();
+
+    assert_eq!(results.len(), 1);
+    let region = &results[0].regions[0];
+    assert!(region.needs_ocr);
+    assert!(region.text.is_empty());
+}
+
+#[test]
+fn test_extract_tables_in_regions_identity_h_needs_ocr() {
+    let buf = std::fs::read("tests/fixtures/shinagawa_identity_h.pdf").unwrap();
+    let results =
+        extract_tables_in_regions_mem(&buf, &[(0, vec![[0.0, 0.0, 1200.0, 1200.0]])]).unwrap();
+
+    assert_eq!(results.len(), 1);
+    let region = &results[0].regions[0];
+    assert!(region.needs_ocr, "Identity-H font should trigger needs_ocr");
+}
+
+#[test]
+fn test_extract_tables_in_regions_not_a_pdf() {
+    let result =
+        extract_tables_in_regions_mem(b"not a pdf", &[(0, vec![[0.0, 0.0, 100.0, 100.0]])]);
+    assert!(result.is_err());
+}
+
+#[test]
+fn test_extract_tables_in_regions_nonexistent_page() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let results =
+        extract_tables_in_regions_mem(&buf, &[(9999, vec![[0.0, 0.0, 1200.0, 1200.0]])]).unwrap();
+
+    assert_eq!(results.len(), 1);
+    let region = &results[0].regions[0];
+    assert!(region.needs_ocr);
+    assert!(region.text.is_empty());
+}
+
+#[test]
+fn test_bits_pilani_page4_table_detection() {
+    // Page 4 (0-indexed 3) has a table with multi-line wrapped headers and
+    // numeric data columns. The heuristic detector previously failed because:
+    // 1. Header items at different X positions than data created extra column
+    //    clusters (6 cols instead of 4)
+    // 2. Spanning super-header row ("First Degree | First Degree") produced
+    //    duplicate header cells that looks_like_partial_table_ex rejected
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+    let results =
+        extract_tables_in_regions_mem(&buf, &[(3, vec![[0.0, 0.0, 612.0, 792.0]])]).unwrap();
+    assert_eq!(results.len(), 1);
+    let region = &results[0].regions[0];
+    assert!(
+        !region.needs_ocr,
+        "Page 4 table should be detected, got needs_ocr=true"
+    );
+    assert!(
+        region.text.contains("BIO"),
+        "Should contain department name BIO"
+    );
+    assert!(region.text.contains("8.23"), "Should contain numeric data");
+}
+
+#[test]
+fn test_bits_pilani_page8_table_detection() {
+    // Page 8 (0-indexed 7) has a numbered-row table that already worked.
+    // Verify it still works after changes.
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+    let results =
+        extract_tables_in_regions_mem(&buf, &[(7, vec![[0.0, 0.0, 612.0, 792.0]])]).unwrap();
+    assert_eq!(results.len(), 1);
+    let region = &results[0].regions[0];
+    assert!(!region.needs_ocr, "Page 8 table should still be detected");
+}
+
+// =========================================================================
+// extract_tables_with_structure_mem tests (TSR-aware path)
+// =========================================================================
+
+/// Build an 8-element 4-corner polygon `[x1,y1, x2,y1, x2,y2, x1,y2]` from
+/// an axis-aligned rect — matches the format SLANet emits for cell bboxes.
+fn poly(x1: f32, y1: f32, x2: f32, y2: f32) -> Vec<f32> {
+    vec![x1, y1, x2, y1, x2, y2, x1, y2]
+}
+
+fn synthetic_dense_table_pdf() -> Vec<u8> {
+    use lopdf::content::{Content, Operation};
+    use lopdf::{dictionary, Document, Object, Stream};
+
+    let mut doc = Document::with_version("1.5");
+    let pages_id = doc.new_object_id();
+    let page_id = doc.new_object_id();
+    let font_id = doc.new_object_id();
+    let content_id = doc.new_object_id();
+
+    doc.objects.insert(
+        font_id,
+        dictionary! {
+            "Type" => "Font",
+            "Subtype" => "Type1",
+            "BaseFont" => "Helvetica",
+        }
+        .into(),
+    );
+
+    let operations = vec![
+        Operation::new("BT", vec![]),
+        Operation::new("Tf", vec!["F1".into(), 10.into()]),
+        Operation::new("Td", vec![20.into(), 700.into()]),
+        Operation::new("Tj", vec![Object::string_literal("Branch Name")]),
+        Operation::new("Td", vec![100.into(), 0.into()]),
+        Operation::new("Tj", vec![Object::string_literal("Deposits")]),
+        Operation::new("Td", vec![Object::Integer(-100), Object::Real(-16.8)]),
+        Operation::new("Tj", vec![Object::string_literal("Oak Street")]),
+        Operation::new("Td", vec![100.into(), 0.into()]),
+        Operation::new("Tj", vec![Object::string_literal("100")]),
+        Operation::new("Td", vec![Object::Integer(-100), Object::Real(-16.8)]),
+        Operation::new("Tj", vec![Object::string_literal("Boardwalk")]),
+        Operation::new("Td", vec![100.into(), 0.into()]),
+        Operation::new("Tj", vec![Object::string_literal("200")]),
+        Operation::new("ET", vec![]),
+    ];
+    let content = Content { operations }.encode().unwrap();
+    doc.objects
+        .insert(content_id, Stream::new(dictionary! {}, content).into());
+
+    doc.objects.insert(
+        page_id,
+        dictionary! {
+            "Type" => "Page",
+            "Parent" => pages_id,
+            "MediaBox" => vec![0.into(), 0.into(), 200.into(), 800.into()],
+            "Resources" => dictionary! {
+                "Font" => dictionary! {
+                    "F1" => font_id,
+                },
+            },
+            "Contents" => content_id,
+        }
+        .into(),
+    );
+    doc.objects.insert(
+        pages_id,
+        dictionary! {
+            "Type" => "Pages",
+            "Kids" => vec![page_id.into()],
+            "Count" => 1,
+        }
+        .into(),
+    );
+    let catalog_id = doc.add_object(dictionary! {
+        "Type" => "Catalog",
+        "Pages" => pages_id,
+    });
+    doc.trailer.set("Root", catalog_id);
+
+    let mut bytes = Vec::new();
+    doc.save_to(&mut bytes).unwrap();
+    bytes
+}
+
+#[test]
+fn test_extract_tables_with_structure_real_pdf_bits_pilani() {
+    use pdf_inspector::{extract_tables_with_structure_mem, TsrTableInput};
+    // Hand-crafted TSR fixture targeting page 4 (0-indexed=3) of
+    // bits_pilani_feedback.pdf, which contains a clean tabular layout.
+    //
+    // We construct a 2×2 table:
+    //   row 0 (header): "Department"   "Core Courses"
+    //   row 1 (data):   "BIO"          "8.23"
+    //
+    // The PDF page is US Letter (792pt tall). We render at 72 dpi so
+    // image-px maps 1:1 to PDF-pt — that lets us write cell bboxes in
+    // the same units as our hand-measured page-pt coordinates.
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+
+    // The PDF page is A4 in points (≈595.44 × 841.68). The table sits in
+    // the upper part of the page; we crop a window large enough to enclose
+    // both rows we care about.
+    //
+    // Crop bounds in PDF points (top-left origin):
+    //   x: 80..280, y: 170..240
+    let crop = [80.0_f32, 170.0, 280.0, 240.0];
+    let dpi = 72.0_f32;
+
+    // Cell bboxes in CROP image-pixel space (= crop-relative PDF-pt at
+    // 72 dpi). The y ranges are tightened against neighbouring rows
+    // ("First Degree" above the header at native y=666.7, "Feedback Score"
+    // between the header and data rows at native y=640.9, "CE" below the
+    // BIO row at native y=591.1) so each cell only overlaps its target
+    // text item.
+    let cell_bboxes = vec![
+        // Header row: y crop-relative (7, 18) → page-pt y (177, 188)
+        poly(10.0, 7.0, 100.0, 18.0), // "Department"   (item at page-pt x=107.1)
+        poly(110.0, 7.0, 200.0, 18.0), // "Core Courses" (item at page-pt x=199.0)
+        // Data row: y crop-relative (35, 60) → page-pt y (205, 230)
+        poly(10.0, 35.0, 100.0, 60.0), // "BIO"  (item at page-pt x=104.1)
+        poly(110.0, 35.0, 200.0, 60.0), // "8.23" (item at page-pt x=221.2)
+    ];
+
+    // Minimal SLANet-style token stream: a 2-row table with a thead and tbody.
+    let tokens: Vec<String> = [
+        "<table>",
+        "<thead>",
+        "<tr>",
+        "<th></th>",
+        "<th></th>",
+        "</tr>",
+        "</thead>",
+        "<tbody>",
+        "<tr>",
+        "<td></td>",
+        "<td></td>",
+        "</tr>",
+        "</tbody>",
+        "</table>",
+    ]
+    .into_iter()
+    .map(String::from)
+    .collect();
+
+    let inputs = vec![TsrTableInput {
+        page: 3,
+        crop_pdf_pt_bbox: crop,
+        render_dpi: dpi,
+        structure_tokens: tokens,
+        cell_bboxes,
+    }];
+
+    let mds = extract_tables_with_structure_mem(&buf, &inputs).unwrap();
+    assert_eq!(mds.len(), 1);
+    let md = &mds[0];
+
+    // Hand-written gold standard for the rendered markdown.
+    let expected = "|Department|Core Courses|\n|---|---|\n|BIO|8.23|\n";
+    assert_eq!(
+        md, expected,
+        "structured-table markdown should match the gold standard exactly\nactual: {md}"
+    );
+}
+
+#[test]
+fn test_extract_tables_with_structure_dense_overlapping_slanet_boxes() {
+    use pdf_inspector::{extract_tables_with_structure_mem, TsrTableInput};
+
+    let buf = synthetic_dense_table_pdf();
+    let tokens: Vec<String> = [
+        "<table>",
+        "<thead>",
+        "<tr>",
+        "<th></th>",
+        "<th></th>",
+        "</tr>",
+        "</thead>",
+        "<tbody>",
+        "<tr>",
+        "<td></td>",
+        "<td></td>",
+        "</tr>",
+        "<tr>",
+        "<td></td>",
+        "<td></td>",
+        "</tr>",
+        "</tbody>",
+        "</table>",
+    ]
+    .into_iter()
+    .map(String::from)
+    .collect();
+
+    // Rows are spaced 16.8pt apart, while the SLANet-style boxes are 40pt
+    // tall and overlap adjacent rows. Text must still land in only one row.
+    let cell_bboxes = vec![
+        poly(10.0, 72.0, 100.0, 112.0),
+        poly(90.0, 72.0, 180.0, 112.0),
+        poly(10.0, 88.8, 100.0, 128.8),
+        poly(90.0, 88.8, 180.0, 128.8),
+        poly(10.0, 105.6, 100.0, 145.6),
+        poly(90.0, 105.6, 180.0, 145.6),
+    ];
+
+    let mds = extract_tables_with_structure_mem(
+        &buf,
+        &[TsrTableInput {
+            page: 0,
+            crop_pdf_pt_bbox: [0.0, 0.0, 200.0, 800.0],
+            render_dpi: 72.0,
+            structure_tokens: tokens,
+            cell_bboxes,
+        }],
+    )
+    .unwrap();
+
+    let expected = "|Branch Name|Deposits|\n|---|---|\n|Oak Street|100|\n|Boardwalk|200|\n";
+    assert_eq!(mds[0], expected);
+    assert!(!mds[0].contains("Branch Name Oak Street"));
+    assert!(!mds[0].contains("Oak Street Boardwalk"));
+}
+
+#[test]
+fn test_extract_tables_with_structure_input_order_preserved() {
+    use pdf_inspector::{extract_tables_with_structure_mem, TsrTableInput};
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+
+    // Two inputs; both target the same page but with different shapes.
+    // We just need to confirm we get 2 outputs in the same order.
+    let make_input = |toks: Vec<&str>, cells: Vec<Vec<f32>>| TsrTableInput {
+        page: 3,
+        crop_pdf_pt_bbox: [80.0, 170.0, 280.0, 240.0],
+        render_dpi: 72.0,
+        structure_tokens: toks.into_iter().map(String::from).collect(),
+        cell_bboxes: cells,
+    };
+
+    let inputs = vec![
+        make_input(
+            vec!["<table>", "<tr>", "<td></td>", "</tr>", "</table>"],
+            vec![poly(10.0, 35.0, 100.0, 60.0)],
+        ),
+        make_input(
+            vec!["<table>", "<tr>", "<td></td>", "</tr>", "</table>"],
+            vec![poly(110.0, 35.0, 200.0, 60.0)],
+        ),
+    ];
+
+    let mds = extract_tables_with_structure_mem(&buf, &inputs).unwrap();
+    assert_eq!(mds.len(), 2);
+    assert!(
+        mds[0].contains("BIO"),
+        "input 0 should pull 'BIO': {}",
+        mds[0]
+    );
+    assert!(
+        mds[1].contains("8.23"),
+        "input 1 should pull '8.23': {}",
+        mds[1]
+    );
+}
+
+#[test]
+fn test_extract_tables_with_structure_out_of_range_page() {
+    use pdf_inspector::{extract_tables_with_structure_mem, TsrTableInput};
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+
+    let inputs = vec![TsrTableInput {
+        page: 9999,
+        crop_pdf_pt_bbox: [0.0, 0.0, 100.0, 100.0],
+        render_dpi: 72.0,
+        structure_tokens: vec![
+            "<table>".into(),
+            "<tr>".into(),
+            "<td></td>".into(),
+            "</tr>".into(),
+            "</table>".into(),
+        ],
+        cell_bboxes: vec![poly(0.0, 0.0, 50.0, 50.0)],
+    }];
+
+    let mds = extract_tables_with_structure_mem(&buf, &inputs).unwrap();
+    assert_eq!(mds.len(), 1);
+    assert!(
+        mds[0].is_empty(),
+        "out-of-range page should yield empty string"
+    );
+}
+
+#[test]
+fn test_extract_tables_with_structure_not_a_pdf() {
+    use pdf_inspector::extract_tables_with_structure_mem;
+    let result = extract_tables_with_structure_mem(b"not a pdf", &[]);
+    assert!(result.is_err());
+}
+
+#[test]
+fn test_extract_tables_with_structure_empty_inputs() {
+    use pdf_inspector::extract_tables_with_structure_mem;
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+    let mds = extract_tables_with_structure_mem(&buf, &[]).unwrap();
+    assert!(mds.is_empty());
+}
+
+#[test]
+fn test_extract_tables_with_structure_cells_real_pdf_bits_pilani() {
+    use pdf_inspector::{extract_tables_with_structure_cells_mem, TsrTableInput};
+    // Same fixture as test_extract_tables_with_structure_real_pdf_bits_pilani
+    // but exercising the cell-level API. Verifies that callers receive
+    // structured per-cell metadata (row/col/spans/is_header/page_pt_bbox)
+    // alongside the extracted text.
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+
+    let crop = [80.0_f32, 170.0, 280.0, 240.0];
+    let dpi = 72.0_f32;
+    let cell_bboxes = vec![
+        poly(10.0, 7.0, 100.0, 18.0),
+        poly(110.0, 7.0, 200.0, 18.0),
+        poly(10.0, 35.0, 100.0, 60.0),
+        poly(110.0, 35.0, 200.0, 60.0),
+    ];
+    let tokens: Vec<String> = [
+        "<table>",
+        "<thead>",
+        "<tr>",
+        "<th></th>",
+        "<th></th>",
+        "</tr>",
+        "</thead>",
+        "<tbody>",
+        "<tr>",
+        "<td></td>",
+        "<td></td>",
+        "</tr>",
+        "</tbody>",
+        "</table>",
+    ]
+    .into_iter()
+    .map(String::from)
+    .collect();
+
+    let inputs = vec![TsrTableInput {
+        page: 3,
+        crop_pdf_pt_bbox: crop,
+        render_dpi: dpi,
+        structure_tokens: tokens,
+        cell_bboxes,
+    }];
+
+    let cells_lists = extract_tables_with_structure_cells_mem(&buf, &inputs).unwrap();
+    assert_eq!(cells_lists.len(), 1);
+    let cells = &cells_lists[0];
+    assert_eq!(cells.len(), 4);
+
+    // Header row: both cells flagged as headers (they were in <thead>/<th>).
+    assert!(cells[0].is_header);
+    assert!(cells[1].is_header);
+    assert_eq!((cells[0].row, cells[0].col), (0, 0));
+    assert_eq!((cells[1].row, cells[1].col), (0, 1));
+    assert_eq!(cells[0].text, "Department");
+    assert_eq!(cells[1].text, "Core Courses");
+
+    // Data row: not flagged as header.
+    assert!(!cells[2].is_header);
+    assert!(!cells[3].is_header);
+    assert_eq!((cells[2].row, cells[2].col), (1, 0));
+    assert_eq!((cells[3].row, cells[3].col), (1, 1));
+    assert_eq!(cells[2].text, "BIO");
+    assert_eq!(cells[3].text, "8.23");
+
+    // Every cell carries a non-degenerate page-pt bbox.
+    for c in cells {
+        let [x1, y1, x2, y2] = c.page_pt_bbox;
+        assert!(
+            x1 < x2 && y1 < y2,
+            "cell bbox should be non-empty: {:?}",
+            c.page_pt_bbox
+        );
+    }
+}
+
+#[test]
+fn test_extract_tables_with_structure_separator_after_thead() {
+    use pdf_inspector::{extract_tables_with_structure_mem, TsrTableInput};
+    // Re-run the same 2x2 fixture but assert exact markdown output: with
+    // <thead> + <th> headers, the separator should land after the header
+    // row (which is also row 0 here, so the gold-standard hasn't changed).
+    let buf = std::fs::read("tests/fixtures/bits_pilani_feedback.pdf").unwrap();
+
+    let crop = [80.0_f32, 170.0, 280.0, 240.0];
+    let dpi = 72.0_f32;
+    let cell_bboxes = vec![
+        poly(10.0, 7.0, 100.0, 18.0),
+        poly(110.0, 7.0, 200.0, 18.0),
+        poly(10.0, 35.0, 100.0, 60.0),
+        poly(110.0, 35.0, 200.0, 60.0),
+    ];
+    let tokens: Vec<String> = [
+        "<table>",
+        "<thead>",
+        "<tr>",
+        "<th></th>",
+        "<th></th>",
+        "</tr>",
+        "</thead>",
+        "<tbody>",
+        "<tr>",
+        "<td></td>",
+        "<td></td>",
+        "</tr>",
+        "</tbody>",
+        "</table>",
+    ]
+    .into_iter()
+    .map(String::from)
+    .collect();
+
+    let mds = extract_tables_with_structure_mem(
+        &buf,
+        &[TsrTableInput {
+            page: 3,
+            crop_pdf_pt_bbox: crop,
+            render_dpi: dpi,
+            structure_tokens: tokens,
+            cell_bboxes,
+        }],
+    )
+    .unwrap();
+    assert_eq!(mds.len(), 1);
+    assert_eq!(mds[0], "|Department|Core Courses|\n|---|---|\n|BIO|8.23|\n");
+}
+
+// =========================================================================
+// extract_pages_markdown_mem tests
+// =========================================================================
+
+#[test]
+fn test_extract_pages_markdown_basic() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let result = extract_pages_markdown_mem(&buf, Some(&[0, 1])).unwrap();
+
+    assert_eq!(result.pages.len(), 2);
+    assert_eq!(result.pages[0].page, 0);
+    assert_eq!(result.pages[1].page, 1);
+    // Text-based PDF should produce non-empty markdown
+    assert!(!result.pages[0].markdown.is_empty());
+    assert!(!result.pages[0].needs_ocr);
+}
+
+#[test]
+fn test_extract_pages_markdown_page_ordering() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    // Request pages in non-sequential order
+    let result = extract_pages_markdown_mem(&buf, Some(&[1, 0])).unwrap();
+
+    assert_eq!(result.pages.len(), 2);
+    // Results should match input order, not document order
+    assert_eq!(result.pages[0].page, 1);
+    assert_eq!(result.pages[1].page, 0);
+}
+
+#[test]
+fn test_extract_pages_markdown_out_of_range() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let result = extract_pages_markdown_mem(&buf, Some(&[9999])).unwrap();
+
+    assert_eq!(result.pages.len(), 1);
+    assert_eq!(result.pages[0].page, 9999);
+    assert!(result.pages[0].markdown.is_empty());
+    assert!(result.pages[0].needs_ocr);
+    assert!(result.pages_needing_ocr.contains(&10000)); // 1-indexed
+}
+
+#[test]
+fn test_extract_pages_markdown_empty_pages_list() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let result = extract_pages_markdown_mem(&buf, Some(&[])).unwrap();
+    assert!(result.pages.is_empty());
+}
+
+#[test]
+fn test_extract_pages_markdown_single_page() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let result = extract_pages_markdown_mem(&buf, Some(&[0])).unwrap();
+
+    assert_eq!(result.pages.len(), 1);
+    assert_eq!(result.pages[0].page, 0);
+    assert!(!result.pages[0].markdown.is_empty());
+    assert!(!result.pages[0].needs_ocr);
+}
+
+#[test]
+fn test_extract_pages_markdown_invalid_buffer() {
+    let result = extract_pages_markdown_mem(b"not a pdf", Some(&[0]));
+    assert!(result.is_err());
+}
+
+#[test]
+fn test_extract_pages_markdown_gid_pages_need_ocr() {
+    // shinagawa_identity_h.pdf has GID-encoded fonts
+    let buf = std::fs::read("tests/fixtures/shinagawa_identity_h.pdf").unwrap();
+    let result = extract_pages_markdown_mem(&buf, Some(&[0])).unwrap();
+
+    assert_eq!(result.pages.len(), 1);
+    assert!(result.pages[0].needs_ocr);
+    assert!(result.pages_needing_ocr.contains(&1)); // 1-indexed
+}
+
+#[test]
+fn test_extract_pages_markdown_classification_with_tables() {
+    // nexo-price-en.pdf is known to have tables
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let page_count = process_pdf_mem(&buf).unwrap().page_count;
+    let page_indices: Vec<u32> = (0..page_count).collect();
+    let result = extract_pages_markdown_mem(&buf, Some(&page_indices)).unwrap();
+
+    assert!(
+        !result.pages_with_tables.is_empty(),
+        "nexo-price-en.pdf should have pages with tables"
+    );
+    assert!(result.is_complex);
+}
+
+#[test]
+fn test_extract_pages_markdown_simple_pdf_no_complexity() {
+    // bare_name_struct.pdf is a simple document with a heading and code block
+    let buf = std::fs::read("tests/fixtures/bare_name_struct.pdf").unwrap();
+    let result = extract_pages_markdown_mem(&buf, Some(&[0])).unwrap();
+
+    assert!(result.pages_with_tables.is_empty());
+    assert!(result.pages_with_columns.is_empty());
+    assert!(!result.is_complex);
+}
+
+#[test]
+fn test_extract_pages_markdown_classification_matches_process_pdf() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let full = process_pdf_mem(&buf).unwrap();
+    let page_count = full.page_count;
+    let page_indices: Vec<u32> = (0..page_count).collect();
+    let result = extract_pages_markdown_mem(&buf, Some(&page_indices)).unwrap();
+
+    assert_eq!(
+        result.pages_with_tables, full.layout.pages_with_tables,
+        "pages_with_tables should match process_pdf"
+    );
+    assert_eq!(
+        result.pages_with_columns, full.layout.pages_with_columns,
+        "pages_with_columns should match process_pdf"
+    );
+}
+
+#[test]
+fn test_extract_pages_markdown_consistency_with_process_pdf() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+
+    // Get full process_pdf output
+    let full = process_pdf_mem(&buf).unwrap();
+    let full_md = full.markdown.unwrap_or_default();
+
+    // Get per-page output for all pages
+    let page_count = full.page_count;
+    let page_indices: Vec<u32> = (0..page_count).collect();
+    let result = extract_pages_markdown_mem(&buf, Some(&page_indices)).unwrap();
+
+    // Concatenated per-page markdown should contain substantial overlap with
+    // the full output (exact match not expected due to header/footer stripping
+    // and cross-page paragraph merging differences)
+    let concat: String = result
+        .pages
+        .iter()
+        .map(|p| p.markdown.as_str())
+        .collect::<Vec<_>>()
+        .join("\n");
+
+    // Both should be non-empty for a text-based PDF
+    assert!(!full_md.is_empty());
+    assert!(!concat.is_empty());
+
+    // The per-page version should contain at least 50% of the full content's
+    // length (accounting for header/footer stripping differences)
+    assert!(
+        concat.len() * 2 >= full_md.len(),
+        "per-page concat ({} chars) is too short vs full ({} chars)",
+        concat.len(),
+        full_md.len()
+    );
+}
+
+#[test]
+fn test_extract_pages_markdown_none_returns_all_pages() {
+    let buf = std::fs::read("tests/fixtures/nexo-price-en.pdf").unwrap();
+    let page_count = process_pdf_mem(&buf).unwrap().page_count;
+
+    let result = extract_pages_markdown_mem(&buf, None).unwrap();
+
+    assert_eq!(result.pages.len() as u32, page_count);
+    for (i, page) in result.pages.iter().enumerate() {
+        assert_eq!(page.page, i as u32, "pages should be in document order");
+    }
+}
+
+#[test]
+fn test_extract_pages_markdown_path_api() {
+    let path = "tests/fixtures/nexo-price-en.pdf";
+    let buf = std::fs::read(path).unwrap();
+
+    let via_path = extract_pages_markdown(path, Some(&[0])).unwrap();
+    let via_mem = extract_pages_markdown_mem(&buf, Some(&[0])).unwrap();
+
+    assert_eq!(via_path.pages.len(), via_mem.pages.len());
+    assert_eq!(via_path.pages[0].markdown, via_mem.pages[0].markdown);
+    assert_eq!(via_path.pages[0].needs_ocr, via_mem.pages[0].needs_ocr);
+    assert_eq!(via_path.is_complex, via_mem.is_complex);
+}
+
+#[test]
+fn test_extract_pages_markdown_path_none_returns_all_pages() {
+    let path = "tests/fixtures/nexo-price-en.pdf";
+    let page_count = process_pdf_mem(&std::fs::read(path).unwrap())
+        .unwrap()
+        .page_count;
+
+    let result = extract_pages_markdown(path, None).unwrap();
+    assert_eq!(result.pages.len() as u32, page_count);
+}

tests/snapshots/nexo-price-en.md

@@ -1,9 +1,9 @@
 본 가격표는 국내 거주 중인 외국인을 위한 한국어 가격표의 비공식 번역본입니다. ※ The post-tax benefit sales price is provided for your reference only, reflecting the current tax benefits and eco-friendly vehicle individual consumption tax reductions. 본 가격표와 한국어 가격표의 내용이 상이한 경우 한국어 가격표의 내용이 우선하므로, 반드시 한국어 가격표의 내용을 확인하십시오. The final sales price may vary depending on the addition of optional items and whether the eco-friendly vehicle criteria are met, so please be sure to check the quotation. This price list is an unofficial translation of the Korean price list for the convenience of foreign residents in South Korea. ※ Please check the Korean price list for information on colors, details, and fuel consumption for each model. If the price list differs from the Korean price list, please check the contents of the Korean price list first. ※ All optional item prices are listed based on pre-tax reduction amounts. The actual sales price, which reflects the total individual consumption tax reduction including optional items, may differ depending on applicable tax benefits. ※ The items (specifications, colors, etc.) and prices listed in this pricing table are subject to change without prior notice depending on the holding of new car launch events, improvements made in automobile performance, introduction of related laws and regulations, and changes in company circumstances. The all-new NEXO Release Date: June 10, 2025 / (Unit: KRW)
 
-|Classification Exclusive Exclusive|Selling price before tax benefit Supply value(surtax) 80,509,000 73,190,000(7,319,000)|Selling price after tax benefit 76,435,000|Standard equipment • Powertrain/Performance: Fuel cell system(150kW drive motor, lithium-ion battery, and reducer), Regenerative braking system, Column-Type Shift By Wire(vibration warning), Drive mode select • Safety: 9 airbag system(1st-row advanced/center side airbags, 1st/2nd-row side airbags, and rollover-resistant curtain airbags), Multi-Collision Brake System, Active hood system(for pedestrian protection), Safety unlock function, Artificial engine sound(for pedestrian protection), Child seat fastening system (2 in 2nd-row), Fire extinguisher for vehicles, Pedal Misapplication Safety Assist • Smart Safety Technology: Forward Collision-avoidance Assist(vehicles/ pedestrians/two-wheeled vehicles/junction turning/front oncoming), Smart Cruise Control with Stop & Go, Lane Keeping Assist, Lane Following Assist 2, Blind-spot Collision Warning(driving), Blind-spot Collision-avoidance Assist(forward exit), Rear Cross-traffic Collision-avoidance Assist, Safety Exit Assist, Driver Attention Warning, High Beam Assist, Advanced Rear Occupant Alert, Intelligent Speed Limit Assist, Hands-On Detection, Highway Driving Assist, Navigation-based Smart Cruise Control(safety speed zone/curve control), Vibration warning steering wheel • Exterior: Full LED headlamps(projection type), LED turn signal lamps (front and rear), LED Daytime Running Lights, LED positioning lights, LED rear combination lamps, LED third brake lights, 18-inch alloy wheels & tires, Solar glass(windshield), Double-glazed soundproof glass(windshield, and 1st/ 2nd-row doors), Outside mirror(heating, power-folding, power adjustment,|Options (before tax benefit) ▶ Hi-pass(e hi-pass) [200,000]|
+|Classification|Selling price before tax benefit Supply value(surtax)|Selling price after tax benefit|Standard equipment|Options (before tax benefit)|
 |---|---|---|---|---|
-|Special|with 3.5% individual consumption tax applied 79,287,000 83,500,000 75,909,091(7,590,909) with 3.5% individual consumption tax applied 82,232,000|with 3.5% individual consumption tax applied 76,435,000 79,275,000 with 3.5% individual consumption tax applied 79,275,000|and LED turn signal lamps), Auto flush door handles, Black door garnish • Interior: Panoramic curved display, 12.3-inch color LCD cluster, Leather- upholstered steering wheel(with heating, two-tone color, and Interactive Pixel Lights), LED interior lamp (map lamp, personal lamp, sun visor lamp, and luggage lamp), Metallic door scuff plate • Seat: Synthetic leather seats, 1st-row manual seats, Heated 1st-row seats, 2nd-row 60/40-split folding seats(reclining) • Convenience: Proximity key with push-button start, Smart key remote start, Electronic Parking Brake(with automatic vehicle hold), Paddle shift(regenerative control), Dual-zone full automatic air conditioning(with high-performance antibacterial combination filter, auto defog system, fine dust sensor, air cleaning mode, and after-blow function), 2nd-row seat air vent, Auto light control system, USB Type-C Ports(1×27W switchable charging/data port in 1st-row, and 2×100W charging ports in both 1st and 2nd-row), ECM room mirror(frameless), Rain sensor, Power windows with pinch protection(1st/2nd-row), Power outlet (1 in 1st-row), Parking Distance Warning-Forward/Reverse, Rear View Monitor, Wireless phone charger(single), Walk-away lock, Route planner, Hyundai AI Assistant • Infotainment: 12.3-inch navigation(Bluelink, phone projection, Bluetooth hands-free, and In-car Payment), Audio system(6 speakers), Over-The-Air navigation updates ▶ Standard equipment of Exclusive plus • Smart Safety Technology: Forward Collision-avoidance Assist(intersection crossing/changing lanes in oncoming traffic/approaching from either side/ evasive steering assist), Highway Driving Assist 2, Navigation-based Smart Cruise Control(access road) • Exterior: Roof rack • Interior: Metallic pedal, Driving mode-dependent ambient mood lighting(crash pad, 1st/2nd-row door trim) • Seat: Synthetic leather seats(patch applied), Power-adjustable driver's|▶ [600,000] Built-in Cam 2 Plus, Augmented reality navigation ▶ [850,000] Indoor/outdoor V2L ▶ [950,000] Parking Assist ▶ [1,150,000] Audio by BANG & OLUFSEN|
-|Prestige|87,893,000 79,902,727(7,990,273) with 3.5% individual consumption tax applied 86,559,000|83,445,000 with 3.5% individual consumption tax applied 83,445,000|seat(8-way, lumbar support, and Integrated Memory System(driver's seat and outside mirror connected)), Power-adjustable front passenger’s seat(8-way), Ventilated 1st-row seats, Heated 2nd-row seats • Convenience: Hi-pass(e hi-pass), In-car fingerprint authentication system(personalization, startup, payment, and etc.), Smart power tailgate ▶ Standard equipment of Exclusive Special plus • Smart Safety Technology: Remote Smart Parking Assist 2, Parking Collison- avoidance Assist(front/side/rear) • Exterior: Intelligent Front-Lighting System(IFS), Dynamic welcome/escort lighting(1 type), Sequential turn signals(front and rear), Ambient lighting auto flush door handles, Two-tone door garnish, Glossy black rear diffuser • Interior: Recycled PET suede interior materials(headlining/sunvisor), Fabric upholstered crash pad • Seat: BIO-processed natural leather seats(metal patch applied, embossed design punching), Passenger's seat walk-in device, 1st-row relaxation comfort seats(leg rest included), Ventilated 2nd-row seats • Convenience: Parking Distance Warning-Side, Head-Up Display, Digital key 2, Wireless phone charger(dual), Surround View Monitor, Blind-spot View Monitor, LED reverse light guide • Infotainment: Audio by BANG & OLUFSEN sound system(14 speakers, including external amp), Active road noise control, Active Sound Design|sound system ▶ [250,000] 19-inch alloy wheels & tires ▶ [600,000] Built-in Cam 2 Plus, Augmented reality navigation ▶ [850,000] Indoor/outdoor V2L ▶ [900,000] Vision roof ▶ [1,380,000] Digital side mirror ▶ [750,000] Camera package ▶ [250,000] 19-inch alloy wheels & tires|
+|Exclusive|80,509,000 73,190,000(7,319,000) with 3.5% individual consumption tax applied 79,287,000|76,435,000 with 3.5% individual consumption tax applied 76,435,000|• Powertrain/Performance: Fuel cell system(150kW drive motor, lithium-ion battery, and reducer), Regenerative braking system, Column-Type Shift By Wire(vibration warning), Drive mode select • Safety: 9 airbag system(1st-row advanced/center side airbags, 1st/2nd-row side airbags, and rollover-resistant curtain airbags), Multi-Collision Brake System, Active hood system(for pedestrian protection), Safety unlock function, Artificial engine sound(for pedestrian protection), Child seat fastening system (2 in 2nd-row), Fire extinguisher for vehicles, Pedal Misapplication Safety Assist • Smart Safety Technology: Forward Collision-avoidance Assist(vehicles/ pedestrians/two-wheeled vehicles/junction turning/front oncoming), Smart Cruise Control with Stop & Go, Lane Keeping Assist, Lane Following Assist 2, Blind-spot Collision Warning(driving), Blind-spot Collision-avoidance Assist(forward exit), Rear Cross-traffic Collision-avoidance Assist, Safety Exit Assist, Driver Attention Warning, High Beam Assist, Advanced Rear Occupant Alert, Intelligent Speed Limit Assist, Hands-On Detection, Highway Driving Assist, Navigation-based Smart Cruise Control(safety speed zone/curve control), Vibration warning steering wheel • Exterior: Full LED headlamps(projection type), LED turn signal lamps (front and rear), LED Daytime Running Lights, LED positioning lights, LED rear combination lamps, LED third brake lights, 18-inch alloy wheels & tires, Solar glass(windshield), Double-glazed soundproof glass(windshield, and 1st/ 2nd-row doors), Outside mirror(heating, power-folding, power adjustment, and LED turn signal lamps), Auto flush door handles, Black door garnish • Interior: Panoramic curved display, 12.3-inch color LCD cluster, Leather- upholstered steering wheel(with heating, two-tone color, and Interactive Pixel Lights), LED interior lamp (map lamp, personal lamp, sun visor lamp, and luggage lamp), Metallic door scuff plate • Seat: Synthetic leather seats, 1st-row manual seats, Heated 1st-row seats, 2nd-row 60/40-split folding seats(reclining) • Convenience: Proximity key with push-button start, Smart key remote start, Electronic Parking Brake(with automatic vehicle hold), Paddle shift(regenerative control), Dual-zone full automatic air conditioning(with high-performance antibacterial combination filter, auto defog system, fine dust sensor, air cleaning mode, and after-blow function), 2nd-row seat air vent, Auto light control system, USB Type-C Ports(1×27W switchable charging/data port in 1st-row, and 2×100W charging ports in both 1st and 2nd-row), ECM room mirror(frameless), Rain sensor, Power windows with pinch protection(1st/2nd-row), Power outlet (1 in 1st-row), Parking Distance Warning-Forward/Reverse, Rear View Monitor, Wireless phone charger(single), Walk-away lock, Route planner, Hyundai AI Assistant • Infotainment: 12.3-inch navigation(Bluelink, phone projection, Bluetooth hands-free, and In-car Payment), Audio system(6 speakers), Over-The-Air navigation updates|▶ Hi-pass(e hi-pass) [200,000]|
+|Exclusive Special|83,500,000 75,909,091(7,590,909) with 3.5% individual consumption tax applied 82,232,000|79,275,000 with 3.5% individual consumption tax applied 79,275,000|▶ Standard equipment of Exclusive plus • Smart Safety Technology: Forward Collision-avoidance Assist(intersection crossing/changing lanes in oncoming traffic/approaching from either side/ evasive steering assist), Highway Driving Assist 2, Navigation-based Smart Cruise Control(access road) • Exterior: Roof rack • Interior: Metallic pedal, Driving mode-dependent ambient mood lighting(crash pad, 1st/2nd-row door trim) • Seat: Synthetic leather seats(patch applied), Power-adjustable driver's seat(8-way, lumbar support, and Integrated Memory System(driver's seat and outside mirror connected)), Power-adjustable front passenger’s seat(8-way), Ventilated 1st-row seats, Heated 2nd-row seats • Convenience: Hi-pass(e hi-pass), In-car fingerprint authentication system(personalization, startup, payment, and etc.), Smart power tailgate|▶ [600,000] Built-in Cam 2 Plus, Augmented reality navigation ▶ [850,000] Indoor/outdoor V2L ▶ [950,000] Parking Assist ▶ [1,150,000] Audio by BANG & OLUFSEN sound system ▶ [250,000] 19-inch alloy wheels & tires|
+|Prestige|87,893,000 79,902,727(7,990,273) with 3.5% individual consumption tax applied 86,559,000|83,445,000 with 3.5% individual consumption tax applied 83,445,000|▶ Standard equipment of Exclusive Special plus • Smart Safety Technology: Remote Smart Parking Assist 2, Parking Collison- avoidance Assist(front/side/rear) • Exterior: Intelligent Front-Lighting System(IFS), Dynamic welcome/escort lighting(1 type), Sequential turn signals(front and rear), Ambient lighting auto flush door handles, Two-tone door garnish, Glossy black rear diffuser • Interior: Recycled PET suede interior materials(headlining/sunvisor), Fabric upholstered crash pad • Seat: BIO-processed natural leather seats(metal patch applied, embossed design punching), Passenger's seat walk-in device, 1st-row relaxation comfort seats(leg rest included), Ventilated 2nd-row seats • Convenience: Parking Distance Warning-Side, Head-Up Display, Digital key 2, Wireless phone charger(dual), Surround View Monitor, Blind-spot View Monitor, LED reverse light guide • Infotainment: Audio by BANG & OLUFSEN sound system(14 speakers, including external amp), Active road noise control, Active Sound Design|▶ [600,000] Built-in Cam 2 Plus, Augmented reality navigation ▶ [850,000] Indoor/outdoor V2L ▶ [900,000] Vision roof ▶ [1,380,000] Digital side mirror ▶ [750,000] Camera package ▶ [250,000] 19-inch alloy wheels & tires|
 
 **Classification Details** **Indoor/outdoor V2L** Indoor V2L, Outdoor V2L(connectorless type) **Parking Assist** Surround View Monitor, Blind-spot View Monitor, Parking Distance Warning-Side, Parking Collison-avoidance Assist-Rear **Audio by BANG & OLUFSEN** Audio by BANG & OLUFSEN sound system(14 speakers, including external amp.), Active road noise control, Active Sound Design **sound system** **Camera package** Digital center mirror(with camera sensor cleaning system), Driver monitoring system THE ALL-NEW NEXO /// ECO-FRIENDLY CAR
-

tests/snapshots/td9264.md

@@ -13,8 +13,10 @@ IDENTIFICATION NUMBER OF CORPORATION] ELECTS TO BE TREATED AS A COMPONENT MEMBER
 (v) Election-- (A) Election filed. An election filed under paragraph (d)(2)(iv) of
 this section is irrevocable and effective until paragraph (d)(2)(ii) or (iii) of §1.1563-3 applies or until a change in the stock ownership of the corporation results in
 
-|termination of membership in the controlled group in which such corporation has been included. (B) Election not filed.|In the event no election is filed in accordance with the|
+|termination of membership in the controlled group in which such corporation has||
 |---|---|
+|been included.||
+|(B) Election not filed.|In the event no election is filed in accordance with the|
 |provisions of paragraph (d)(2)(iv) of this section, then the Internal Revenue Service||
 |will determine the group in which such corporation is to be included. Such||
 |determination will be binding for all subsequent years unless the corporation files a||
@@ -101,45 +103,32 @@ section and paragraph
 (c)(4), and (c)(5) of this section, and paragraph
 (c)(2) of §1.382-8T
 
+||§1.382-8(g), Example|
+|---|---|
+||The first sentence of §1.382-8(g), Example §1.382-8(g), Example §1.382-8(g), Example|
+
+(2)(c)
+(2)(e)
+(3)(b)
+(3)(c)(1)(B)
+
+||The second sentence of|
+|---|---|
+||§1.382-8(g), Example The second sentence of §1.382-8(g), Example The first sentence of §1.1502-32(b)(4)(v)(A) The first sentence of §1.1502-32(b)(4)(v)(B)|
+
+(4)(c)
+(5)(c)
+
 |The fifth sentence of|paragraph (c) of this|paragraphs (c)(1), (c)(3),|
 |---|---|---|
 |§1.382-8(f)|section|(c)(4), and (c)(5) of this section, and paragraph (c)(2) of §1.382-8T|
 |§1.382-8(g), Example|paragraph (c) of this|paragraphs (c)(1), (c)(3), section, and paragraph (c)(2) of §1.382-8T|
 |The second sentence of|paragraph (c) of this|paragraphs (c)(1), (c)(3),|
-|§1.382-8(g), Example|section|(c)(4), and (c)(5) of this|
+|§1.382-8(g), Example|section paragraph (c)(2) of this section paragraph (c)(2) of this section paragraph (c)(2) of this section paragraphs (c)(1) and (2) of this section paragraph (c)(2) of this section paragraph (c)(2) of this section paragraph (b)(4)(iv) of this section paragraph (b)(4)(iv) of this section|(c)(4), and (c)(5) of this (c)(2) of §1.382-8T paragraph (c)(2) of §1.382-8T paragraph (c)(2) of §1.382-8T paragraph (c)(2) of §1.382-8T paragraph (c)(1) of this section and paragraph (c)(2) of §1.382-8T paragraph (c)(2) of §1.382-8T paragraph (c)(2) of §1.382-8T paragraph (b)(4)(iv) of §1.1502-32T paragraph (b)(4)(iv) of §1.1502-32T|
 
 (1)(b)(2) section (c)(4), and (c)(5) of this
 (1)(c) section, and paragraph
 
-(c)(2) of §1.382-8T
-
-|§1.382-8(g), Example|paragraph (c)(2) of this|paragraph (c)(2) of|
-|---|---|---|
-|The first sentence of|paragraph (c)(2) of this|paragraph (c)(2) of|
-|§1.382-8(g), Example|section|§1.382-8T|
-|§1.382-8(g), Example|paragraph (c)(2) of this|paragraph (c)(2) of|
-|§1.382-8(g), Example|paragraphs (c)(1) and (2)|paragraph (c)(1) of this|
-
-(2)(c) section §1.382-8T
-(2)(e)
-(3)(b) section §1.382-8T
-(3)(c)(1)(B) of this section section and paragraph
-
-(c)(2) of §1.382-8T
-
-|The second sentence of|paragraph (c)(2) of this|paragraph (c)(2) of|
-|---|---|---|
-|§1.382-8(g), Example|section|§1.382-8T|
-|The second sentence of|paragraph (c)(2) of this|paragraph (c)(2) of|
-|§1.382-8(g), Example|section|§1.382-8T|
-|The first sentence of|paragraph (b)(4)(iv) of|paragraph (b)(4)(iv) of|
-|§1.1502-32(b)(4)(v)(A)|this section|§1.1502-32T|
-|The first sentence of|paragraph (b)(4)(iv) of|paragraph (b)(4)(iv) of|
-|§1.1502-32(b)(4)(v)(B)|this section|§1.1502-32T|
-
-(4)(c)
-(5)(c)
-
 |§1.1502-35(c)(4)(ii)(B)|§1.1502-76(b)(2)(ii)(D)|§1.1502-76T(b)(2)(ii)(D)|
 |---|---|---|
 |§1.1502-76(b)(2)(ii)(A)(2)|paragraph (b)(2)(ii)(D) of this section|paragraph (b)(2)(ii)(D) of §1.1502-76T|
@@ -168,14 +157,33 @@ section and paragraph
 |§1.6043-2(a)|or 1.1081-11|3T(a), or §1.1081-11T|
 |The first sentence of §301.6011-5T(a) (twice)|§1.6012-2|paragraphs (a), (b) and (d) through (j) of §1.6012- 2, and paragraph (c) of §1.6012-2T|
 
-|||PART 602--OMB CONTROL NUMBERS UNDER THE PAPERWORK||
-|---|---|---|---|
-||REDUCTION ACT Authority: 26 U.S.C. 7805. 1. The following entries to the table are removed: §602.101 OMB Control numbers.|Par. 54. The authority citation for part 602 continues to read as follows: Par. 55. In §602.101, paragraph (b) is amended to read as follows:||
-|* * * * *|(b) * * * CFR part or section where identified or described||Current OMB control No.|
-|* * * * *|1.332-6………………………………………………………………….|1.382-11……………………………………………………………….. 1545-2019 1.351-3…………………………………………………………………. 1545-2019 1.355-5…………………………………………………………………. 1545-2019 1.368-3…………………………………………………………………. 1545-2019 1.1081-11………………………………………………………………. 1545-2019|1545-2019|
-|* * * * *|§602.101 OMB Control numbers.|______________________________________________________________ 2. The following entries are added in numerical order to the table:||
-|* * * * *|(b) * * * CFR part or section where identified or described||Current OMB control No.|
-|* * * * *|1.302-2T………………………………………………………………… 1545 1.302-4T………………………………………………………………… 1545||-2019 -2019|
+PART 602--OMB CONTROL NUMBERS UNDER THE PAPERWORK REDUCTION ACT Par. 54. The authority citation for part 602 continues to read as follows: Authority: 26 U.S.C. 7805. Par. 55. In §602.101, paragraph (b) is amended to read as follows:
+
+1. The following entries to the table are removed:
+§602.101 OMB Control numbers.
+
+* * * * *
+(b) * * *
+CFR part or section where Current OMB identified or described control No.
+
+* * * * *
+1.332-6…………………………………………………………………. 1545-2019
+1.382-11……………………………………………………………….. 1545-2019
+1.351-3…………………………………………………………………. 1545-2019
+1.355-5…………………………………………………………………. 1545-2019
+1.368-3…………………………………………………………………. 1545-2019
+1.1081-11………………………………………………………………. 1545-2019
+* * * * * **______________________________________________________________**
+2. The following entries are added in numerical order to the table:
+§602.101 OMB Control numbers.
+
+* * * * *
+(b) * * *
+CFR part or section where Current OMB identified or described control No.
+
+* * * * *
+1.302-2T………………………………………………………………… 1545-2019
+1.302-4T………………………………………………………………… 1545-2019
 
 |1.331-1T………………………………………………………………… 1545|-2019|
 |---|---|
@@ -193,4 +201,3 @@ section and paragraph
 Deputy Commissioner for Services and Enforcement.
 
 Approved: May 19, 2006 Eric Solomon Acting Deputy Assistant Secretary of the Treasury (Tax Policy).
-

tests/test_python.py

@@ -270,6 +270,78 @@ class TestExtractTextInRegions:
             )
 
 
+# ---------------------------------------------------------------------------
+# extract_pages_markdown / extract_pages_markdown_bytes
+# ---------------------------------------------------------------------------
+
+
+class TestExtractPagesMarkdown:
+    def test_default_returns_all_pages(self):
+        result = pdf_inspector.extract_pages_markdown(
+            fixture_path("thermo-freon12.pdf")
+        )
+        assert len(result.pages) == 3
+        assert [p.page for p in result.pages] == [0, 1, 2]
+        assert all(isinstance(p.markdown, str) for p in result.pages)
+
+    def test_bytes_default_returns_all_pages(self):
+        data = fixture_bytes("thermo-freon12.pdf")
+        result = pdf_inspector.extract_pages_markdown_bytes(data)
+        assert len(result.pages) == 3
+
+    def test_selected_pages_preserve_order(self):
+        result = pdf_inspector.extract_pages_markdown(
+            fixture_path("thermo-freon12.pdf"), pages=[2, 0]
+        )
+        assert [p.page for p in result.pages] == [2, 0]
+
+    def test_bytes_selected_pages_preserve_order(self):
+        data = fixture_bytes("thermo-freon12.pdf")
+        result = pdf_inspector.extract_pages_markdown_bytes(data, pages=[1])
+        assert len(result.pages) == 1
+        assert result.pages[0].page == 1
+
+    def test_page_fields(self):
+        result = pdf_inspector.extract_pages_markdown(
+            fixture_path("thermo-freon12.pdf"), pages=[0]
+        )
+        page = result.pages[0]
+        assert isinstance(page.page, int)
+        assert isinstance(page.markdown, str)
+        assert isinstance(page.needs_ocr, bool)
+        assert not page.needs_ocr  # text-based fixture
+        assert len(page.markdown) > 0
+
+    def test_result_fields(self):
+        result = pdf_inspector.extract_pages_markdown(
+            fixture_path("thermo-freon12.pdf")
+        )
+        assert isinstance(result.pages, list)
+        assert isinstance(result.pages_with_tables, list)
+        assert isinstance(result.pages_with_columns, list)
+        assert isinstance(result.pages_needing_ocr, list)
+        assert isinstance(result.is_complex, bool)
+
+    def test_out_of_range_page_marks_needs_ocr(self):
+        result = pdf_inspector.extract_pages_markdown(
+            fixture_path("thermo-freon12.pdf"), pages=[9999]
+        )
+        assert len(result.pages) == 1
+        assert result.pages[0].needs_ocr
+        assert result.pages[0].markdown == ""
+
+    def test_repr(self):
+        result = pdf_inspector.extract_pages_markdown(
+            fixture_path("thermo-freon12.pdf"), pages=[0]
+        )
+        assert "PagesExtractionResult" in repr(result)
+        assert "PageMarkdown" in repr(result.pages[0])
+
+    def test_not_a_pdf(self):
+        with pytest.raises(ValueError):
+            pdf_inspector.extract_pages_markdown_bytes(b"not a pdf")
+
+
 # ---------------------------------------------------------------------------
 # Error handling
 # ---------------------------------------------------------------------------