Bump version from 1.8.15 to 1.9.0

Background. ItemType::Image, MarkdownOptions::include_images, and the
markdown emitter's image-collection path have all been in the tree
for a while, but no producer ever populated them — content_stream.rs
explicitly `// Skip images — text extraction only` at the Do
operator, and the nested Form-XObject walker in xobjects.rs only
matched XObjectType::Form, silently dropping Image entries. The
declared types were dead code.

This PR lights them up. At every Do that resolves to an Image
XObject (both top-level and nested inside Form XObjects), we now
compute the page-space bbox from the current CTM via a new
`image_bbox_from_ctm` helper — handling both axis-aligned and
rotated/sheared placements via 4-corner AABB — and emit a TextItem
with `item_type: ItemType::Image` and the legacy `[Image: <name>]`
text payload that the markdown emitter already knows how to render.

Callers can now find raster figures via `extract_text_with_positions`
(and the `_mem` variant, newly re-exported at the crate root) without
needing to re-parse the PDF or run a vision/layout model. The intended
consumer is layout-aware text pipelines that want to crop figures and
caption them out-of-band.

Two backstops to avoid silent breakage for existing callers:

  1. `MarkdownOptions::include_images` default flipped `true → false`.
     If it stayed at `true`, every existing user of
     `extract_pages_markdown` would suddenly see `![Image: Im0](image)`
     placeholders inserted throughout their output the moment they
     upgraded. Image data is still available structurally via
     `extract_text_with_positions`; rendering it into markdown is now
     an opt-in. New regression test asserts `extract_pages_markdown`
     output is unchanged for the image-bearing fixture.

  2. Image items now also skip the layout heuristics
     (`detect_columns`, `detect_tables_from_rects`) via a new
     `is_text_layout_item` predicate. Without this filter, an image's
     left edge would land in the column-projection profile and skew
     table column detection — surfaced by
     `vector_grid_tests::upstage_key_functions_four_cols` going from 4
     detected columns to 5 in CI before the filter was added.

Re-exporting `extract_text_with_positions_mem` at the crate root —
strictly additive; mirrors how `extract_pages_markdown_mem` is already
available there.

Tests:

  - test_extract_text_with_positions_emits_image_bboxes — minimal PDF
    with one 200×100 image at (50, 600); asserts one Image item with
    correct bbox + page + text.
  - test_image_xobject_bbox_handles_rotated_ctm — 90° rotated image
    via shear-component CTM; asserts AABB is correct (handles non-
    axis-aligned placements via 4-corner clamp).
  - test_image_emission_does_not_change_default_markdown — asserts no
    `Image:` token leaks into default markdown output, regression
    guard for the include_images flip.
  - test_markdown_options_default_has_include_images_false — explicit
    sentinel so anyone flipping it back catches it in CI.

napi/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@firecrawl/pdf-inspector",
-  "version": "1.9.4",
+  "version": "1.9.0",
   "description": "Fast PDF classification and text extraction. Detect text-based vs scanned PDFs, extract text by region with quality checks. Native Rust performance via napi-rs.",
   "main": "index.js",
   "types": "index.d.ts",

src/lib.rs

@@ -823,10 +823,7 @@ pub fn extract_tables_in_regions_mem(
                     // symmetrically low under font-decode failure — this
                     // guard breaks that symmetry by comparing against
                     // bbox area, which is independent of extraction.
-                    if source != TableCandidateSource::KeyValue
-                        && region_text_density_too_low(region_text_chars, region_area)
-                        && !markdown_table_body_is_dense(&md)
-                    {
+                    if region_text_density_too_low(region_text_chars, region_area) {
                         return None;
                     }
                     let shape = markdown_table_shape(&md);
@@ -837,11 +834,7 @@ pub fn extract_tables_in_regions_mem(
                         Some(TableCandidateIssue::LineRowUndercount)
                     } else if wide_table_sparse_prefix_undercount(&md) {
                         Some(TableCandidateIssue::SparseWideUndercount)
-                    } else if !matches!(
-                        source,
-                        TableCandidateSource::Line | TableCandidateSource::KeyValue
-                    ) && text_cluster_column_undercount(&matched, shape)
-                    {
+                    } else if text_cluster_column_undercount(&matched, shape) {
                         Some(TableCandidateIssue::TextColumnUndercount)
                     } else if prose_grid_fragment_needs_ocr(&md) {
                         Some(TableCandidateIssue::ProseGridFragment)
@@ -884,16 +877,6 @@ pub fn extract_tables_in_regions_mem(
             {
                 candidates.push(candidate);
             }
-            if let Some(table) = tables::try_build_table_from_columns(&matched, page_1idx) {
-                if let Some(candidate) = evaluate(TableCandidateSource::Column, &table) {
-                    candidates.push(candidate);
-                }
-            }
-            if let Some(table) = tables::try_build_key_value_table_from_rows(&matched, page_1idx) {
-                if let Some(candidate) = evaluate(TableCandidateSource::KeyValue, &table) {
-                    candidates.push(candidate);
-                }
-            }
 
             match select_table_candidate(&candidates) {
                 Some(candidate) => page_results.push(RegionText {
@@ -3765,8 +3748,6 @@ enum TableCandidateSource {
     Rect,
     Line,
     Heuristic,
-    Column,
-    KeyValue,
 }
 
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -3801,12 +3782,8 @@ fn select_table_candidate(candidates: &[TableCandidate]) -> Option<&TableCandida
     // serving a tidy-looking fragment.
     if first.issue == Some(TableCandidateIssue::LineRowUndercount) {
         return candidates.iter().find(|candidate| {
-            matches!(
-                candidate.source,
-                TableCandidateSource::Heuristic
-                    | TableCandidateSource::Column
-                    | TableCandidateSource::KeyValue
-            ) && candidate.issue.is_none()
+            candidate.source == TableCandidateSource::Heuristic
+                && candidate.issue.is_none()
                 && candidate.shape.cols * 10 >= first.shape.cols * 13
         });
     }
@@ -3824,31 +3801,14 @@ fn select_table_candidate(candidates: &[TableCandidate]) -> Option<&TableCandida
         TableCandidateSource::Rect | TableCandidateSource::Line
     ) {
         if let Some(heuristic) = candidates.iter().find(|candidate| {
-            matches!(
-                candidate.source,
-                TableCandidateSource::Heuristic
-                    | TableCandidateSource::Column
-                    | TableCandidateSource::KeyValue
-            ) && candidate.issue.is_none()
+            candidate.source == TableCandidateSource::Heuristic
+                && candidate.issue.is_none()
                 && heuristic_substantially_better(candidate.shape, accepted.shape)
         }) {
             accepted = heuristic;
         }
     }
 
-    if accepted.source == TableCandidateSource::Heuristic {
-        if let Some(layout_candidate) = candidates.iter().find(|candidate| {
-            matches!(
-                candidate.source,
-                TableCandidateSource::Column | TableCandidateSource::KeyValue
-            ) && candidate.issue.is_none()
-                && candidate.shape.cols >= accepted.shape.cols
-                && candidate.shape.rows > accepted.shape.rows
-        }) {
-            accepted = layout_candidate;
-        }
-    }
-
     Some(accepted)
 }
 
@@ -4293,21 +4253,12 @@ fn looks_like_partial_table_ex(markdown: &str, layout_assisted: bool) -> bool {
     }
 
     // Failure mode 2: header has empty cells in a multi-column table.
-    // When layout-assisted, tolerate merged/spanning header gaps if the
-    // body is dense. Region bboxes from a layout model often start at a
-    // visual table whose header cannot be represented faithfully in a
-    // flat pipe table, while the body rows are still complete enough to use.
-    let header_empty_indices: Vec<usize> = header_cells
-        .iter()
-        .enumerate()
-        .filter_map(|(idx, cell)| cell.is_empty().then_some(idx))
-        .collect();
-    let empty_count = header_empty_indices.len();
+    // When layout-assisted, allow up to 1 empty header cell (common in
+    // tables with merged/spanning header cells that we can't represent).
+    let empty_count = header_cells.iter().filter(|c| c.is_empty()).count();
     if layout_assisted {
-        if n_cols >= 3
-            && empty_count >= 2
-            && !layout_assisted_empty_header_has_dense_body(markdown, n_cols)
-        {
+        // Reject only if >1 empty header cell (2+ means serious boundary issue)
+        if n_cols >= 3 && empty_count >= 2 {
             return true;
         }
     } else if n_cols >= 3 && empty_count >= 1 {
@@ -4345,16 +4296,7 @@ fn looks_like_partial_table_ex(markdown: &str, layout_assisted: bool) -> bool {
             // (totals, subtotals) are common.
             let threshold = if layout_assisted { 2 } else { 3 };
             if n_cols >= 3 && empty_data * threshold >= n_cols {
-                let sparse_row_shares_header_spacer = layout_assisted
-                    && data_inner.iter().enumerate().any(|(idx, cell)| {
-                        cell.trim().is_empty() && header_empty_indices.contains(&idx)
-                    })
-                    && layout_assisted_empty_header_has_dense_body(markdown, n_cols);
-                let sparse_row_is_section_label = layout_assisted
-                    && layout_assisted_sparse_section_row_is_ok(data_inner, markdown, n_cols);
-                if !sparse_row_shares_header_spacer && !sparse_row_is_section_label {
-                    return true;
-                }
+                return true;
             }
         }
     }
@@ -4412,88 +4354,6 @@ fn looks_like_partial_table_ex(markdown: &str, layout_assisted: bool) -> bool {
     false
 }
 
-fn layout_assisted_empty_header_has_dense_body(markdown: &str, n_cols: usize) -> bool {
-    let rows = markdown_pipe_rows(markdown);
-    let data_rows: Vec<&Vec<&str>> = rows
-        .iter()
-        .skip(1)
-        .filter(|row| row.iter().any(|cell| !cell.trim().is_empty()))
-        .collect();
-    if data_rows.len() < 2 || n_cols < 3 {
-        return false;
-    }
-
-    let total_cells = data_rows.len() * n_cols;
-    let mut filled_cells = 0usize;
-    let mut rows_with_multiple_cells = 0usize;
-    let mut max_filled_in_row = 0usize;
-    for row in &data_rows {
-        let filled = row.iter().filter(|cell| !cell.trim().is_empty()).count();
-        filled_cells += filled;
-        max_filled_in_row = max_filled_in_row.max(filled);
-        if filled >= 2 {
-            rows_with_multiple_cells += 1;
-        }
-    }
-
-    // Dense enough to be a useful extraction despite lossy merged headers.
-    // The row-count gate avoids accepting a single tidy row under a broken
-    // header, and the density gate keeps sparse fragments on the OCR path.
-    rows_with_multiple_cells * 2 >= data_rows.len()
-        && max_filled_in_row >= n_cols.min(3)
-        && filled_cells * 100 >= total_cells * 45
-}
-
-fn layout_assisted_sparse_section_row_is_ok(row: &[&str], markdown: &str, n_cols: usize) -> bool {
-    let labels: Vec<&str> = row
-        .iter()
-        .map(|cell| cell.trim())
-        .filter(|cell| !cell.is_empty())
-        .collect();
-    if labels.len() != 1 {
-        return false;
-    }
-    let label = labels[0];
-    if label.len() > 40 || !label.chars().any(|ch| ch.is_alphabetic()) {
-        return false;
-    }
-    if label.ends_with('.') || label.ends_with('!') || label.ends_with('?') || label.ends_with(':')
-    {
-        return false;
-    }
-    layout_assisted_empty_header_has_dense_body(markdown, n_cols)
-}
-
-fn markdown_table_body_is_dense(markdown: &str) -> bool {
-    let rows = markdown_pipe_rows(markdown);
-    let data_rows: Vec<&Vec<&str>> = rows
-        .iter()
-        .skip(1)
-        .filter(|row| row.iter().any(|cell| !cell.trim().is_empty()))
-        .collect();
-    if data_rows.len() < 3 {
-        return false;
-    }
-
-    let cols = rows.iter().map(|row| row.len()).max().unwrap_or_default();
-    if cols < 3 {
-        return false;
-    }
-
-    let mut filled_cells = 0usize;
-    let mut rows_with_multiple_cells = 0usize;
-    for row in &data_rows {
-        let filled = row.iter().filter(|cell| !cell.trim().is_empty()).count();
-        filled_cells += filled;
-        if filled >= cols.min(3) {
-            rows_with_multiple_cells += 1;
-        }
-    }
-
-    let total_cells = data_rows.len() * cols;
-    rows_with_multiple_cells * 2 >= data_rows.len() && filled_cells * 100 >= total_cells * 45
-}
-
 /// Original strict validation (no layout assistance). Used by tests and
 /// full-page extraction paths that don't have layout model assistance.
 #[cfg(test)]
@@ -4845,16 +4705,6 @@ mod table_candidate_selection_tests {
         assert_eq!(selected.source, TableCandidateSource::Heuristic);
     }
 
-    #[test]
-    fn prefers_clean_column_fallback_when_it_recovers_more_rows() {
-        let candidates = vec![
-            candidate(TableCandidateSource::Heuristic, 6, 5, None),
-            candidate(TableCandidateSource::Column, 7, 5, None),
-        ];
-        let selected = select_table_candidate(&candidates).unwrap();
-        assert_eq!(selected.source, TableCandidateSource::Column);
-    }
-
     #[test]
     fn line_candidate_collapsing_captured_y_clusters_is_suspicious() {
         let long = "value value value value value value value value value value value value";
@@ -4959,9 +4809,7 @@ mod table_candidate_selection_tests {
 
 #[cfg(test)]
 mod looks_like_partial_table_tests {
-    use super::{
-        looks_like_partial_table, looks_like_partial_table_ex, markdown_table_body_is_dense,
-    };
+    use super::{looks_like_partial_table, looks_like_partial_table_ex};
 
     #[test]
     fn good_table_passes() {
@@ -5096,29 +4944,11 @@ mod looks_like_partial_table_tests {
 
     #[test]
     fn two_empty_headers_still_rejected_when_layout_assisted() {
-        // A single tidy row is not enough evidence to trust a badly gapped header.
+        // 2+ empty headers is still bad even with layout assistance.
         let md = "|A|||D|\n|---|---|---|---|\n|x|y|z|w|";
         assert!(
             looks_like_partial_table_ex(md, true),
-            "2 empty headers with only one body row are rejected even layout-assisted"
-        );
-    }
-
-    #[test]
-    fn dense_body_with_empty_merged_header_passes_when_layout_assisted() {
-        let md = "|Year||Unadjusted Basis|||\n\
-                  |---|---|---|---|---|\n\
-                  |1|.1667|$100,000|$16,670|$16,670|\n\
-                  |2|.3333|$100,000|$33,330|$50,000|\n\
-                  |3|.3333|$100,000|$33,330|$88,330|\n\
-                  |4|.1667|$100,000|$16,670|$100,000|";
-        assert!(
-            looks_like_partial_table(md),
-            "strict mode still rejects merged-header gaps"
-        );
-        assert!(
-            !looks_like_partial_table_ex(md, true),
-            "layout-assisted should trust a dense body under a merged header"
+            "2 empty headers rejected even layout-assisted"
         );
     }
 
@@ -5143,40 +4973,6 @@ mod looks_like_partial_table_tests {
         );
     }
 
-    #[test]
-    fn sparse_first_row_with_header_spacer_passes_when_layout_assisted() {
-        let md = "|Properties|Instruction||Training Datasets Alignment|\n\
-                  |---|---|---|---|\n\
-                  ||Alpaca-GPT4 OpenOrca Synth. Math-Instruct||Orca DPO Pairs Ultrafeedback Cleaned|\n\
-                  |Total # Samples|52K 2.91M 126K||12.9K 60.8K 126K|";
-        assert!(
-            looks_like_partial_table(md),
-            "strict mode rejects the sparse first row"
-        );
-        assert!(
-            !looks_like_partial_table_ex(md, true),
-            "layout-assisted should allow sparse rows that share a header spacer column"
-        );
-    }
-
-    #[test]
-    fn sparse_section_row_passes_when_layout_assisted_body_is_dense() {
-        let md = "|Properties|Conditions|Method|Typical values|Units|\n\
-                  |---|---|---|---|---|\n\
-                  |Rheology|||||\n\
-                  |Melt Flow Rate|230 C/2.16 kg|ASTM D1238|3.0|g/10 min|\n\
-                  |Tensile Stress at Yield|50 mm/min|ASTM D638|31|MPa|\n\
-                  |Elongation at Yield|50 mm/min|ASTM D638|8|%|";
-        assert!(
-            looks_like_partial_table(md),
-            "strict mode rejects the sparse first row"
-        );
-        assert!(
-            !looks_like_partial_table_ex(md, true),
-            "layout-assisted should allow a short section label above dense table rows"
-        );
-    }
-
     #[test]
     fn paragraph_still_rejected_when_layout_assisted() {
         // Paragraph detection is not relaxed — it's a genuine extraction issue.
@@ -5230,27 +5026,6 @@ mod looks_like_partial_table_tests {
             "duplicate headers rejected even layout-assisted"
         );
     }
-
-    #[test]
-    fn dense_numeric_table_body_is_structurally_trusted() {
-        let md = "|Year|3-Year|5-Year|7-Year|\n\
-                  |---|---|---|---|\n\
-                  |1|33.0%|20.00%|14.29%|\n\
-                  |2|44.45%|32.00%|24.49%|\n\
-                  |3|14.81%|19.20%|17.49%|\n\
-                  |4|7.41%|11.52%|12.49%|";
-        assert!(markdown_table_body_is_dense(md));
-    }
-
-    #[test]
-    fn sparse_markdown_fragment_is_not_structurally_trusted() {
-        let md = "|A|B|C|D|\n\
-                  |---|---|---|---|\n\
-                  |x||||\n\
-                  |||y||\n\
-                  ||||z|";
-        assert!(!markdown_table_body_is_dense(md));
-    }
 }
 
 /// Analyse extracted items and rects for layout complexity.

src/tables/format.rs

@@ -208,24 +208,6 @@ fn looks_like_compact_entry_label(cell: &str) -> bool {
     (1..=6).contains(&words)
 }
 
-fn looks_like_plain_section_label(cell: &str) -> bool {
-    let trimmed = cell.trim();
-    if trimmed.len() < 4 || trimmed.len() > 40 {
-        return false;
-    }
-    if trimmed.ends_with(['.', ',', ';', ':']) || trimmed.contains(|ch: char| ch.is_ascii_digit()) {
-        return false;
-    }
-    if trimmed.len() <= 4 && trimmed.chars().all(|ch| !ch.is_lowercase()) {
-        return false;
-    }
-    trimmed
-        .chars()
-        .all(|ch| ch.is_alphabetic() || ch.is_whitespace() || matches!(ch, '&' | '/' | '-'))
-        && starts_with_uppercase_alpha(trimmed)
-        && (1..=4).contains(&alpha_word_count(trimmed))
-}
-
 fn ends_like_incomplete_phrase(cell: &str) -> bool {
     let lower = cell.trim_end().to_ascii_lowercase();
     lower.ends_with(" and")
@@ -323,10 +305,6 @@ fn clean_table_cells(cells: &[Vec<String>]) -> (Vec<Vec<String>>, Vec<String>) {
             .and_then(|r| r.first())
             .map(|c| c.trim())
             .unwrap_or("");
-        let header_filled = cleaned
-            .first()
-            .map(|r| r.iter().filter(|c| !c.trim().is_empty()).count())
-            .unwrap_or(num_cols);
         let looks_like_spanning_first_column_row = first_cell.is_empty()
             && row.len() >= 4
             && non_first_cells.len() == row.len().saturating_sub(1)
@@ -350,10 +328,6 @@ fn clean_table_cells(cells: &[Vec<String>]) -> (Vec<Vec<String>>, Vec<String>) {
             && non_first_cells
                 .iter()
                 .any(|cell| looks_like_compact_entry_label(cell));
-        let looks_like_section_label_row = !first_cell.is_empty()
-            && filled_cells == 1
-            && header_filled >= 3
-            && looks_like_plain_section_label(first_cell);
         // Classic continuation: first cell empty, content in other cells
         let is_classic_continuation = first_cell.is_empty()
             && !non_first_cells.is_empty()
@@ -370,6 +344,10 @@ fn clean_table_cells(cells: &[Vec<String>]) -> (Vec<Vec<String>>, Vec<String>) {
             .last()
             .map(|r| r.iter().filter(|c| !c.trim().is_empty()).count())
             .unwrap_or(0);
+        let header_filled = cleaned
+            .first()
+            .map(|r| r.iter().filter(|c| !c.trim().is_empty()).count())
+            .unwrap_or(num_cols);
         // Merge when the row has significantly fewer filled cells than header.
         // For wide tables (5+ cols), require ≤50% of header cells.
         // For narrow tables (2-4 cols), require fewer than header cells.
@@ -391,7 +369,6 @@ fn clean_table_cells(cells: &[Vec<String>]) -> (Vec<Vec<String>>, Vec<String>) {
             && !looks_like_spanning_first_column_row
             && !looks_like_hierarchical_subrow
             && !looks_like_new_first_column_entry
-            && !looks_like_section_label_row
             && !is_short_subheader;
 
         let is_continuation = is_classic_continuation || is_wrapped_continuation;
@@ -537,44 +514,6 @@ mod tests {
         assert!(cleaned[1][1].contains("continued text here"));
     }
 
-    #[test]
-    fn test_clean_table_cells_first_column_section_label_not_merged() {
-        let cells = vec![
-            vec![
-                "Properties".into(),
-                "Conditions".into(),
-                "Method".into(),
-                "Typical values".into(),
-                "Units".into(),
-            ],
-            vec![
-                "Melt Flow Rate".into(),
-                "230 C/2.16 kg".into(),
-                "ASTM D1238".into(),
-                "3.0".into(),
-                "g/10 min".into(),
-            ],
-            vec![
-                "Mechanical".into(),
-                "".into(),
-                "".into(),
-                "".into(),
-                "".into(),
-            ],
-            vec![
-                "Tensile Stress at Yield".into(),
-                "50 mm/min".into(),
-                "ASTM D638".into(),
-                "31".into(),
-                "MPa".into(),
-            ],
-        ];
-        let (cleaned, _) = clean_table_cells(&cells);
-
-        assert_eq!(cleaned.len(), 4);
-        assert_eq!(cleaned[2][0], "Mechanical");
-    }
-
     #[test]
     fn test_clean_table_cells_short_subheader_not_merged() {
         let cells = vec![