mu-ref/pdf-inspector
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(extractor): descriptor/embedded-font style flags + geometric strikeout detection (#125)

Browse Source 
* feat(extractor): descriptor/embedded-font style flags + geometric strikeout detection

Two style-recall gaps, both invisible to the existing name-based
heuristics:

1. Subset fonts with opaque BaseFont names ("Tc1", "AAAAAB+Amplitude")
   defeat is_italic_font/is_bold_font. New descriptor_style_flags reads
   the FontDescriptor (ItalicAngle beyond 4 degrees, Flags bit 7 Italic,
   bit 19 ForceBold) and, when the descriptor claims upright, falls back
   to the embedded font file: ttf-parser's OS/2 fsSelection + post
   italicAngle for sfnt fonts, and the CFF Name INDEX PostScript name
   for bare-CFF FontFile3 (descriptor rewritten to ItalicAngle 0 while
   embedding "Amplitude-LightItalic" was observed in the wild).
   ORed into is_bold/is_italic at item creation (content streams and
   form XObjects).

2. No strikeout signal existed. New is_strikeout on TextItem, detected
   in the same pass as underline: same rules pipeline (stroked lines /
   thin filled rects, table-ruling suppression), different vertical
   window — a rule crossing the glyphs at 12-55% of the em above the
   baseline instead of sitting at it. Exposed through napi and python
   bindings and pdf2md --items-json.

Verified on public ParseBench corpus docs: previously-missed italic
council titles and bold CJK itinerary headings now flagged (render-
checked); 24/508 docs gain flags, none lose any; 35 strikeout items
detected corpus-wide, disjoint from underline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB

* fix(review): quote-op advance width, Ts text rise, doc-level font style cache (PR #125 review)

Address three valid findings from review:

- The ' (move-to-next-line-and-show-text) operator emitted zero-width
  items and never advanced the text matrix, so geometric underline/
  strikeout detection (which requires width > 0) could never mark its
  text, and following show ops overlapped it. Reuse Tj's advance-width
  computation and matrix advance.

- Ts (text rise) was dropped entirely: raised/lowered runs kept the
  unshifted baseline, so rules drawn at the risen glyph position missed
  the strike/underline windows. Track rise in the text state (saved and
  restored with q/Q) and shift the rendering position through the text
  matrix's y column; advances stay on the unshifted matrix per spec.

- descriptor_style_flags re-decompressed and re-parsed the same embedded
  font program on every page whenever the descriptor left a style flag
  unset (the common case). Add a document-scoped FontStyleCache keyed by
  the FontFile2/FontFile3 object id, threaded through page and form
  extraction alongside the existing CMapDecisionCache.

The fourth finding (Form XObject rules never reach geometric detection)
is real but pre-existing for underline and needs the form walker to grow
path/paint tracking plus a new return type; deferred as a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB

* fix(review): ActualText items render at their glyphs' text rise (PR #125 review)

The EMC-built ActualText item used the captured text matrix without the
rise adjustment the ordinary Tj/TJ/' emission sites apply, so a tagged
run shown with Ts landed on the unshifted baseline — off the strikeout/
underline windows and inconsistent with untagged runs. The rise is
captured together with the first-glyph matrix (and at BDC for the
entry-position fallback): the item must render at the rise of its
GLYPHS, not whatever rise is set by EMC time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB

* fix(review): capture ActualText glyph position after the quote op's line move (PR #125 review)

The `'` handler skipped the entire suppressed-extraction block, so a
tagged span whose show op is `'` never captured its glyph matrix/rise —
the EMC item fell back to the BDC-entry matrix, which sits on the
PREVIOUS line (the `'` line move happens after BDC) with no rise. The
capture now happens right after the line move, matching the Tj/TJ
paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB

* fix(review): style-boundary gate on subscript merge + strikeout suppression coverage (PR #125 review)

merge_subscript_items absorbed a script digit into its parent
regardless of underline/strikeout flags — dropping the digit's own mark
or widening the parent's over it. The merged item carries one flag, so
differing marks now break the merge, mirroring merge_text_items'
style-boundary rule (pre-existing for underline as well).

Also extends the table-suppression test to assert is_strikeout is
cleared alongside is_underline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L3U6BKYCS73DVA83odAfYB

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Abimael Martell
2026-07-10 17:30:47 -07:00
committed by GitHub
co-authored by Claude Fable 5
parent 15bc7894a4
commit 57335f8bcf
25 changed files with 814 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/extractor/mod.rs
+71 -4
View File
@@ -24,6 +24,7 @@ use links::{extract_form_fields, extract_page_links};
// Re-export public types so existing `crate::extractor::X` paths keep working.
pub use crate::text_utils::{is_bold_font, is_italic_font};
pub use crate::types::{ItemType, TextLine};
pub(crate) use fonts::FontStyleCache;
pub(crate) use layout::detect_columns;
pub use layout::group_into_lines;
pub(crate) use layout::group_into_lines_with_thresholds;
@@ -161,6 +162,9 @@ fn extract_positioned_text_impl(
let mut all_lines = Vec::new();
let mut page_thresholds: PageThresholds = HashMap::new();
let mut gid_encoded_pages: HashSet<u32> = HashSet::new();
// Embedded-font style flags are document-scoped: the same font program
// is shared across pages, so parse it once, not once per page.
let mut style_cache = FontStyleCache::new();
// Build page ObjectId → page number map for form field extraction
let page_id_to_num: HashMap<ObjectId, u32> =
@@ -172,8 +176,14 @@ fn extract_positioned_text_impl(
continue;
}
}
let ((mut items, rects, lines), has_gid_fonts, _coords_rotated) =
extract_page_text_items(doc, page_id, *page_num, font_cmaps, include_invisible)?;
let ((mut items, rects, lines), has_gid_fonts, _coords_rotated) = extract_page_text_items(
doc,
page_id,
*page_num,
font_cmaps,
include_invisible,
&mut style_cache,
)?;
if has_gid_fonts {
gid_encoded_pages.insert(*page_num);
}
@@ -234,7 +244,10 @@ fn suppress_table_underlines(
lines: &[PdfLine],
page: u32,
) {
if !items.iter().any(|item| item.is_underline) {
if !items
.iter()
.any(|item| item.is_underline || item.is_strikeout)
{
return;
}
@@ -256,6 +269,7 @@ fn suppress_table_underlines(
for index in table_item_indices {
if let Some(item) = items.get_mut(index) {
item.is_underline = false;
item.is_strikeout = false;
}
}
}
@@ -576,6 +590,7 @@ pub(crate) fn merge_text_items(items: Vec<TextItem>) -> Vec<TextItem> {
if next.is_bold != first.is_bold
|| next.is_italic != first.is_italic
|| next.is_underline != first.is_underline
|| next.is_strikeout != first.is_strikeout
{
break;
}
@@ -638,6 +653,7 @@ pub(crate) fn merge_text_items(items: Vec<TextItem>) -> Vec<TextItem> {
is_bold: first.is_bold,
is_italic: first.is_italic,
is_underline: first.is_underline,
is_strikeout: first.is_strikeout,
item_type: first.item_type.clone(),
mcid: first.mcid,
});
@@ -715,7 +731,9 @@ pub(crate) fn merge_subscript_items(items: Vec<TextItem>) -> Vec<TextItem> {
.chars()
.last()
.is_some_and(|c| c.is_alphabetic());
if parent.font_size >= sub_threshold && ends_with_letter {
let same_marks = parent.is_underline == item.is_underline
&& parent.is_strikeout == item.is_strikeout;
if parent.font_size >= sub_threshold && ends_with_letter && same_marks {
let parent_right = parent.x + parent.width;
let gap = item.x - parent_right;
// Subscripts must be tightly adjacent (within ~1pt)
@@ -789,6 +807,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}
@@ -991,6 +1010,7 @@ mod tests {
items[3].y = 470.0;
for item in &mut items {
item.is_underline = true;
item.is_strikeout = true;
}
let lines = vec![
make_line(100.0, 500.0, 300.0, 500.0),
@@ -1004,6 +1024,30 @@ mod tests {
suppress_table_underlines(&mut items, &[], &lines, 1);
assert!(items.iter().all(|item| !item.is_underline));
assert!(items.iter().all(|item| !item.is_strikeout));
}
#[test]
fn subscript_digit_with_different_marks_is_not_absorbed() {
// A struck-out word followed by an unmarked footnote digit: merging
// would widen the parent's strikeout claim over the digit (and the
// reverse would drop the digit's own mark). Style boundaries break
// the merge, as in merge_text_items.
let mut word = make_merge_item("word", 100.0, 24.0);
word.font_size = 10.0;
word.is_strikeout = true;
let mut digit = make_merge_item("2", 124.5, 4.0);
digit.font_size = 6.0;
digit.y = word.y + 3.0;
let merged = merge_subscript_items(vec![word.clone(), digit.clone()]);
assert_eq!(merged.len(), 2);
// Same marks still merge (footnote ref inside the strike).
digit.is_strikeout = true;
let merged = merge_subscript_items(vec![word, digit]);
assert_eq!(merged.len(), 1);
assert!(merged[0].text.starts_with("word"));
}
#[test]
@@ -1021,6 +1065,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1036,6 +1081,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1051,6 +1097,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1106,6 +1153,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1121,6 +1169,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1136,6 +1185,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1162,6 +1212,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1177,6 +1228,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1192,6 +1244,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1220,6 +1273,7 @@ mod tests {
is_bold: true,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}
@@ -1255,6 +1309,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}
@@ -1291,6 +1346,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1306,6 +1362,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1321,6 +1378,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1344,6 +1402,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}
@@ -1457,6 +1516,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1472,6 +1532,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1497,6 +1558,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1512,6 +1574,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
},
@@ -1553,6 +1616,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}],
@@ -1598,6 +1662,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}],
@@ -1643,6 +1708,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}],
@@ -1681,6 +1747,7 @@ mod tests {
is_bold: false,
is_italic: false,
is_underline: false,
is_strikeout: false,
item_type: ItemType::Text,
mcid: None,
}