Compare commits

...
Author SHA1 Message Date
Abimael MartellandClaude Fable 5 573bc3cc90 review: block table roles from heading promotion too
Add Table/TR/TH/TD/THead/TBody/TFoot to is_non_heading_content. When
table reconstruction falls back and cells reach the line loop as plain
text, a short isolated cell (a TH column header in particular) could be
promoted to a heading. Defensive: no change across either regression
corpus, so pure hardening for the fallback path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 22:22:12 -07:00
Abimael MartellandClaude Fable 5 8951d90958 review: extend non-heading role gate; centralize on StructRole method
Move the non-heading-role check to StructRole::is_non_heading_content
and extend it to the content roles the inline allowlist missed: Quote,
Index, Note, Reference, BibEntry, Formula, Form (in addition to the
existing list/quote/caption/toc/code roles).

Figure is deliberately excluded: cover and banner pages routinely tag
the document title inside a Figure next to a seal/logo, and that title
is a real heading — including Figure demoted the LA County protocol
cover title from headings to bold. Verified against the reference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 22:14:24 -07:00
Abimael MartellandClaude Fable 5 2349474432 fix(markdown): keep isolated headings on sparse pages; gate tagged roles
The isolated-line density guard wiped every isolated line on a page
where they exceeded 25% of lines. On sparse pages (covers, ToC pages
with a lone "CONTENTS" title, section-divider pages) a single heading
is trivially >25%, so the guard erased exactly the line it exists to
find. Require the page to have >=10 lines before the guard runs — the
25% ratio only signals a multi-column misfire on a dense page.

That let more isolated lines through, exposing that the visual heading
heuristic could promote lines already tagged with a non-heading struct
role (list item, blockquote, code, caption, ToC) or set in a monospace
font. Gate the heuristic on those in both converter paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 22:02:07 -07:00
3 changed files with 145 additions and 3 deletions
+50 -2
View File
@@ -141,8 +141,11 @@ fn find_isolated_lines(lines: &[TextLine], base_size: f32, para_threshold: f32)
}
}
for (&page, &(total, isolated)) in &page_line_counts {
if total > 0 && isolated as f32 / total as f32 > 0.25 {
// Too many isolated lines on this page — remove them all
// The ratio only means something on pages dense enough for a
// multi-column misfire; on sparse pages (covers, ToC pages with a
// lone title) one isolated line is 25%+ of the page and exactly the
// line isolation exists to find.
if total >= 10 && isolated as f32 / total as f32 > 0.25 {
set.retain(|&i| lines[i].page != page);
}
}
@@ -700,7 +703,15 @@ pub(super) fn to_markdown_from_lines_with_tables_and_images(
_ => false,
};
// Lines explicitly tagged with a non-heading content role must never
// be promoted by the visual heuristic — a tagged list item, quote, or
// code line can look exactly like a heading (short, isolated).
let non_heading_role = struct_role
.as_ref()
.is_some_and(StructRole::is_non_heading_content);
let heuristic_heading = if options.detect_headers
&& !non_heading_role
&& !is_code_line
&& !looks_like_list_continuation
&& plain_trimmed.len() > 3
&& plain_trimmed.split_whitespace().count() <= 15
@@ -1053,6 +1064,7 @@ pub fn to_markdown_from_lines(lines: Vec<TextLine>, options: MarkdownOptions) ->
&& !is_toc_entry_line(plain_trimmed)
&& !is_heading_fragment(plain_trimmed)
&& toc_suppress_page != Some(line.page)
&& !(options.detect_code && line.items.iter().any(|i| is_monospace_font(&i.font)))
{
let line_font_size = line.items.first().map(|i| i.font_size).unwrap_or(base_size);
if let Some(header_level) =
@@ -1227,6 +1239,42 @@ mod tests {
}
}
fn line_at(text: &str, page: u32, y: f32) -> TextLine {
let mut item = make_item(text, page, None);
item.y = y;
make_line(vec![item])
}
#[test]
fn isolated_lines_kept_on_sparse_pages() {
// A ToC page with a lone title and one entry far below: the density
// ratio is 50% but the page is too sparse for the multi-column
// misfire the guard targets — the title must stay isolated.
let lines = vec![
line_at("CONTENTS", 1, 700.0),
line_at("Chapter One 5", 1, 500.0),
];
let isolated = find_isolated_lines(&lines, 12.0, 20.0);
assert!(
isolated.contains(&0),
"sparse-page title must stay isolated"
);
}
#[test]
fn isolated_lines_wiped_on_dense_pages() {
// 12 short lines all with paragraph gaps — the multi-column misfire
// shape. The guard must clear them all.
let lines: Vec<TextLine> = (0..12)
.map(|i| line_at("Short column line", 1, 700.0 - i as f32 * 50.0))
.collect();
let isolated = find_isolated_lines(&lines, 12.0, 20.0);
assert!(
isolated.is_empty(),
"dense page of isolated lines must be wiped"
);
}
#[test]
fn test_struct_role_heading() {
let lines = vec![
+94
View File
@@ -76,6 +76,52 @@ pub enum StructRole {
}
impl StructRole {
/// Content roles whose text must never be promoted to a heading by the
/// visual heuristic. These carry an explicit non-heading meaning in the
/// struct tree (lists, quotes, notes, references, captions, formulas,
/// forms, ToC entries), yet their text is often short and visually
/// isolated — exactly what the heuristic keys on. Heading roles (H, H1H6)
/// and generic container/flow roles (P, Div, Sect, Span, …) are excluded
/// so the heuristic can still fire there.
///
/// `Figure` is deliberately NOT in this set: cover/banner pages routinely
/// tag the document title inside a Figure (alongside a seal or logo), and
/// that title is a real heading. `Formula` and `Form` stay — a line
/// explicitly tagged as an equation or form field is never a heading.
///
/// Table roles (Table/TR/TH/TD/THead/TBody/TFoot) are included so that
/// when table reconstruction falls back and cells reach the line loop as
/// plain text, a short isolated cell — a `TH` column header especially —
/// is not promoted to a heading.
pub(crate) fn is_non_heading_content(&self) -> bool {
matches!(
self,
Self::L
| Self::LI
| Self::Lbl
| Self::LBody
| Self::BlockQuote
| Self::Quote
| Self::Caption
| Self::TOC
| Self::TOCI
| Self::Index
| Self::Note
| Self::Reference
| Self::BibEntry
| Self::Code
| Self::Formula
| Self::Form
| Self::Table
| Self::TR
| Self::TH
| Self::TD
| Self::THead
| Self::TBody
| Self::TFoot
)
}
fn from_name(name: &str) -> Self {
match name {
"Document" => Self::Document,
@@ -856,6 +902,54 @@ fn contains_bytes(haystack: &[u8], needle: &[u8]) -> bool {
mod tests {
use super::*;
#[test]
fn non_heading_content_roles() {
for r in [
StructRole::L,
StructRole::LI,
StructRole::BlockQuote,
StructRole::Quote,
StructRole::Caption,
StructRole::TOC,
StructRole::TOCI,
StructRole::Index,
StructRole::Note,
StructRole::Reference,
StructRole::BibEntry,
StructRole::Code,
StructRole::Formula,
StructRole::Form,
StructRole::Table,
StructRole::TR,
StructRole::TH,
StructRole::TD,
StructRole::THead,
StructRole::TBody,
StructRole::TFoot,
] {
assert!(
r.is_non_heading_content(),
"{r:?} should block heading promotion"
);
}
// Heading and generic container/flow roles must NOT block promotion
for r in [
StructRole::H,
StructRole::H1,
StructRole::H3,
StructRole::P,
StructRole::Div,
StructRole::Sect,
StructRole::Span,
StructRole::Figure,
] {
assert!(
!r.is_non_heading_content(),
"{r:?} should allow heading promotion"
);
}
}
#[test]
fn test_struct_role_from_name() {
assert_eq!(StructRole::from_name("H1"), StructRole::H1);
+1 -1
View File
@@ -76,6 +76,6 @@ forms simpler, we would be happy to hear from you. You can write to the Tax Form
**Unreported Tips.—**If you received tips of $20 or more for any month while working for one employer but did not report them to your employer, you must figure and pay social security and Medicare taxes on the unreported tips when you file your tax return. If you have unreported tips, you **must** use Form 1040 and **Form 4137,** Social Security and Medicare Tax on Unreported Tip Income, to report them. You may **not** use Form 1040A or 1040EZ. Employees subject to the Railroad Retirement Tax Act **cannot** use Form 4137 to pay railroad retirement tax on unreported tips. To get railroad retirement credit, you must report tips to your employer. If you do not report tips to your employer as required, you may be charged a penalty of 50% of the social security and Medicare taxes (or railroad retirement tax) due on the unreported tips unless there was reasonable cause for not reporting them. **Additional Information.—**Get **Pub. 531,** Reporting Tip Income, and Form 4137 for more information on tips. If you are an employee of certain large food or beverage establishments, see Pub. 531 for tip allocation rules. **Recordkeeping.—**If you do not keep a daily record of tips, you must keep other reliable proof of the tip income you received. This proof includes copies of restaurant bills and credit card charges that show amounts customers added as tips. Keep your tip income records for as long as the information on them may be needed in the administration of any Internal Revenue law.
**Instructions** *(continued)*
### Instructions (continued)
Use this space to total your tips for the year