mu-ref/pdf-inspector
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
28823bd849a2481f1e73d5ddfa01f9803af044bb
pdf-inspector/tests/snapshots/td9264.md
T
Abimael MartellandClaude Opus 4.6 44092bcc9e feat(tables): borderless table detection improvements (#17) 
* feat(tables): improve heuristic detection for borderless wrapped-cell tables

Three changes to the body-font heuristic detector:

1. Adaptive Y-gap in find_table_regions_strict: use median qualifying-row
   spacing × 3 instead of fixed 25pt. Tables with wrapped cells have
   larger gaps between qualifying rows (those with 3+ X-clusters).

2. Y-only region filtering: use full X range when collecting region items.
   The strict X bounds from qualifying rows excluded continuation lines
   in wrapped cells, starving find_column_boundaries of items.

3. Merged-band retry: when split_side_by_side splits a page into bands
   but no band produces a table, retry heuristic detection on all items
   merged as a single band.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tables): gap-histogram column detection for small tables + lower avg_cells

Two changes to fix PDF 045 (borderless table with narrow "No." column):

1. Extend gap-histogram column threshold to small tables: when the gap
   between within-column jitter and between-column spacing is >10pt
   (unambiguous bimodal signal), use the detected threshold even with
   fewer than 500 items. Previously only triggered for dense tables.

2. Lower BodyFont avg_cells_per_row minimum from 2.5 to 2.0 to handle
   tables with wrapped multi-line cells where continuation lines have
   only 1 filled cell.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tables): trim empty outer columns + relax partial H-line validation

- Rect detection: trim empty first/last columns instead of rejecting
  the whole table. Rect edges often extend beyond text boundaries.
- Line detection: accept tables with 6+ partial horizontal lines
  (>15% width) when <3 full-spanning lines exist. Handles tables
  with column-level separators instead of full-width rules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tables): cell-rect fallback for tables with variable-width backgrounds

When rect clustering produces a grid that fails validation (empty
interior columns from variable-width cell backgrounds), fall through
to a new strategy: use rect Y-edges for row boundaries and text
X-position clustering for columns. This handles tables like the
opendataloader-bench 088-090 comparison tables where each cell has
its own background rect at different widths.

Also widen failed-cluster hint width cap for large clusters (≥30 rects)
to allow page-spanning table regions.

TEDS score on opendataloader-bench: 0.300 → 0.353.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tables): relax vertical line spanning validation for partial borders

Accept tables with 4+ partial vertical lines (>10% table height) when
fewer than 2 span >30%. Handles tables like opendataloader-bench 053
with column-level vertical separators that don't extend the full height.

TEDS: 0.353 → 0.377 on opendataloader-bench.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tables): lower cell-rect density threshold + add validation logging

- Lower cell-rect density minimum from 25% to 15% to accept sparser
  tables with decorative backgrounds (fixes 147).
- Add debug logging to all heuristic validation paths for diagnosability.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tables): relax body-font validations for text-only and 2-column tables

Four fixes closing 71% of the TEDS gap vs opendataloader:

1. Validation 7 (table-like content): bypass numeric content requirement
   for tables with 3+ columns that passed all structural checks. Text-only
   tables (category lists, program descriptions) are legitimate.

2. Qualifying row threshold: lower from 3+ to 2+ X-clusters per row.
   Enables 2-column body-font table detection (fixes 166).

3. Row-stripe max cell length: raise from 500 to 2000 for 3+ column
   tables. Tables with paragraph descriptions in one column are valid
   (fixes 121).

4. Row-stripe empty-column trimming: apply the same outer-column trim
   as grid detection (fixes 121 column-0 rejection).

TEDS: 0.377 → 0.438 on opendataloader-bench (gap: -0.056 vs odl).
TEDS=0 docs: 14 → 9.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tables): enable 2-column body-font tables + lower all minimums

- Lower BodyFont minimum columns from 3 to 2 in detect_table_in_region
- Lower BodyFont minimum rows from 3 to 2
- Lower avg_cells_per_row minimum from 2.0 to 1.5 (handles wrapped cells
  in 2-column tables)
- Apply empty-outer-column trimming to row-stripe detection (not just grid)

TEDS: 0.438 → 0.468 on opendataloader-bench (gap: -0.027 vs odl).
TEDS=0 docs: 9 → 8. 86% of original gap closed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tables): text-based row fallback + fix 120 flow-chart and 188 leaderboard

Three changes that push TEDS past opendataloader:

1. Cell-rect Y-edge fallback: when rects have too few Y-edges for row
   structure, derive rows from text Y-position clustering within the
   rect bounding box. Fixes flow-chart tables (120) and column-header-
   only rects (188).

2. Lower cell-rect minimum from 20 to 6 rects to catch smaller tables.

3. Relax validation 1 (first-column presence) from 50% to 25% of rows.
   Tables with wrapped model names have continuation lines without first
   column content.

TEDS: 0.468 → 0.508 on opendataloader-bench.
Now BEATS opendataloader (0.508 vs 0.494, gap=+0.014).
TEDS=0 docs: 8 → 5.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tables): wrapped-cell continuation row merging

Merge rows that have fewer filled cells than the header row into the
previous row. Handles wrapped multi-line cells where text overflow
creates extra rows (e.g., "Direct" + "communications" → "Direct
communications").

Conditions: fewer filled cells than header, more than previous row had,
not a data row (numeric), not a short subheader label.

TEDS: 0.508 → 0.522 on opendataloader-bench (now +0.028 vs odl).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tables): tighten cell-rect validation + fix continuation-row merging

Cell-rect false positives:
- Raise density threshold back to 25% (from 15%)
- Add max cell length check (500 chars) to reject paragraph content
- Reject disproportionate grids (>20 rows, <4 cols)

Continuation-row merging:
- Wide tables (5+ cols): only merge rows with ≤50% header cells
- Narrow tables (2-4 cols): merge rows with fewer cells than header
- Prevents merging normal data rows in large tables (6_KE_Chart)
  while keeping wrapped-cell merging for narrow tables (178)

TEDS: 0.498 on opendataloader-bench (still +0.004 vs odl).
pdf-evals: 191/192 passed, 0 regressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 16:57:51 -07:00

197 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
(e) [Reserved]. For further guidance, see §1.1563-3T(e)(1). Par. 50. Section 1.1563-3T is added to read as follows:
§1.1563-3T Rules for determining stock ownership (temporary).
(a) through (d)(2)(iii) [Reserved]. For further guidance, see §1.1563-3(a)
through (d)(2)(iii). (iv) Statement. If the application of paragraph (d)(2)(ii) or (iii) of §1.1563-3 does not result in a corporation being treated as a component member of only one controlled group of corporations on a December 31, then such corporation will be treated as a component member of only one such group on such date. Such corporation may elect the group in which it is to be included by including on or with its income tax return a statement entitled, “STATEMENT TO ELECT CONTROLLED GROUP PURSUANT TO §1.1563-3T(d)(2)(iv).” The statement must include--
(A) A description of each of the controlled groups in which the corporation
could be included. The description must include the name and employer identification number of each component member of each such group and the stock ownership of the component members of each such group; and
(B) The following representation: [INSERT NAME AND EMPLOYER
IDENTIFICATION NUMBER OF CORPORATION] ELECTS TO BE TREATED AS A COMPONENT MEMBER OF THE [INSERT DESIGNATION OF GROUP].
(v) Election-- (A) Election filed. An election filed under paragraph (d)(2)(iv) of
this section is irrevocable and effective until paragraph (d)(2)(ii) or (iii) of §1.1563-3 applies or until a change in the stock ownership of the corporation results in
|termination of membership in the controlled group in which such corporation has been included. (B) Election not filed.|In the event no election is filed in accordance with the|
|---|---|
|provisions of paragraph (d)(2)(iv) of this section, then the Internal Revenue Service||
|will determine the group in which such corporation is to be included. Such||
|determination will be binding for all subsequent years unless the corporation files a||
|valid election with respect to any such subsequent year or until a change in the||
|stock ownership of the corporation results in termination of membership in the||
|controlled group in which such corporation has been included.||
|(d)(3) [Reserved]. For further guidance, see §1.1563-3(d)(3).||
|(e) Effective date-- (1) Applicability date.|This section applies to any original|
Federal income tax return (including any amended return filed on or before the due date (including extensions) of such original return) timely filed on or after May 30,
2006.
(2) Expiration date. The applicability of this section will expire on May 26,
2009. Par. 51. Section 1.6012-2 is amended by revising paragraph (c) and adding paragraph (k) to read as follows: §1.6012-2 Corporations required to make returns of income.
* * * * *
(c) [Reserved]. For further guidance, see §1.6012-2T(c).
* * * * *
(k) [Reserved]. For further guidance, see §1.6012-2T(k)(1).
Par. 52. Section 1.6012-2T is added to read as follows: §1.6012-2T Corporations required to make returns of income (temporary).
(a) through (b) [Reserved]. For further guidance, see §1.6012-2(a) through
(b).
(c) Insurance companies-- (1) Domestic life insurance companies-- (i) In
general. A life insurance company subject to tax under section 801 shall make a return on Form 1120L. Except as provided in paragraph (c)(4) of this section, such company shall file with its return--
(A) A copy of its annual statement which shows the reserves used by the
company in computing the taxable income reported on its return; and
(B) A copy of Schedule A (real estate) and of Schedule D (bonds and stocks),
or any successor thereto, of such annual statement. (ii) Mutual savings banks. Mutual savings banks conducting life insurance business and meeting the requirements of section 594 are subject to partial tax computed on Form 1120 and partial tax computed on Form 1120L. The Form 1120L is attached as a schedule to Form 1120, together with the annual statement and schedules required to be filed with Form 1120L.
(2) Domestic nonlife insurance companies. Every domestic insurance
company other than a life insurance company shall make a return on Form 1120PC. This includes organizations described in section 501(m)(1) that provide commercial- type insurance and organizations described in section 833. Except as provided in paragraph (c)(4) of this section, such company shall file with its return a copy of its
annual statement (or a pro forma annual statement), including the underwriting and investment exhibit for the year covered by such return.
||(3) Foreign insurance companies. The provisions of paragraphs (c)(1) and|
|---|---|
||(c)(2) of this section concerning the returns and statements of insurance companies subject to tax under section 801 or section 831 also apply to foreign insurance companies subject to tax under those sections, except that the copy of the annual statement required to be submitted with the return shall, in the case of a foreign insurance company that is not required to file an annual statement, be a copy of the pro forma annual statement relating to the United States business of such company. (4) Exception for insurance companies filing their Federal income tax returns electronically. If an insurance company described in paragraph (c)(1), (c)(2), or (c)(3) of this section files its Federal income tax return electronically, it should not include on or with such return its annual statement (or pro forma annual statement), or any portion thereof. Such statement must be available at all times for inspection by authorized Internal Revenue Service officers or employees and retained for so long as such statements may be material in the administration of any internal revenue law. See §1.6001-1(e). (5) Definition. For purposes of this section, the term annual statement means the annual statement, the form of which is approved by the National Association of Insurance Commissioners (NAIC), which is filed by an insurance company for the year with the insurance departments of States, Territories, and the District of|
Columbia. The term annual statement also includes a pro forma annual statement if the insurance company is not required to file the NAIC annual statement.
(d) through (j) [Reserved]. For further guidance, see §1.6012-2(d) through (j).
(k) Effective date-- (1) Applicability date. This section applies to any original
Federal income tax return (including any amended return filed on or before the due date (including extensions) of such original return) timely filed on or after May 30,
2006.
(2) Expiration date. The applicability of this section will expire on May 26,
2009.
|||Par. 53. For each entry in the “Location” column of the following table,|
|---|---|---|
||remove the language in the “Remove” column and add the language in the “Add”||
|column in its place:|||
|Location|Remove|Add|
|The last sentence of the|The following rules shall|The rules described in|
|introductory text to|be applicable in|paragraph (a) of §1.302-|
|§1.302-4|determining whether the specific requirements of section 302(c)(2) are met:|4T and in paragraphs (b) through (g) of this section apply in determining whether the specific|
requirements of section 302(c)(2) are met.
|§1.338(h)(10)-1(f)|§1.331-1(d), and §1.332-|§1.331-1T(d) and §1.332-|
|---|---|---|
||6|6T|
|The last sentence of|paragraph (a)(2)(ii) of this|paragraph (a) of §1.382-|
|§1.382-2T(h)(4)(vi)(B)|section|11T|
|The first sentence of §1.382-6(b)(2)(i)|§1.382-2T(a)(2)(ii)|§1.382-11T(a)|
|The second sentence of|paragraph (c) of this|paragraphs (c)(1), (c)(3),|
|§1.382-8(a)|section|(c)(4) and (c)(5) of this|
section and paragraph
(c)(2) of §1.382-8T
|The third sentence of|paragraph (c) of this|paragraphs (c)(1), (c)(3),|
|---|---|---|
|§1.382-8(a)|section|(c)(4) and (c)(5) of this section and paragraph (c)(2) of §1.382-8T|
|§1.382-8(c)(3)|paragraph (c)(2) of this section|paragraph (c)(2) of §1.382-8T|
|The first sentence of|paragraphs (c)(1), (2),|paragraphs (c)(1) and|
|§1.382-8(c)(4)|and (3) of this section|(c)(3) of this section and paragraph (c)(2) of §1.382-8T|
|§1.382-8(c)(5)|this paragraph (c)|paragraphs (c)(1), (c)(3),|
(c)(4), and (c)(5) of this section, and paragraph
(c)(2) of §1.382-8T
|The fifth sentence of|paragraph (c) of this|paragraphs (c)(1), (c)(3),|
|---|---|---|
|§1.382-8(f)|section|(c)(4), and (c)(5) of this section, and paragraph (c)(2) of §1.382-8T|
|§1.382-8(g), Example|paragraph (c) of this|paragraphs (c)(1), (c)(3), section, and paragraph (c)(2) of §1.382-8T|
|The second sentence of|paragraph (c) of this|paragraphs (c)(1), (c)(3),|
|§1.382-8(g), Example|section|(c)(4), and (c)(5) of this|
(1)(b)(2) section (c)(4), and (c)(5) of this
(1)(c) section, and paragraph
(c)(2) of §1.382-8T
|§1.382-8(g), Example|paragraph (c)(2) of this|paragraph (c)(2) of|
|---|---|---|
|The first sentence of|paragraph (c)(2) of this|paragraph (c)(2) of|
|§1.382-8(g), Example|section|§1.382-8T|
|§1.382-8(g), Example|paragraph (c)(2) of this|paragraph (c)(2) of|
|§1.382-8(g), Example|paragraphs (c)(1) and (2)|paragraph (c)(1) of this|
(2)(c) section §1.382-8T
(2)(e)
(3)(b) section §1.382-8T
(3)(c)(1)(B) of this section section and paragraph
(c)(2) of §1.382-8T
|The second sentence of|paragraph (c)(2) of this|paragraph (c)(2) of|
|---|---|---|
|§1.382-8(g), Example|section|§1.382-8T|
|The second sentence of|paragraph (c)(2) of this|paragraph (c)(2) of|
|§1.382-8(g), Example|section|§1.382-8T|
|The first sentence of|paragraph (b)(4)(iv) of|paragraph (b)(4)(iv) of|
|§1.1502-32(b)(4)(v)(A)|this section|§1.1502-32T|
|The first sentence of|paragraph (b)(4)(iv) of|paragraph (b)(4)(iv) of|
|§1.1502-32(b)(4)(v)(B)|this section|§1.1502-32T|
(4)(c)
(5)(c)
|§1.1502-35(c)(4)(ii)(B)|§1.1502-76(b)(2)(ii)(D)|§1.1502-76T(b)(2)(ii)(D)|
|---|---|---|
|§1.1502-76(b)(2)(ii)(A)(2)|paragraph (b)(2)(ii)(D) of this section|paragraph (b)(2)(ii)(D) of §1.1502-76T|
|§1.1502-92(e)(1)|§1.382-2T(a)(2)(ii)|§1.382-11T(a)|
|The first sentence of §1.1502-92(e)(2)|§1.382-2T(a)(2)(ii)|§1.382-11T(a)|
|The first sentence of §1.1502-94(d)|§1.382-2T(a)(2)(ii)|§1.382-11T(a)|
|The second sentence of §1.1502-94(d)|§1.382-2T(a)(2)(ii)|§1.382-11T(a)|
|The last sentence of|paragraph (f) of this|paragraph (f) of §1.1502-|
|§1.1502-95(b)(3)|section|95T|
|The last sentence of|subdivision (ii) of this|paragraph (c)(2)(i) of|
|§1.1563-1(c)(2)(iv), Example (1)|subparagraph|§1.1563-1T|
|The last sentence of|the district director with|the Internal Revenue|
|§1.1563-1(c)(2)(iv), Example (1)|audit jurisdiction of N’s return|Service|
|The third sentence of|subdivision (iii) of this|paragraph (c)(2)(ii) of|
|§1.1563-1(c)(2)(iv), Example (2)|subparagraph|§1.1563-1T|
|The third sentence of|the district director with|the Internal Revenue|
|§1.1563-1(c)(2)(iv), Example (2)|audit jurisdiction of the return of the corporation whose taxable year ends on the earliest date|Service|
|The last sentence of|district director|Internal Revenue Service|
§1.1563-1(c)(2)(iv), Example (2)
|The second sentence of|subdivisions (ii), (iii), and|paragraphs (d)(2)(ii) and|
|---|---|---|
|§1.1563-3(d)(2)(i)|(iv) of this subparagraph|(iii) of this section, and paragraph (d)(2)(iv) of §1.1563-3T|
|The first sentence of|§1.332-6(b), 1.368-3(a),|§1.332-6T(a), §1.368-|
|§1.6043-2(a)|or 1.1081-11|3T(a), or §1.1081-11T|
|The first sentence of §301.6011-5T(a) (twice)|§1.6012-2|paragraphs (a), (b) and (d) through (j) of §1.6012- 2, and paragraph (c) of §1.6012-2T|
|||PART 602--OMB CONTROL NUMBERS UNDER THE PAPERWORK||
|---|---|---|---|
||REDUCTION ACT Authority: 26 U.S.C. 7805. 1. The following entries to the table are removed: §602.101 OMB Control numbers.|Par. 54. The authority citation for part 602 continues to read as follows: Par. 55. In §602.101, paragraph (b) is amended to read as follows:||
|* * * * *|(b) * * * CFR part or section where identified or described||Current OMB control No.|
|* * * * *|1.332-6………………………………………………………………….|1.382-11……………………………………………………………….. 1545-2019 1.351-3…………………………………………………………………. 1545-2019 1.355-5…………………………………………………………………. 1545-2019 1.368-3…………………………………………………………………. 1545-2019 1.1081-11………………………………………………………………. 1545-2019|1545-2019|
|* * * * *|§602.101 OMB Control numbers.|______________________________________________________________ 2. The following entries are added in numerical order to the table:||
|* * * * *|(b) * * * CFR part or section where identified or described||Current OMB control No.|
|* * * * *|1.302-2T………………………………………………………………… 1545 1.302-4T………………………………………………………………… 1545||-2019 -2019|
|1.331-1T………………………………………………………………… 1545|-2019|
|---|---|
||1.332-6T………………………………………………………………... 1545-2019 1.338-10T………………………………………………………………. 1545-2019|
|1.351-3T………………………………………………………………… 1545|-2019|
|1.355-5T………………………………………………………………… 1545|-2019|
|1.368-3T………………………………………………………………… 1545|-2019 1.381(b)-1T…………………………………………………………….. 1545-2019|
|1.382-8T………………………………………………………………… 1545|-2019 1.382-11T………………………………………………………………. 1545-2019 1.1081-11T……………………………………………………………… 1545-2019 1.1221-2T……………………………………………………………….. 1545-2019|
|1.1502-13T………………………………………………………………|1545-2019 1.1502-31T……………………………………………………………… 1545-2019 1.1502-32T……………………………………………………………… 1545-2019 1.1502-33T……………………………………………………………… 1545-2019 1.1502-35T……………………………………………………………… 1545-2019 1.1502-76T……………………………………………………………… 1545-2019|
|1.1502-95T……………………………………………………………… 1545|-2019 1.1563-1T……………………………………………………………….. 1545-2019|
1.1563-3T……………………………………………………………….. 1545-2019
1.6012-2T……………………………………………………………….. 1545-2019
* * * * * Mark E. Matthews
Deputy Commissioner for Services and Enforcement.
Approved: May 19, 2006 Eric Solomon Acting Deputy Assistant Secretary of the Treasury (Tax Policy).
Reference in New Issue View Git Blame Copy Permalink