fix(cjk): Skip Latin case heuristics for CJK text in fallback spacing path

The fallback estimated-width path in should_join_items() used Latin
case-based thresholds (uppercase/lowercase transitions) to decide word
boundaries. For CJK text emitted as per-glyph items, these heuristics
incorrectly inserted spaces within words (e.g., "ス テ ップ" instead of
"ステップ"). Now detects CJK characters and uses a generous join
threshold, matching the behavior already applied in the accurate-width path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Abimael Martell
2026-02-13 13:25:22 -08:00
co-authored by Claude Opus 4.6
parent 17634e80f4
commit d2c96e2842
+7
View File
@@ -755,6 +755,13 @@ fn should_join_items(prev_item: &TextItem, curr_item: &TextItem) -> bool {
// Calculate gap between items // Calculate gap between items
let gap = curr_item.x - prev_end_x; let gap = curr_item.x - prev_end_x;
// CJK text: always join adjacent items — CJK languages don't use spaces between words.
// The Latin case-based heuristics below would incorrectly insert spaces within CJK words.
let is_cjk = prev_last.is_some_and(is_cjk_char) || curr_first.is_some_and(is_cjk_char);
if is_cjk {
return gap < char_width * 0.8;
}
// Use different thresholds based on character case // Use different thresholds based on character case
// Same-case sequences (ALL CAPS or all lowercase) are more likely to be // Same-case sequences (ALL CAPS or all lowercase) are more likely to be
// word fragments that got split. Mixed case suggests word boundaries. // word fragments that got split. Mixed case suggests word boundaries.