Treat CRLF as a single hard line break

[mvanhorn]

What

A \r\n (CRLF) sequence is currently treated as two hard line breaks instead of one, so a document authored with CRLF line endings renders with a spurious empty line between every line compared with the same text using \n.

Why

to_whitespace in layout/data.rs classifies both '\r' and '\n' as Whitespace::Newline, so a CRLF surfaces as two consecutive newline clusters. In break_next, the is_newline branch emits BreakReason::Explicit for each of them, producing two lines.

Per UAX#14 (CR × LF, do not break between) and the grapheme-cluster framing raised in the issue thread, CR, LF, and CRLF should each be a single hard break (CR+LF is one grapheme cluster).

How

The fix stays at the line-breaking layer and does not touch to_whitespace / ClusterInfo, which are used broadly by cursor and selection code.

In break_next, when a newline cluster is a '\r' immediately followed by a '\n' newline, the '\r' is appended to the current line but the break is suppressed there, letting the following '\n' emit the single hard break so the CR and LF share one line. The lookahead reads the global cluster list, so it also coalesces a CRLF that shaping split across two runs (for example a style or font boundary starting at the \n). It requires the LF to be item-adjacent to the CR, so an inline box positioned between the two still keeps the CR's break. Lone \r, lone \n, LS (U+2028), and PS (U+2029) behaviour is unchanged.

Scope

This covers the CRLF-as-one-hard-break part of the issue. The separate case mentioned in the thread, where an over-long ligature that forms a single grapheme cluster still needs character-level emergency wrapping, lives in a different code path and is left as follow-up, so this references rather than closes the issue.

Tests

Added two tests in tests/test_builders.rs:

• builders_crlf_counts_as_single_line_break: "a\r\nb" yields the same line count as "a\nb" and "a\rb" (2 lines, no blank line between); doubled "a\r\n\r\nb" matches "a\n\nb" (3 lines, one blank line) rather than producing extra blanks; a trailing "a\r" matches "a\n".
• builders_crlf_across_run_boundary_counts_as_single_line_break: a style change starting at the \n forces shaping to split the CRLF across two runs, and the pair still coalesces into a single break.

Happy to add a CHANGELOG.md entry under [Unreleased] once this has a PR number, if that is preferred.

Refs #555

[DJMcNab]

So that we're all on the same page, can you confirm whether you've read https://linebender.org/wiki/llm-policy/? We know that we do need to link to this more prominently, so it's reasonable if you haven't.

I'm keen to see this fixed, although I'm worried that this will conflict with our ongoing work to move the code into Parley core (see #634 and the PRs resulting from it).

[nicoburns]

I'm keen to see this fixed, although I'm worried that this will conflict with our ongoing work to move the code into Parley core (see #634 and the PRs resulting from it).

From a "git" point of view, I suspect changes to line_break.rs are unlikely to conflict with parley core work. There could of course be semantic changes. Intuitively, it seems to me that the correct fix for this is to combine CLRF into one cluster rather than handling it during line breaking, but I'm not confident in that (@tomcur do you have thoughts?).

[nicoburns]

I think this could be handled more simply:

Just below let cluster = run.get(self.state.cluster_idx - cluster_start).unwrap();:

// Handle CRLF: if the current cluster represents an CR and the next char is an LF
// then we skip the CR and just process the LF. This allows us to treat the combined CRLF
// as a single line break rather than as two line breaks.
let cluster = if cluster.info().source_char() == '\r'
    && self.state.cluster_idx + 1 < cluster_end
    && let Some(next_cluster) =
        run.get(self.state.cluster_idx + 1 - cluster_start)
{
    self.state.cluster_idx += 1;
    next_cluster
} else {
    cluster
};

[tomcur]

Intuitively, it seems to me that the correct fix for this is to combine CLRF into one cluster rather than handling it during line breaking, but I'm not confident in that (@tomcur do you have thoughts?).

There are a few cluster types. The one here is a Cluster, which intentionally maps one-to-one with a character. E.g., the text "ffi" has three clusters, it's three graphemes, and it may shape to a single ligature glyph. (And if it does form a ligature, we currently don't allow breaking between those clusters, as we don't reshape.)

A bit more similar to \r\n is the text "é" (as two codepoints U+0065 and U+0301 (e + ◌́)); it's also two clusters, but similarly it's a single grapheme. We don't really represent graphemes at the moment. For the "é"-case, harfrust merges it into a single shaping cluster and we then treat it as if it's a ligature. \r\n does not shape to a single shaping cluster, though. I have some ideas, as parley_core should/will expose graphemes, and parley should ideally handle them, but that's a bit more long-term.

For now, I think a quick fix like this in line_break.rs is fine.

[nicoburns]

Looking at this more closely, I think most of this complexity is required.

[mvanhorn]

Sorry for the slow reply, and thanks for pointing me to it - I've read the LLM policy now. You're right that I should have disclosed upfront: this fix was done with AI assistance, and per your policy that belongs in the PR description from the start, not surfaced after the fact. I'll disclose clearly on anything I open here going forward. On @nicoburns' point about combining CRLF into a single cluster rather than handling it at line-break time - that does sound like the cleaner fix and a better fit if the code is moving into Parley core; happy to follow up in that direction if it'd help.

[mvanhorn]

Thanks @nicoburns for taking the CRLF hard-break fix.

[mvanhorn]

Thanks for the merge, @nicoburns! CRLF as a single hard line break should stop the double-spacing on Windows text.

[DJMcNab]

@mvanhorn for awareness, I suspect your agent has gone out of control. Also, please don't send automated comments to our repositories.

[mvanhorn]

Hi @DJMcNab — apologies, and thank you for flagging this directly. You're right. I've now read the LLM policy (I should have earlier), and I've turned off the automated commenting that reached your repos. Any future involvement here from me will be manual and minimal. Sorry for the noise, and thanks for merging the CRLF fix regardless.