Scanned candidate emitted with corrupted bytes (`child` → `U+FFFD \x06 U+FFFD \x01`) by a long-lived `next dev` (Turbopack) process — evidence against the scanner's `from_utf8_unchecked` safety assumption

[skirchner2323-kinetic]

Summary

A long-running next dev (Turbopack) process in a pnpm monorepo entered a state
where Tailwind's generated CSS contained a corrupted copy of one arbitrary-variant
candidate. The bytes of child in the candidate [&>*:last-child]:col-span-2
(present verbatim in a .tsx source file) were replaced by the four characters
U+FFFD U+0006 U+FFFD U+0001, producing unparseable CSS. Every route then failed
with lightningcss's Parsing CSS source code failed, and the state persisted
in memory until the dev server was restarted. A production next build of the
identical tree compiled cleanly every time, before, during, and after.

We do not have a reproduction recipe — this is a forensic report of a corrupted
state captured live. We're filing it because the corruption is demonstrably in
the candidate string itself, before variant expansion (see Analysis), which
bears directly on the safety assumption that led to closing #19410
(from_utf8_unchecked in the scanner is fine because "the extractor's
implementation should guarantee that the resulting candidates are already
UTF-8"). The U+FFFD characters indicate a lossy UTF-8 conversion happened
somewhere downstream of bytes that were not valid UTF-8.

Environment

• tailwindcss / @tailwindcss/postcss: 4.3.3 (@tailwindcss/oxide-darwin-arm64@4.3.3)
• next: 16.2.12, dev server (Turbopack), via the PostCSS plugin (postcss.config.mjs → @tailwindcss/postcss)
• node: v26.5.0 (nvm), pnpm 11, pnpm workspace monorepo (app at apps/web)
• macOS 26.5.2, Apple Silicon (arm64)
• Dev server had been running for a long stretch (hours+) and had served many
  on-demand route compiles; the machine was under heavy load at times
  (parallel Playwright suites against sibling dev servers, several next dev
  processes in worktrees of the same repo).

The source candidate

apps/web/src/app/(practitioner)/dashboard/page.tsx:

className="grid grid-cols-2 gap-px overflow-hidden rounded-xl border border-inst-hairline bg-inst-hairline sm:grid-cols-3 lg:grid-cols-5 [&>*:last-child]:col-span-2 lg:[&>*:last-child]:col-span-1"

The error (verbatim, from the dev-overlay payload)

Every route on the affected server returned HTTP 500 with this compilation
error (control characters rendered here as <0x06> / <0x01>; � is a literal
U+FFFD REPLACEMENT CHARACTER present in the server's output):

./apps/web/src/app/globals.css:2848:51
Parsing CSS source code failed
  2846 |     }
  2847 |   }
> 2848 |   .\[\&\>\*\:last-�\6 �\1 \]\:col-span-2 > :last-�<0x06>�<0x01> {
       |                                               ^
  2849 |     grid-column: span 2 / span 2;
  2850 |   }
  2851 | }

Unexpected token Delim('\u{6}')

Generated code of PostCSS transform of file content of apps/web/src/app/globals.css:
./apps/web/src/app/globals.css:2848:51
[...same excerpt...]

Import trace:
  Client Component Browser:
    ./apps/web/src/app/globals.css [Client Component Browser]
    ./apps/web/src/app/layout.tsx [Server Component]

Note the class-name position shows the CSS-escaped forms \6 and \1
(escapes of U+0006 / U+0001), while the selector tail shows the same characters
raw — both derived from one candidate whose child had become
U+FFFD U+0006 U+FFFD U+0001.

Analysis: the corruption is in the candidate, not the CSS text

The utility [&>*:last-child]:col-span-2 generates a rule whose selector uses
the candidate twice, through two different code paths:

1. the escaped class name — .\[\&\>\*\:last-child\]\:col-span-2
   (string-escaping of the candidate), and
2. the expanded variant — > :last-child (obtained by parsing the
   arbitrary-variant body out of the candidate and substituting &).

In the corrupted output, both positions carry the identical 4-character
replacement of the 5 characters child. Random corruption of the generated
CSS text (e.g. in Turbopack's string handling downstream) cannot plausibly hit
two different positions, through two different transforms (one CSS-escaped, one
raw), with the same replacement. The candidate string already contained
last-�<0x06>�<0x01> when the compiler parsed and expanded it. The corruption
therefore happened at or before candidate intake — i.e. in the scanner/extractor
or in the transfer of its results.

Two details that may help locate it:

• U+FFFD means a lossy UTF-8 decode happened after invalid bytes
  appeared. The scanner converts extracted byte slices with
  String::from_utf8_unchecked (crates/oxide/src/scanner/mod.rs); a slice
  containing invalid UTF-8 would sail through unchecked and only be replaced
  with U+FFFD later (e.g. at the napi string boundary). The surviving
  0x06/0x01 are valid single-byte UTF-8, consistent with a raw byte
  pattern like <invalid> 06 <invalid> 01 at extraction time.
• The affected .tsx file is plain ASCII at that position and was not being
  edited when the corruption arose; the persistent on-disk chunk for
  globals.css in .next was healthy (…:last-child… intact), so the
  corrupted candidate lived only in the long-running process's incremental
  state (the plugin holds a persistent Scanner whose candidate set
  accumulates across scan() calls — once a corrupted candidate enters that
  set, every subsequent rebuild re-emits the broken rule, which matches the
  observed "stuck until restart" behavior).

What we can and can't provide

• Full captured 500 payload (dev-overlay JSON incl. ANSI variant) — available
  on request; excerpted verbatim above.
• No reproduction recipe: restarting the dev server cleared it, and we have
  not been able to trigger it on demand. Conditions at the time: hours-old
  dev server, many on-demand route compiles, heavy parallel filesystem/CPU
  load on the machine.
• next build (non-Turbopack CSS path, fresh scanner) of the same tree:
  always clean.

We understand a report without a repro may not be actionable on its own; we're
filing primarily as concrete field evidence that extracted candidates can
contain invalid UTF-8 in real conditions, contra the invariant cited when
closing #19410 — a cheap from_utf8 (or debug assertion) at that boundary
would turn this silent state-corruption into a loud, attributable error.

[thejamesgore]

So I reproduced this without the race and it looks like a soundness bug potentially.

A .tsx file with one stray continuation byte in a class: <div class="text-red-500 [color:var(--x\x80)]"></div>

Scanner::scan() on main commit 4be6110 returns candidates that aren't valid UTF-8

INVALID  [45, 45, 120, 128]                     (invalid utf-8 sequence of 1 bytes from index 3)
INVALID  [91, 99, ... 45, 45, 120, 128, 41, 93] (invalid utf-8 sequence of 1 bytes from index 14)
ok       "text-red-500"

This is the same with 0xFE 0xFF and with a truncated E2 80 and when the bad byte is removed everything comes back clean

From what I understand the extractor validates bytes one at a time but from_utf8_unchecked needs the whole sequence to be valid UTF-8. CssVariableMachine accepts 0x80..=0xff as identifier characters which is correct for the CSS spec but a lone 0x80 passes that check and still isn't valid UTF-8.

It doesn't need the dev server race either as any Latin-1 encoded source file does it. The fuzz target can't catch it because it drops non-UTF-8 input before the scanner runs so only the &str path is covered and never the byte path fs::read feeds.

So I see a few different approaches on how to fix this with the simplest being a checked conversion at the three call sites in scanner/mod.rs dropping any candidate that fails. However, a more thorough approach could be ensuring we validate complete code points rather than individual bytes but I'm not completely sure as the spec linked in that code comment is about code points and an emoji only satisfies it as a full sequence.

So should that range be matching whole sequences rather than single bytes? Those two aren't exlcusive and both could make sense together. I'd avoid from_utf8_lossy as that would turn the U+FFFD in this report into expected behaviour.

Happy to PR a fix if useful based on one of those solutions or an alternative if suggested.

[blackwellmiranda8]

🚀⚙️✨ 结构化输出强制 JSON 稳不稳

[hooperjohn8]

💡✨😄 本地开发一键依赖太香了

[martintraci7040]

💡🚀 想贡献 PR，从哪块入手合适

[harrisonmichael809]

和现有脚手架集成顺畅吗 💡🧪

[harrisonmichael809]

事务边界会不会过大 🔥⚙️👍

[rodriguezrussell93]

无障碍方面有考虑过吗 🙌

[peter-pro-50]

It might be worth investigating how the from_utf8_unchecked function handles input during long-lived processes, particularly focusing on any potential memory corruption that could lead to lossy conversions. Additionally, if the corrupted bytes emerge consistently under specific conditions, that could help narrow down the root cause related to the Turbopack dev server state management.

[mvanhorn]

Implemented in #20389.