Skip to content

feat(compiler,harness): i18n v2 — a delve ships every language and the client picks (spec-0029) - #314

Open
stellarfeline wants to merge 3 commits into
mainfrom
worker/i18n-v2
Open

feat(compiler,harness): i18n v2 — a delve ships every language and the client picks (spec-0029)#314
stellarfeline wants to merge 3 commits into
mainfrom
worker/i18n-v2

Conversation

@stellarfeline

@stellarfeline stellarfeline commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Implements spec-0029 — i18n v2: the client picks the language (owner directive 2026-08-06).
Depends on the spec PR (docs/spec-0029-i18n-v2) landing; nothing here duplicates that file.

nobodys-cave-island v1.0.0 shipped Chinese-only, because build --lang zh-cn swaps every
player-visible string before compiling: the datapack carries exactly one language and there is
no fallback path at all. This makes a delve carry all of them.

The emission shape

Every authored player-visible string becomes a text component carrying its l10n key and its
English source; one assets/delvewright/lang/<mc_code>.json per declared language, plus
en_us.json, goes into the resource pack the release already ships.

Real before/after, nobodys-cave-island, function/campaign_complete.mcfunction:

- tellraw @a [{"color":"gold","text":"Nobody's Isle — complete."},{"text":"\n"},{"color":"gray","text":"A Delvewright delve."}]
+ tellraw @a [{"color":"gold","fallback":"Nobody's Isle","translate":"world.title"},{"color":"gold","text":" — complete."},{"text":"\n"},{"color":"gray","text":"A Delvewright delve."}]

Note the literal concatenation {title} — complete. splitting into two components: a translate
key cannot carry a suffix. Same shape in SNBT (CustomName:{fallback:"The Keeper",translate:"npc.keeper.name"})
and in dialogs ("label": {"fallback":"Who are you?","translate":"dlg.keeper.greeting.opt.0.label"}).

fallback rides the component, not the pack's en_us.json — deliberately, and this is the
one call not to "simplify". A player who declines the resource-pack prompt has no lang files at
all; a design leaning on pack-level fallback fails exactly that player.

Three things kept this cheap, all confirmed before designing anything: each_string already
visits every player-visible string under a stable key across all five effect roots (that is the
key set — no new scheme, no new inventory); resourcepack::build_pack(skins, extra) already
takes arbitrary extra assets (lang files are a caller change); DW0330/DW0331 already
width-check source and every declared translation — now load-bearing rather than
belt-and-braces, since any declared language may be what a player sees.

The risk, closed as an invariant rather than an audit

An authored string that does not land in a text component cannot carry a translate key.
Enumerating the emission sites once and trusting the list is exactly the kind of proof that
rots. Instead:

dsl::l10n::tag_translatables rewrites each inventoried string to <U+E000><key><U+E000><English>
once, before Plan::build. An emitter either lowers it through emit::tr / emit::snbt_component,
or reads it through dsl::l10n::plain. A tag still present in the finished tree is a site that
did neither
DW0185, build-tier, feature-blind, run last over the finished output beside
DW0497, so it guards emitters not yet written. The scan is total: a file that is neither
UTF-8 text nor a classified verbatim binary output (.nbt, .png, resourcepack.zip) also
fails, so a new binary artifact cannot quietly opt out.

DW0183 reserves the private-use block in authored and translated content, so no string can
forge or shadow a tag. DW0184 rejects a declared language with no Minecraft lang-file code —
never a silently dropped language.

Named exclusions (also in compiler.md §2)

Site Artifact Why it is not a component
emit::artifact_title packtest-datapack/** #> test descriptions A generated test's own description; read by the validation server and a maintainer, never rendered to a player.
emit::emit_packtest (dialogue-visibility test) v04_dialogue_visibility.mcfunction Same — the option label sits in the test description line.
render_plan::{npc_name, area_name_of, first_clause} + the NPC shot expect render-plan.json The reviewer/vision artifact: its prose is read by a vision model against a frame, in English, whatever the delve ships.

critical-path.json, validation/*.json, combat-plan.json and manifest.json carry ids, never
authored prose — the bot contract was already language-neutral, so they need no exclusion.
validate, analyze, l10n-inventory, snapshot and edit never see a tag: tagging happens
inside build, after validation.

One design consequence worth flagging

emit_narrate used to to_ascii_uppercase() an art-styled narrate — a transform a
{"translate": …} component cannot express, because the client resolves the lang file after the
compiler is gone. Rather than bake case into content or into the lang values (which would break
AC2), the delve:art font gained a second bitmap provider over the same atlas addressed by the
lowercase letters
. A lowercase letter renders through its uppercase bitmap: identical pixels,
now reachable by translation. Cells with no lowercase form use vanilla's unused-cell marker
(U+0000), so no character is claimed twice.

Acceptance criteria — spec-0029, with binding counts

All in crates/compiler/tests/i18n_v2.rs, driven through the shipped delvec build path
(a hand-assembled Plan emits the same components with literal bodies and would prove nothing
about delivery). Every set states its binding; a zero binding fails.

AC How it is satisfied Binding
1 — a ["zh-cn"] campaign emits exactly en_us.json + zh_cn.json, key sets equal ac1_a_declared_language_ships_its_lang_file_beside_english; the emitter re-proves equality over the bytes it is about to write (DW0180/DW0181) keep-trial: 21 keys × 2 languages. Island, measured by hand: 431 × 2, sets equal
2 — every en_us.json value equals the English source, compared against a fresh inventory, not a fixture ac2_english_lang_file_is_the_live_inventoryen_us.json is dsl::l10n::inventory of the stage docs 21 inventory keys compared, key and value
3 — no emitted datapack file ships an authored literal where a translate key belongs ac3_ac4_… walks every fixture's tree for "text":"<English>" outside the exclusion table; DW0185 proves the complementary half 5 campaigns. Island checked by hand: 0 literal occurrences over 431 strings
4 — every translatable component carries a non-empty fallback equal to the English source; binding printed; zero examined = failure ac3_ac4_… + every_translate_component_carries_its_own_fallback (JSON and SNBT forms) 5 campaigns, 118 distinct keys, 118 component occurrences; the fallback walk examines 69 components on keep-trial alone. Island: 419 of 431 keys emitted as components (the other 12 — see finding below)
5 — an unmappable declared language is a compile error naming language + table ac5_an_unmappable_language_is_dw0184; DW0184 at validate (exit 1), message names kl-ingon and mc_lang_code 1 red asserted
6 — determinism holds, lang files included ac6_double_build_is_byte_identical_lang_files_included; lang maps are BTreeMaps inside the existing STORE zip keep-trial: whole tree identical, 2 lang files. Island: 609 files, byte-identical across two builds
7 — the bot's critical path passes and the run report states the candidate-preference binding See "Harness" below. The bot ladder itself is tier 3 (release candidates) and is not run in this PR Report field always emitted, unbound stated explicitly
8 — the island builds with both languages; locale-unset reads English Island builds green (exit 0, only pre-existing advisories) with both lang files; AC4 proves every component carries its English on itself, which is the locale-unset rendering 431 keys × 2

Harness / PackTest audit — measured, not assumed

displayNameOf (harness/src/executor.ts) reads an entity's custom name to prefer among
same-type candidates. A translate component has no text field, so the old reader would have
fallen through to toString() — whose result depends on whether the installed prismarine-chat
resolves an unknown key to its fallback or renders the raw key. It now reads fallback
explicitly and first
: that is by construction the English the plan's actors[].name carries,
so the heuristic keeps matching instead of depending on a library detail. Pinned by a new unit
test including the "toString returns the raw key" case.

Per the spec this is measured, not assumed: findActorBody counts every
candidate-preference decision, how many had a usable name, and the candidate totals, and the run
report emits

"name_preference": { "decisions": n, "with_usable_name": n, "candidates": n, "named_candidates": n, "unbound": false }

always — including the all-zero shape with unbound: true, which is a finding, not a pass. A
drop to with_usable_name: 0 with non-zero decisions is the specific regression to watch.
This PR does not run the bot ladder (tier 3, release candidates only), so no live decision
count exists yet; the field is what makes the next run answer the question rather than assume it.

PackTest: 111 assert lines across 41 generated tests on the island — zero assert on
rendered text
(assert lines mentioning translate/fallback: 0). The only text component in a
generated PackTest is a setup line (collect_container.mcfunction pre-loads the stack the
objective counts), emitted by the same helper the datapack uses so the two cannot drift; and
adjudication matches on item id, never on name.

Expected scope: every campaign's bytes change

Literals become components, so every campaign's emitted bytes move. That is the change, not a
regression. What is proven unchanged is the property that matters: the double-build
byte-identity gate passes
, on keep-trial in CI and on nobodys-cave-island by hand (609 files,
diff -r clean). Two smaller intended shifts: a --lang bake now ships no
resourcepack.zip/SKINS.md (there is nothing to select between), and every l10n/<code>.json
is now a build input of every build and is hashed into manifest.json — the sidecar's bytes ship
in the pack, so they are as much an input as a stage document.

The reds each new gate was watched producing

A gate never watched failing is not known to work.

  • DW0185, on real emitters. First run over hello-world: 13 files — the campaign-complete
    advancement description, three dialog labels, the completion tellraw, a kit item's
    custom_name, the NPC summon's CustomName, five PackTest descriptions and render-plan.json.
    keep-trial then produced 4 more, including three false hits on prefab .nbt payloads — which
    is how the scan learned to classify verbatim binary outputs instead of guessing at
    from_utf8_lossy. All 17 are sites this PR fixed; the count is now 0.
  • DW0185, on a synthetic tree (dw0185_catches_an_authored_string_emitted_as_a_literal):
    a clean tree passes, one un-lowered tagged string fails naming key + artifact, and an
    unclassified non-UTF-8 output fails rather than skipping the scan. Driven against the checker
    directly, because a red that needs a defective emitter to exist is a red nobody can re-run.
  • DW0183 — watched red twice: an authored world.title carrying U+E000, and a zh-cn.json
    value carrying U+E001. Both exit 1.
  • DW0184 — watched red on languages: ["kl-ingon"] (exit 1, names the code and the table),
    and confirmed not firing for pt-br, which is mapped.
  • Lang-file hole — removing area.keep.name from the sidecar goes red as DW0180 at
    validate. Which means the emit-time key-equality re-proof is unreachable from the CLI: it is
    defence-in-depth for library callers of emit::build, the same standing this file's other
    "callable without validation" checks have. Stated rather than claimed as coverage.
  • Existing tests that went red, and why — 5, each an intended shape change, none weakened:
    marker_name_fields_never_leak_a_raw_id (an untagged CustomName keeps the bare quoted-string
    form; the compound is only for tagged strings); the_rest_dialog_offers_exactly_two_options
    and two v08_option_tooltip tests (dialog labels/tooltips are components now, so a fixture's
    compiler-baked English reads ["label"]["text"]); lang_build_localizes_only_strings…, which
    now asserts positively that the default build ships the language carrier and a --lang bake
    does not. call_graph_integrity went red because keep-trial's build now legitimately requires
    its sidecar among the inputs — fixed by passing real inputs, never by relaxing the requirement.

Two findings surfaced by the binding counts (neither a blocker, neither fixed here)

1. Compiler chrome has no l10n key at all

Walking the island's emitted datapack for surviving "text" literals leaves exactly the
machine marker channel plus 12 compiler-owned player-visible strings. Five of them already
have an author override that translates when authored — world.boundary.message, a
close-gate's sealed_hint, and a bonfire's prompt/rest_label/save_label. Eight have
no override and no key, so they are English in every language
: New objective: ,
Objective complete: , — complete., A Delvewright delve., Delve Complete,
Choose your class, Pick the kit you will carry., Waiting for the party — .

This is not a regression — a --lang zh-cn bake left them English too, since they never
pass through a Campaign field and so were never in the inventory. But v2 makes shipping every
language the normal case, so a Chinese player now reads Chinese content wrapped in English
chrome, and that is newly conspicuous rather than newly broken. The fix is cheap now that the
delivery exists (compiler-owned keys under a reserved prefix, with translations the compiler
ships in every lang file), but it adds keys no campaign sidecar covers — an l10n-surface
decision, so it is flagged, not taken.

2. Twelve inventoried keys reach no player

Island: 419 of 431 inventoried keys are emitted as components. 12 never reach a player at
all
area.island.name, area.open-sea.name, and all 10 quest.<q>.goals. Area names are
consumed only by render-plan.json; a quest goal is emitted only as the campaign-complete
description, and only when world.outro is unauthored (the island authors one). This is
pre-existing — under v1 those strings were swapped into a campaign that never emitted them
either — but it means a translator is asked for 12 strings no player reads. Trimming the
inventory changes every campaign's sidecar coverage, so it is an owner-facing decision, not a
drive-by.

Out of scope, needs a content-repo PR

release.yml's lang step — the one reading languages[0] as "the primary authored language"
and passing --lang — lives in delvewright-campaigns, not here. Deleting it (spec-0029
Consequences) and re-releasing the island as v1.1 is a separate PR in that repo. This engine is
ready for it: delvec build with no --lang is now the release build.

Residual risk identified, deliberately not acted on

A % in a player-visible string is a format specifier inside a translate component, where it
was an ordinary character inside {"text":…}. Zero occurrences today — checked across all 4
campaigns and all 5 fixtures. I did not add a gate: the exact degradation path needs a live-client
observation I cannot make offline, and inventing a diagnostic on speculation is unrequested
change. Proposal, one line, awaiting a call: add a validate-tier warning when a player-visible
string or translation contains %, telling the author to write %%.

Where the fallback path is already exercised

validation/ never sets RESOURCE_PACK, and the emitted server/server.properties carries no
resource-pack lines — so the whole validation ladder already runs with no pack at all. Every
green tier-2 run is therefore a run of exactly the AC8 scenario: no lang files, every string
rendered from the fallback on its own component. The lang files themselves are proven by the
unit/CLI tier, which is where they belong.

CI

All ten required jobs green on this branch, including tier 2 (datapack load + the whole
generated PackTest suite)
on a live pinned server and the mecha cross-check, which
independently re-validates every emitted mcfunction — i.e. the new JSON and SNBT component forms
parse for an implementation that is not this compiler.

cargo fmt --check, clippy -D warnings, cargo test --workspace green; harness typecheck
green, 377 harness tests green; check-dw-codes.py (210 codes, both directions),
check-doc-dupes.py, check-required-contexts.py, check-skill-version.py,
check-storybook-version.py, check-harness-dsl-version.py, check-compose-isolation.py,
check-shell-pipe-shortcircuit.py, check-versions.sh, check-world-settings.sh all green
locally. docs/reference/compiler.md (delivery section, named-exclusion table,
DW0183/DW0184/DW0185), docs/reference/i18n.md, docs/reference/tools.md and the
/new-delve skill are updated in this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AjQ5p1Kv5MrkGPumi7yXWL

stellarfeline and others added 3 commits August 5, 2026 23:47
…e client picks (spec-0029)

`nobodys-cave-island v1.0.0` shipped Chinese-only: `build --lang zh-cn` swaps
every player-visible string before compiling, so the datapack carries exactly one
language and an English player reading it has no fallback path at all.

Emission now lowers every authored player-visible string into
`{"translate": "<l10n key>", "fallback": "<English source>"}` and writes one
`assets/delvewright/lang/<mc_code>.json` per declared language — plus `en_us.json`
— into the resource pack the release already ships. The client auto-selects its
own locale; an unshipped locale, a missed key, and a player who DECLINES the pack
all resolve through the fallback riding the component itself.

The key set is the existing l10n inventory: `each_string` stays the single
authority over what is translatable, so measured and translated cannot drift.

`DW0185` makes spec-0029's central risk an invariant instead of an audit: each
inventoried string enters emission carrying its key in a reserved private-use tag,
an emitter either lowers it through a component helper or reads it through
`l10n::plain`, and a tag still in the finished tree is a site that did neither.
Feature-blind and last, so it guards emitters not yet written.

`--lang` survives unchanged as the single-language bake. No DSL change, no
`dsl_version` bump: emission only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AjQ5p1Kv5MrkGPumi7yXWL
`emit_narrate` no longer case-folds an `art` narrate on its way into the title
command — a `{"translate": …}` component resolves in the client, long after the
compiler could uppercase anything — so the fold moved into the font: a second
bitmap provider addresses the same atlas cells by their lowercase letters.

Nothing downstream looks at a texture, so losing that provider is a silent
VISUAL regression: a translated or lowercase-authored art banner renders as
missing-glyph boxes and every gate stays green. Watched red by deleting the
provider (`uppercase + lowercase over one atlas`); binding count asserted at all
26 letters, not a subset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AjQ5p1Kv5MrkGPumi7yXWL
…osts (spec-0029)

The resource pack stops being optional dressing and becomes the language carrier,
so `SKINS.md` — the note the packaging task reads while wiring `RESOURCE_PACK` —
now names the languages the delve ships and states plainly that a player who
declines the prompt reads English and can still finish the delve.

Host-facing prose only: which languages, what declining does. No key scheme, no
pipeline (CLAUDE.md audience separation).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AjQ5p1Kv5MrkGPumi7yXWL
@stellarfeline

Copy link
Copy Markdown
Owner Author

Planner review: approved on content, HELD for the owner's playtest batch (engine emission — every campaign's bytes move, so this is hers to clear, not mine).

The thing that makes this PR good

You did not close the central risk the way the spec described it. The spec said "enumerate every emission site and prove each one lands in a component" — and you correctly refused that, because an enumerated site list rots the moment someone adds a site. That is the exact defect class this repo has paid for repeatedly: hand-rolled walks that named 3 of 5 effect roots and were right until they weren't.

Instead the key travels in-band with the string, and an emitter either lowers it or explicitly reads it as a named exclusion. A surviving tag in the finished tree is DW0185, feature-blind, run last, and total. So a new emission site added next year cannot silently ship an untranslated literal — it will carry a tag and fail. The invariant holds without anyone remembering it. That is a strictly better answer than the one I commissioned, and the reasoning is in the code where the next person will find it.

The proof that it works: 17 real sites on first run, and the three false hits on prefab .nbt are the good kind of failure — they taught the scan to classify binary outputs instead of guessing, which is a stronger scan than one that never met a binary.

609 files byte-identical on a double build, 682 translate components / 0 authored literals on the island, and key sets equal across languages by construction rather than by test — all the right shapes.

Judgment calls I agree with

  • fallback on the component, not the pack. A declined pack has no lang files at all; the owner approved this reasoning explicitly.
  • % left alone. Zero occurrences across every campaign and fixture, so a diagnostic today would bind to nothing — which is the vacuity failure, not caution. Proposing it and stopping was right.
  • The art font gaining a lowercase provider rather than uppercasing at emit time. A component resolves client-side, so emit-time casing was never going to survive translation. Identical pixels, now reachable — that is the no-hack answer rather than a workaround.
  • Not running the bot ladder and saying so, instead of implying a count you do not have.

The three calls, answered

  1. Content lang step — mine, and it must land after this merges, not before. Deleting it first would leave releases building English-only on an engine that still bakes one language: not broken, but silently wrong in a way nobody would notice. Tracked.
  2. Compiler chrome (8 untranslatable strings) — a real finding and correctly out of scope here. Chinese content inside English chrome is exactly the kind of seam a player sees and we stop seeing. Filed separately; it needs an owner call on whether chrome gets an l10n surface at all.
  3. 12 of 431 island keys reach no player — filed. Translators paying for strings nobody reads is a real cost, but trimming the inventory changes what is demanded of a translator, so it is an l10n-surface decision and not a cleanup.

Why it is held rather than merged

Every campaign's emitted bytes change. That is expected and you showed determinism holds — but "the bytes are deterministic" is not "the delve reads correctly to a person in the language they chose." Only she can clear that, and this pairs naturally with the island v1.1 release it exists to enable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant