Run compiler in-process for the sbt 2.0 / Scala 3 plugin

[russwyte]

Why

The sbt and Mill plugins previously embedded a 24 MB marklit-cli fat jar and drove it as a subprocess — or a long-lived JSON-RPC daemon — purely because a Scala 2.12 / sbt 1.x plugin couldn't load the Scala 3 compiler as a library. sbt 2.0 plugins compile against Scala 3, which removes that barrier: the plugin can depend on marklit-compiler and call it in-process.

Net: −1797 / +479 lines — the entire subprocess/daemon/JSON-RPC layer is gone.

What changed

Shared facade (Phase A)

• New MarklitRun facade in the compiler module holds the multi-file orchestration (parse directives → resolve deps once → per-major classpaths → process → render/write) as side-effect-free code returning structured results. Compile failures are data; only file/parse/resolution errors fail the effect.
• The CLI's runMarklit becomes a thin shell over it (the Daemon contract is preserved). New MarklitError.ResolutionError + MarklitRunSpec.

Publishing (Phase B)

• Bumped to Scala 3.8.4. marklit-compiler-api, marklit-core, marklit-compiler are now published. The two compiler-shim jars ride as Compile resources inside marklit-compiler, so CompilerFactory resolves them off the classpath wherever the jar lands. CLI stays unpublished (its fat jar still bundles the shims transitively).

sbt 2.0 plugin (Phase C)

• sbt-marklit is now an sbt 2.0 / Scala 3 AutoPlugin published as sbt-marklit_sbt2_3, depending on marklit-compiler.
• MarklitSession holds one warm CompilerFactory + ZIO runtime for the sbt session (replaces the daemon); MarklitInProcess serializes execution and bridges to sbt logging/failure.
• sbt 2.0 specifics handled: fileConverter for the new virtual-file classpath, Def.uncached for the side-effecting tasks, and the new target/out/jvm/scala-<ver>/ layout in cross-build classpath discovery.
• Added a scripted test (marklit/basic): generate writes output; a broken block makes marklitCompile fail.

Mill plugin (Phase D)

• Same in-process treatment — warm CompilerFactory in a Task.Worker, depending on marklit-compiler as an mvnDep. Built and tested locally; not published this round.

Docs

• README + mill-plugin/README.md + examples/mill/README.md updated: removed daemon/JSON-RPC/CLI-bundling prose, fixed coordinates (sbt-marklit_sbt2_3), updated the modules table, performance section, and build-from-source steps.

Verification

• compiler + cli test suites green (63 + 16 tests, incl. MarklitRunSpec and the existing MarklitSpec).
• plugin/scripted marklit/basic passes.
• examples/sbt end-to-end: all 7 files (incl. cross-version 2.13/3.x blocks) render correctly; warm factory drops a 23-block file from ~8s to ~45ms on re-check.
• examples/mill end-to-end on JDK 25: docs.marklitGenerate / marklitCheck / pageDocs.marklitGenerate all pass with the same cross-version output.

Notes for reviewers

• The release workflow needs no change — sbt release now publishes all four artifacts (alias updated in build.sbt).
• marklit artifacts are JDK 25 bytecode; Mill 1.1.x bundles JDK 21, so the Mill examples must run on a JDK 25 (JAVA_HOME). Documented in examples/mill/README.md.
• Example build files pin 0.1.0-LOCAL to match the local publishLocal workflow; docs reference 0.1.0 (the forthcoming release).