feat(st): COLI_MODEL_DIRS ΓÇö split model shards across N drives, no duplication (capacity aggregation; composable with #421 mirror)

[mohamedmastouri2000-boop]

What

Per the green light in #467 (finding 4): COLI_MODEL_DIRS=<dir>[;<dir>...] — split the model's .safetensors shards across N drives with no duplication. Complementary to #421's mirror, not a competitor.

Why

The mirror gives 2× read bandwidth at 2× disk cost. The split gives capacity aggregation, which the mirror fundamentally can't: a 400 GB container fits across two smaller drives that individually can't hold it (or three, or N — ST_MAX_SHARDS is the only bound). Demand preads hit whichever drive holds the shard, so concurrent expert loads still parallelise across spindles when routing spreads.

Design

• The dir list acts as a search path with dedup by basename: a shard present in two dirs is taken from the first-listed one (primary COLI_MODEL first). No layout convention imposed — any partition of the shards works; users move files between drives freely.
• Each shard opens its own fd from its own path, exactly as today — the split needs no read-path changes at all: st_open_fd/pread/O_DIRECT twins/PIPE workers all work per-fd already. The whole feature is in indexing.
• Composable with COLI_MODEL_MIRROR (the init-order ask from [Performance]: RTX 5080 / 128 GB resource matrix (drive x compute x RAM) ΓÇö RAM dominates; COLI_CUDA=0 Windows quirk; PR-ready N-drive shard split #467): st_mirror_init runs after indexing and matches per-shard by basename against the merged index, so a mirror dir may hold a copy of any subset of the split's shards — partial mirrors of a split are valid, divergent files are rejected per-file exactly as on a single-dir model. They cannot fight at init: split resolves which file is primary, mirror only ever adds a replica fd to an already-indexed shard.
• Metadata (config.json, tokenizer, .coli_usage, .coli_kv) stays in the primary COLI_MODEL dir only — extra dirs are weights-only, never written.
• st_init(S, dir) remains as a one-line back-compat wrapper over the new st_init_multi(S, dir, extra); every existing call site and test is untouched.
• Startup logs the layout: [SPLIT] +C:\glm_split_b -> 70 shard(s) / [SPLIT] model across 2 dir(s): 142 shard(s) total (primary ... -> 72 shard(s)), no duplication.

Verification (RTX 5080 / Windows 11, g64 container, 142 shards, 399.8 GB)

• 72 even shards on drive 1 + 70 odd shards on drive 2 (hardlink layout, zero duplication): correct indexing, coherent greedy output, zero fallbacks, CUDA_DENSE=1 full residency.
• Decode parity with single-drive at full RAM: 0.92 vs 0.89–0.90 tok/s — as expected there (84% expert hit ⇒ decode barely touches disk). Honest framing per [Performance]: RTX 5080 / 128 GB resource matrix (drive x compute x RAM) ΓÇö RAM dominates; COLI_CUDA=0 Windows quirk; PR-ready N-drive shard split #467: on this box the value is capacity, not speed; a bandwidth-bound host with dedicated data drives is where the parallel-read side should show.
• Full protocol + raw logs in [Performance]: RTX 5080 / 128 GB resource matrix (drive x compute x RAM) ΓÇö RAM dominates; COLI_CUDA=0 Windows quirk; PR-ready N-drive shard split #467.

Scope

+61/−10 over c/st.h (scan/dedup/init-multi), one call site in c/colibri.c (env read at model_init), docs/ENVIRONMENT.md row. No kernel, no read-path, no format changes. Happy to add a test_st_split.c sibling of test_st_mirror.c (dedup priority, partial-dir, missing-dir error) if you want one in-tree — say the word and I'll push it to this branch.