[Feature Request] Prefix-shard the semantic cache directory (cache/semantic/ab/<hash>.json) — the flat layout puts ~20k entries in one directory at monorepo scale

[gkaganas-mparticle]

Version: verified against current v8 (0.9.8). Env: committed-cache
workflow (semantic cache in git so fresh worktrees/CI warm-start), monorepo
corpus that produces ~20k semantic cache entries.

Problem

The semantic cache layout is flat — every entry is
cache/semantic/{sha256}.json in a single directory. On v8 today:

• load_cached — entry = cache_dir(root, kind) / f"{h}.json"
  (cache.py:364)
• save_cached — entry = target_dir / f"{h}.json" (cache.py:409)
• cached_files / clear_cache — the kind/pattern table uses
  ("ast", "**/*.json"), ("semantic", "*.json") (cache.py:443–445,
  457–460): AST is already globbed recursively (it has per-version
  subdirs), semantic is non-recursive by construction.
• prune_semantic_cache — semantic_dir.glob("*.json") (cache.py:491),
  also non-recursive.

That's fine at hundreds of entries. At monorepo scale it degrades on both
axes we care about:

• Filesystem: our corpus seeds ~20,000 content-hash entries into the one
  directory. Directory operations (globs, ls, editor/tooling scans, some
  network filesystems) degrade with tens of thousands of siblings.
• Git (the committed-cache case, see [RFC] Native graphify update for the semantic layer — a scripted conductor with a pluggable LLM hook (we measured: the math is 0.89s–52s, the agent-walked workaround is 11–14 min) #1710): a directory is one
  tree object. With ~20k entries the cache/semantic/ tree object is
  ~1.8 MB, and every cache commit rewrites the whole tree — adding one
  entry re-serializes all 20k filenames. Prefix subdirs make the rewritten
  tree proportional to the touched prefix (~80 entries), not the corpus.

The cache being content-hash-keyed makes the fix trivially cheap: the key
already starts with uniformly distributed hex.

Proposal

Two-hex prefix subdirectories, the same scheme git itself uses for loose
objects:

cache/semantic/ab/abcd1234….json     # 256 dirs, ~80 entries each at 20k scale

1. Write path (save_cached): write to cache_dir(...)/h[:2]/f"{h}.json"
   (mkdir the prefix dir on demand).
2. Read path (load_cached): try the sharded path first, fall back to the
   legacy flat path — existing caches keep hitting without a migration step,
   and entries migrate organically as content changes (or via an optional
   one-shot graphify cache-migrate).
3. Enumeration: change the semantic pattern from *.json to **/*.json
   in cached_files / clear_cache (a two-character change in the existing
   kind/pattern table — AST already does exactly this), and switch
   prune_semantic_cache to rglob/**/*.json so pruning sees sharded
   entries. The .tmp atomic-write temporaries stay colocated with their
   entry (same prefix dir), preserving the same-filesystem rename guarantee.
4. AST cache: unaffected (already namespaced under ast/v{N}/; it's
   machine-local and version-swept, so tree-churn pressure doesn't apply).
   Sharding it too would be consistent but isn't needed for this problem.

No key change, no format change, no version bump semantics: the entry's
identity is still the content hash; only its directory changes.

Why we hit this

We commit graphify-out/cache/semantic/ so every fresh clone/worktree/CI job
warm-starts extraction (a warm update re-extracted 1 file where a cold run
extracts 207 — the economics are in #1710). Committing works
beautifully at small scale; the flat directory is the first thing that breaks
when the corpus is a monorepo. We are adopting exactly the design above
downstream (sharded write + flat-fallback read + recursive globs).

Offer

Happy to contribute the PR: the four call-site changes above plus tests
(sharded round-trip, flat-legacy fallback hit, cached_files/clear_cache/
prune_semantic_cache seeing both layouts, atomic-rename same-dir property).
It's deliberately small and independent of the larger update-conductor RFC.