ltm: link-score generation is quadratic in an arrayed target's element count ((N+1)^2 equation arms per edge)

[bpowers]

Summary

For a per-element (Ast::Arrayed) target whose N element equations each reference a different element of an arrayed source, db::model_ltm_variables emits N+1 link-score variables, each an arrayed equation with N+1 arms -- i.e. (N+1)^2 generated equation arms for a single causal edge. Each arm is an independently generated ceteris-paribus partial (parse + wrap + print), so wall-clock generation time is quadratic in the target's element count.

Most of that work is redundant: for element e's score, N-1 of the N arms are trivial zeros.

Measured

Release build, on branch roundtrips-track-a. Fixture: dimension Wide with N elements; growth[Wide] a per-element Equation::Arrayed flow whose element e equation is pop[e] * rate * 0.01; pop[Wide] an array stock fed by growth; rate a scalar aux. LTM enabled. Timing db::model_ltm_variables:

N	model_ltm_variables	link_score vars	total equation arms
50	37.3 ms	51	2,551
100	140.9 ms	101	10,101
200	568.0 ms	201	40,201
400	2.304 s	401	160,401

Clean 4x per doubling; arm count is exactly (N+1)^2.

Sub-phase timings show the salsa queries the emission reads are all effectively linear and negligible. At N=400:

• model_ltm_reference_sites: 5.2 ms
• model_element_causal_edges: 1.16 ms
• model_loop_circuits_tiered: 0.81 ms
• total: ~7 ms against 2304 ms

So the cost is in the emission body itself -- not in reference-site classification, not in element causal edges, and not in loop enumeration.

Why each score is arrayed over the whole target

A FixedIndex reference site (pop[e], a literal element) is emitted as one link-score variable per distinct referenced source element, and each such score is shaped over the target's dimensions via build_arrayed_link_score_equation (src/simlin-engine/src/db/ltm/link_scores.rs) -- holding pop[e] live in the slots that reference it, and a trivial-zero guard form in every other slot. With N distinct source elements over an N-slot target, that is N x N arms. The per-slot guard forms are the redundant bulk.

Why it matters

Developer/agent experience and scale. This is generation-side cost paid on every LTM-enabled compile of an arrayed model with this (very ordinary) shape: per-element equations indexing an arrayed source. A 400-element dimension costs 2.3s in a release build for one edge; real models have many such edges and debug builds are far worse. The generated arms also inflate the synthetic-variable count downstream, which compounds the per-fragment overhead in #655 and pushes toward the VM slot ceiling tracked in #654.

None of the redundant work is intrinsic: N-1 of every score's N arms are constant zeros.

Components affected

• src/simlin-engine/src/db/ltm/link_scores.rs (build_arrayed_link_score_equation, emit_per_element_link_scores)
• src/simlin-engine/src/db/ltm/equation.rs (LtmEquation::Arrayed, has_except_default / default-equation machinery)
• src/simlin-engine/src/db.rs (model_ltm_variables)

Possible directions (not investigated in depth)

• Share one guard-form arm instead of materializing N-1 zero partials per score. The has_except_default / default-equation machinery on LtmEquation::Arrayed may already be able to express "this slot, else zero" without N explicit arms.
• Emit per-(source element, target element) SCALAR scores for this shape, the way emit_per_element_link_scores already does for RefShape::PerElement, rather than N arrayed scores each spanning the whole target.
• Either way, add a scaling guard so a regression is caught.

Reproduction harness

db::ltm_tests::per_element_generation_scaling in src/simlin-engine/src/db/ltm_tests.rs, #[ignore]d:

cargo test -p simlin-engine --release --lib -- --ignored --nocapture per_element_generation_scaling

It prints the per-width timings, the sub-phase breakdown, and the score/arm counts above. (Lands with branch roundtrips-track-a.)

Note on gating: a default-suite regression gate must use a small N or assert on structure (arm count) rather than wall-clock, because of the repo's 3-minute cargo test --workspace cap -- see the test-time budgets section of docs/dev/rust.md.

Scope / provenance

Pre-existing; not introduced by roundtrips-track-a. That branch changed how the ceteris-paribus wrap obtains each slot's occurrence classification, not the emission shape. A genuinely quadratic occurrence lookup was fixed on that branch (OccurrenceLookup::for_slot rescanned the whole occurrence stream per slot; now pre-indexed by slot via SlotOccurrences) -- removing that asymptotic term moved total time by only ~2%, which is how this larger, dominating quadratic was found.

Relationship to existing issues

• Distinct from ltm: C-LEARN LTM compile-time dominated by per-fragment fixed overhead (DimensionsContext clone, O(D^2) intern_name, repeated parse) #655 (LTM fragment-compile per-fragment fixed overhead: DimensionsContext clone, O(D^2) intern_name, repeated parse). ltm: C-LEARN LTM compile-time dominated by per-fragment fixed overhead (DimensionsContext clone, O(D^2) intern_name, repeated parse) #655 is about the constant cost paid per emitted fragment; this issue is about the number of emitted fragments/arms being quadratic. They multiply, and fixing ltm: C-LEARN LTM compile-time dominated by per-fragment fixed overhead (DimensionsContext clone, O(D^2) intern_name, repeated parse) #655 does not fix (N+1)^2.
• Distinct from engine: LTM element-level circuit enumeration spends most of its time in string canonicalization/rotation churn (cross-product shapes) #955 (element-level circuit enumeration string churn). Loop enumeration measures at 0.81 ms of 2304 ms here, so it is not the cost being described.
• Related to ltm: residual ~30k synthesized PREVIOUS helper-aux vars on C-LEARN threaten the VM 65k-slot ceiling #654 (residual synthesized helper-aux count vs the VM 65k-slot ceiling) -- quadratic arm emission is one of the generators of that pressure.
• Cross-reference ltm: replace string-backed synthetic equations with typed IR #965 (replace string-backed synthetic equations with typed IR): the typed-IR representation is the natural place to express a shared default/zero arm rather than N materialized ones.
• Part of the LTM epic epic: Loops that Matter (LTM) -- tracking known issues and improvements #488 (Performance & scale).

Discovered

Measured while working on branch roundtrips-track-a (LTM typed-equation round-trips), after fixing the smaller OccurrenceLookup::for_slot quadratic revealed this as the dominant term.