Skip to content

Regenerate TANF data for 2026 (policyengine-us 1.715.3 + Indiana fix) — complete 56/56#10

Merged
hua7450 merged 2 commits into
mainfrom
update-data-2026
May 30, 2026
Merged

Regenerate TANF data for 2026 (policyengine-us 1.715.3 + Indiana fix) — complete 56/56#10
hua7450 merged 2 commits into
mainfrom
update-data-2026

Conversation

@hua7450
Copy link
Copy Markdown
Collaborator

@hua7450 hua7450 commented May 30, 2026

Summary

Regenerates all 56 precomputed TANF data files from policyengine-us 1.715.3 (+ the Indiana TANF fix, PR #8543) at tax year 2026, replacing the stale 1.598.0 / 2025 data. Also adds a vectorized data generator that is ~600× faster than the existing one and validated bit-for-bit identical to it.

Of the 51 jurisdictions, 23 produced changed values; the other 28 recomputed to byte-identical data (no diff from the prior committed values).

New: vectorized generator (scripts/precompute_vec.py)

The old generator built 15,376 Simulation objects per state (one per grid cell) — that construction, not the math, was the bottleneck (~25–45 min/state). The new generator builds 16 per state:

  • Household-structure dims (adults × children) stay a 16-iteration loop — they change entity counts, so they can't be axes.
  • Income dims (31 earned × 31 unearned = 961 cells) become two PolicyEngine axes groups → one Simulation computes all 961 cells in a vectorized pass.

Measured: 4.3s vs ~47.6 min on Illinois (~664×). Full 33-state completion run took 74s.

Validated bit-for-bit identical to the cell-by-cell generator on 10 states covering every code path — plain states, person-level monthly special vars (IL), county selection (CA), and SPM-unit annual vars (CA/CO): 0 mismatches across 138,384+ cells. A per-structure fallback to the cell-by-cell path guarantees the run can never emit wrong/missing data. Documented in scripts/README.md.

Notable changed values (vs 1.598.0, $0-income household)

State before after (2026) why
IN 2p 255 / 3p 320 2p 409 / 3p 513 🐛 the fix — was paying the standard of need, now the amount of assistance (IC 12-14-2-5). Exact benchmark match.
KY 3p 524 3p 341 reflects KY's Nov-2025 benefit cut
CT 3p 833 3p 897 2026 COLA (CT indexes annually)
NH 3p 1,366 2026 COLA
HI 3p 763 3p 611 old data was stale/high; now matches benchmark (~610)
KS / IL / AK 403 / 759 / 924 429 / 782 / 925 2026 updates

Benchmark verification (NC / CBPP "maximum benefit")

Compared all 51 jurisdictions, both household sizes (1 adult + 1 child, 1 adult + 2 children), against the external benchmark:

  • 39/51 match exactly (≤ $4 of source rounding, both household sizes).
  • 12 diverge — all explained, none are computation errors:
    • Zero-rent test household (AZ, NY, VT): no rent input → no shelter allowance; adding rent reproduces the benchmark.
    • Benchmark is an older year (CT, NH 2026 COLA; KY Nov-2025 cut) — PE is current.
    • Bundled allowance (MA): PE includes the clothing allowance (~$41.67/mo per person).
    • Region selection (PA, VA): benchmark uses a specific locality; PA Group 1 matches exactly.
    • Idaho flat $309 all sizes is correct; the benchmark's escalating $389 is the error.
    • Rounding (IL, AK, HI, NE, OH, SC): annual÷12 vs single-month, ≤ $5.

No genuine model concerns remain (the previously-flagged Indiana gap is fixed by #8543, which this data already carries).

Generator / metadata changes (scripts/precompute.py, metadata.json)

  • YEAR 2025 → 2026
  • OUTPUT_DIR fixed: ../frontend/public/data../public/data (repo moved Vite → Next; old path no longer exists)
  • FPG dict → 2026 values (CONTIGUOUS 15,960/5,680 · AK 19,950/7,100 · HI 18,360/6,530)
  • stdout line-buffered + a --workers flag
  • metadata.json: year 2026, FPG 2026, version 1.715.3, CA/PA/VA county-gated

Provenance

Generated locally from policyengine-us 1.715.3 + the Indiana fix (PR #8543). Once #8543 is released, a clean re-run on the released version reproduces this output.

🤖 Generated with Claude Code

hua7450 and others added 2 commits May 30, 2026 09:31
…rtial 23/56

- 9 changed state files, incl. Indiana fix: size-2 $255->$409, size-3 $320->$513
  (amount of assistance per IC 12-14-2-5, replacing the standard of need)
- Also updated: HI (763->611), KY (524->341, Nov-2025 cut), CT (833->897),
  KS (403->429), IL (759->782), AK (924->925), CA_1/CA_2 (region cells)
- precompute.py: YEAR=2026, OUTPUT_DIR fix (frontend/->public/), FPG 2026,
  stdout line-buffering, --workers flag
- metadata.json: year 2026, FPG 2026, version 1.715.3
- README: data version note (partial regeneration)

Remaining 33 files still on 1.598.0 — pending follow-up (draft).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Regenerate the remaining 33 state files at tax year 2026 on
policyengine-us 1.715.3 + Indiana TANF fix (#8543), completing the prior
partial 23/56 update. All 56 files now reflect the same model + year.

Add precompute_vec.py: a PolicyEngine-`axes` vectorized generator that
computes a full state in ~2-5s (~600x faster than the cell-by-cell path)
by building 16 Simulations per state (one per adults x children structure)
instead of 15,376. Validated bit-for-bit identical to precompute.py on 10
states covering every code path (plain, person-monthly special vars,
county selection, SPM-unit annual vars) — 0 mismatches across 138,384+
cells. Includes a per-structure fallback to the cell-by-cell path.

Add scripts/README.md documenting both generators, the vectorization
method, and the validation. Update README data-version line to 56/56.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@hua7450 hua7450 changed the title Regenerate TANF data for 2026 (policyengine-us 1.715.3 + Indiana fix) — partial 23/56 Regenerate TANF data for 2026 (policyengine-us 1.715.3 + Indiana fix) — complete 56/56 May 30, 2026
@hua7450 hua7450 marked this pull request as ready for review May 30, 2026 14:26
@hua7450 hua7450 merged commit 3fd6d41 into main May 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant