Skip to content

Remove US-data Medicaid cost allocator #1152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MaxGhenis merged 2 commits into from
May 29, 2026
Merged

Remove US-data Medicaid cost allocator #1152

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
547987d
Remove US-data Medicaid cost allocator
daphnehanse11 May 28, 2026
6863284
Pin PE-US Medicaid cost release
MaxGhenis May 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/remove-medicaid-cost-allocator.changed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Removed the duplicate Medicaid conditional-cost allocator from US data so PolicyEngine US owns Medicaid cost logic.
16 changes: 0 additions & 16 deletions docs/engineering/pipeline-map.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1188,22 +1188,6 @@ class USGeographyPostProcessorResult

Payload after US geography fields are applied.

### `policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessor`

```python
class USMedicaidCostPostProcessor
```

Preserve source Medicaid conditional costs after local H5 transforms.

### `policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessorResult`

```python
class USMedicaidCostPostProcessorResult
```

Payload after conditional Medicaid cost fields are applied.

### `policyengine_us_data.build_outputs.us_augmentations.USTakeupPostProcessor`

```python
Expand Down
102 changes: 24 additions & 78 deletions docs/generated/pipeline_api.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
"docstring": "\"Add auto loan balance, interest and net_worth variable.",
"id": "add_auto_loan",
"kind": "function",
"line": 3063,
"line": 3058,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_auto_loan_interest_and_net_worth"
Expand All @@ -88,7 +88,7 @@
"docstring": "Populate household-level geography variables used by PolicyEngine US.\n\nArgs:\n cps: Output CPS H5 group receiving derived household variables.\n household: Raw CPS household table.",
"id": "add_household_variables",
"kind": "function",
"line": 1656,
"line": 1651,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_household_variables"
Expand All @@ -115,7 +115,7 @@
"docstring": "Add basic ID and weight variables.\n\nArgs:\n cps (h5py.File): The CPS dataset file.\n person (DataFrame): The person table of the ASEC.\n tax_unit (DataFrame): The tax unit table created from the person table\n of the ASEC.\n family (DataFrame): The family table of the ASEC.\n spm_unit (DataFrame): The SPM unit table created from the person table\n of the ASEC.\n household (DataFrame): The household table of the ASEC.",
"id": "add_id_variables",
"kind": "function",
"line": 1049,
"line": 1044,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_id_variables"
Expand All @@ -142,7 +142,7 @@
"docstring": "Impute ORG-derived labor-market inputs and derive overtime premium.",
"id": "add_org_inputs",
"kind": "function",
"line": 2963,
"line": 2958,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_org_labor_market_inputs"
Expand All @@ -169,7 +169,7 @@
"docstring": "Add income variables.\n\nArgs:\n cps (h5py.File): The CPS dataset file.\n person (DataFrame): The CPS person table.\n year (int): The CPS year",
"id": "add_personal_income_variables",
"kind": "function",
"line": 1348,
"line": 1343,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_personal_income_variables"
Expand All @@ -196,7 +196,7 @@
"docstring": "Add personal demographic variables.\n\nArgs:\n cps (h5py.File): The CPS dataset file.\n person (DataFrame): The CPS person table.",
"id": "add_personal_variables",
"kind": "function",
"line": 1111,
"line": 1106,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_personal_variables"
Expand All @@ -223,7 +223,7 @@
"docstring": "",
"id": "add_previous_year_income",
"kind": "function",
"line": 1698,
"line": 1693,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_previous_year_income"
Expand All @@ -250,7 +250,7 @@
"docstring": "",
"id": "add_rent",
"kind": "function",
"line": 422,
"line": 417,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_rent"
Expand All @@ -277,7 +277,7 @@
"docstring": "",
"id": "add_spm_variables",
"kind": "function",
"line": 1617,
"line": 1612,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_spm_variables"
Expand All @@ -304,7 +304,7 @@
"docstring": "Assign SSN card type using PRCITSHP, employment status, and ASEC-UA conditions.\nCodes:\n- 0: \"NONE\" - Likely undocumented immigrants\n- 1: \"CITIZEN\" - US citizens (born or naturalized)\n- 2: \"NON_CITIZEN_VALID_EAD\" - Non-citizens with work/study authorization\n- 3: \"OTHER_NON_CITIZEN\" - Non-citizens with indicators of legal status",
"id": "add_ssn_card_type",
"kind": "function",
"line": 1804,
"line": 1799,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_ssn_card_type"
Expand All @@ -331,7 +331,7 @@
"docstring": "",
"id": "add_takeup",
"kind": "function",
"line": 570,
"line": 565,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_takeup"
Expand All @@ -358,7 +358,7 @@
"docstring": "",
"id": "add_tips",
"kind": "function",
"line": 2703,
"line": 2698,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.add_tips"
Expand Down Expand Up @@ -815,7 +815,7 @@
"docstring": "Replace clone-half person-level feature variables with donor matches.",
"id": "clone_features",
"kind": "function",
"line": 603,
"line": 600,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.extended_cps._splice_clone_feature_predictions"
Expand Down Expand Up @@ -878,7 +878,7 @@
"docstring": "Assert that final exported variables are leaf inputs.",
"id": "computed_export_contract",
"kind": "function",
"line": 1782,
"line": 1775,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.extended_cps.ExtendedCPS._assert_no_computed_variables_exported"
Expand Down Expand Up @@ -972,7 +972,7 @@
"docstring": "Second-stage QRF: train on CPS, predict for PUF clones.\n\nFor the PUF clone half of the extended CPS we need plausible values\nof CPS-only variables (retirement distributions, transfers, hours,\nSPM components, etc.) that are consistent with the clone's\nPUF-imputed income -- not just naively copied from the CPS donor.\n\nWe train a QRF on CPS person-level data where:\n * predictors = demographics + key income variables\n * outputs = CPS-only variables listed in\n ``CPS_ONLY_IMPUTED_VARIABLES``\n\nFor PUF clone prediction we use the PUF-imputed income values\nfrom the second half of ``data`` (the clone half, which already\nhas PUF-imputed income from stage 1).\n\nUses ``fit_predict()`` with ``max_train_samples`` instead of\nmanual sampling + separate fit/predict.\n\nArgs:\n data: Extended dataset dict after ``puf_clone_dataset()`` --\n already doubled, with PUF-imputed income in the second half.\n time_period: Tax year.\n dataset_path: Path to the CPS h5 file for Microsimulation.\n\nReturns:\n DataFrame with one column per CPS-only variable, containing\n predicted values for the PUF clone half (person-level).",
"id": "cps_only",
"kind": "function",
"line": 642,
"line": 639,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.extended_cps._impute_cps_only_variables"
Expand Down Expand Up @@ -1064,7 +1064,7 @@
"docstring": "Subsample the loaded CPS dataset and preserve downsampled arrays.\n\nArgs:\n frac: Fraction of records to retain.",
"id": "downsample",
"kind": "function",
"line": 389,
"line": 384,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.cps.CPS.downsample"
Expand Down Expand Up @@ -1325,7 +1325,7 @@
"docstring": "Check formula-reconstructed housing assistance before export.\n\nThe final H5 must not export formula outputs such as ``housing_assistance``.\nThis guard verifies that the remaining leaf inputs still make those\nformulas produce nonzero values before the export contract strips or\nrejects computed variables.",
"id": "housing_assistance_microsim_validation",
"kind": "function",
"line": 1552,
"line": 1545,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.extended_cps.ExtendedCPS._validate_housing_assistance_microsimulation"
Expand Down Expand Up @@ -2259,7 +2259,7 @@
"docstring": "Apply US entity IDs and calibrated household weights.",
"id": "local_h5_us_entity_postprocessor",
"kind": "class",
"line": 181,
"line": 151,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USEntityPostProcessor"
Expand All @@ -2286,7 +2286,7 @@
"docstring": "Payload after US entity ID and household-weight fields are applied.",
"id": "local_h5_us_entity_postprocessor_result",
"kind": "class",
"line": 75,
"line": 71,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USEntityPostProcessorResult"
Expand All @@ -2313,7 +2313,7 @@
"docstring": "Apply block-derived US geography overrides.",
"id": "local_h5_us_geography_postprocessor",
"kind": "class",
"line": 234,
"line": 204,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USGeographyPostProcessor"
Expand All @@ -2340,7 +2340,7 @@
"docstring": "Payload after US geography fields are applied.",
"id": "local_h5_us_geography_postprocessor_result",
"kind": "class",
"line": 101,
"line": 97,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USGeographyPostProcessorResult"
Expand All @@ -2363,65 +2363,11 @@
"signature": "class USGeographyPostProcessorResult",
"source_file": "policyengine_us_data/build_outputs/us_augmentations.py"
},
"local_h5_us_medicaid_cost_postprocessor": {
"docstring": "Preserve source Medicaid conditional costs after local H5 transforms.",
"id": "local_h5_us_medicaid_cost_postprocessor",
"kind": "class",
"line": 618,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessor"
],
"description": "Preserve Medicaid cost-if-enrolled inputs in local H5 payloads.",
"id": "local_h5_us_medicaid_cost_postprocessor",
"label": "USMedicaidCostPostProcessor",
"node_type": "library",
"pathways": [
"local_h5"
],
"source_file": "policyengine_us_data/build_outputs/us_augmentations.py",
"stability": "moving",
"status": "current",
"validation_commands": [
"uv run pytest tests/unit/build_outputs/test_us_augmentations.py"
]
},
"object_path": "policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessor",
"signature": "class USMedicaidCostPostProcessor",
"source_file": "policyengine_us_data/build_outputs/us_augmentations.py"
},
"local_h5_us_medicaid_cost_postprocessor_result": {
"docstring": "Payload after conditional Medicaid cost fields are applied.",
"id": "local_h5_us_medicaid_cost_postprocessor_result",
"kind": "class",
"line": 155,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessorResult"
],
"description": "US Medicaid conditional-cost local H5 payload data.",
"id": "local_h5_us_medicaid_cost_postprocessor_result",
"label": "USMedicaidCostPostProcessorResult",
"node_type": "library",
"pathways": [
"local_h5"
],
"source_file": "policyengine_us_data/build_outputs/us_augmentations.py",
"stability": "moving",
"status": "current",
"validation_commands": [
"uv run pytest tests/unit/build_outputs/test_us_augmentations.py"
]
},
"object_path": "policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessorResult",
"signature": "class USMedicaidCostPostProcessorResult",
"source_file": "policyengine_us_data/build_outputs/us_augmentations.py"
},
"local_h5_us_takeup_postprocessor": {
"docstring": "Apply US take-up draws after entity and geography postprocessing.",
"id": "local_h5_us_takeup_postprocessor",
"kind": "class",
"line": 339,
"line": 309,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USTakeupPostProcessor"
Expand All @@ -2448,7 +2394,7 @@
"docstring": "Payload after US take-up fields are applied.",
"id": "local_h5_us_takeup_postprocessor_result",
"kind": "class",
"line": 128,
"line": 124,
"metadata": {
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USTakeupPostProcessorResult"
Expand Down Expand Up @@ -3270,7 +3216,7 @@
"docstring": "Replace PUF clone half of CPS-only variables with QRF predictions.\n\nAfter ``puf_clone_dataset()`` the CPS-only variables in the second\nhalf are naive copies of the CPS donor values. This function\nreplaces them with the second-stage QRF predictions that are\nconsistent with the clone's PUF-imputed income.\n\nArgs:\n data: Extended dataset dict (already doubled).\n predictions: DataFrame from ``_impute_cps_only_variables()``.\n time_period: Tax year.\n dataset_path: Path to CPS h5 file for entity mapping.\n\nReturns:\n Modified data dict with CPS-only variables spliced in.",
"id": "qrf_pass2",
"kind": "function",
"line": 1017,
"line": 1014,
"metadata": {
"api_refs": [
"policyengine_us_data.datasets.cps.extended_cps._splice_cps_only_predictions"
Expand Down
40 changes: 2 additions & 38 deletions docs/generated/pipeline_map.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -967,42 +967,6 @@
"uv run pytest tests/unit/build_outputs/test_us_augmentations.py"
]
},
{
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessor"
],
"description": "Preserve Medicaid cost-if-enrolled inputs in local H5 payloads.",
"id": "local_h5_us_medicaid_cost_postprocessor",
"label": "USMedicaidCostPostProcessor",
"node_type": "library",
"pathways": [
"local_h5"
],
"source_file": "policyengine_us_data/build_outputs/us_augmentations.py",
"stability": "moving",
"status": "current",
"validation_commands": [
"uv run pytest tests/unit/build_outputs/test_us_augmentations.py"
]
},
{
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USMedicaidCostPostProcessorResult"
],
"description": "US Medicaid conditional-cost local H5 payload data.",
"id": "local_h5_us_medicaid_cost_postprocessor_result",
"label": "USMedicaidCostPostProcessorResult",
"node_type": "library",
"pathways": [
"local_h5"
],
"source_file": "policyengine_us_data/build_outputs/us_augmentations.py",
"stability": "moving",
"status": "current",
"validation_commands": [
"uv run pytest tests/unit/build_outputs/test_us_augmentations.py"
]
},
{
"api_refs": [
"policyengine_us_data.build_outputs.us_augmentations.USTakeupPostProcessor"
Expand Down Expand Up @@ -2082,9 +2046,9 @@
}
],
"metadata": {
"api_node_count": 100,
"api_node_count": 98,
"canonical_stage_count": 5,
"decorated_object_count": 160,
"decorated_object_count": 158,
"mapped_decorated_node_count": 60,
"stage_count": 17,
"substage_count": 17
Expand Down
Loading
Loading