You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This draft PR addresses only the immediate causes of the US economic_impact_analysis program-statistics breakage observed while following up on Vahid's JOSS PR (#264). It is not intended to solve the broader design problem of country package program variables changing over time; that follow-up is tracked in #326.
Changes:
Keep ProgramStatistics.program_name as the direct model/output variable to aggregate; no separate alias field is introduced.
Replace the broken US program-statistics mappings with currently valid policyengine-us variables:
payroll_tax -> employee_payroll_tax
medicare -> medicare_cost
state_income_tax remains state_income_tax
Note: payroll_tax was configured as a person-level program, while employee_payroll_tax is a tax-unit-level variable. This means the payroll-tax program-statistics row now reports tax-unit recipient/winner/loser counts, using tax-unit weights, rather than person counts.
Add medicare_cost and state_income_tax to the US output variables materialized by .py.
Update US household snapshots for the newly exposed medicare_cost and state_income_tax output keys.
Validate US program-statistics configuration before expensive simulations run.
Add shared error helpers for constructing typed errors and conditional error detail lines, then use them in the US program-statistics validation error path.
Replace bare aggregate next(...) lookup failures with descriptive ValueErrors.
Add mocked US program-statistics, aggregate error, and error-helper tests that would have caught this issue.
Verification:
make format
uv run --python 3.13 --extra dev python -m pytest tests/test_household_calculator_snapshot.py::test_us_household_snapshot tests/test_errors.py tests/test_aggregate.py tests/test_change_aggregate.py tests/test_us_program_statistics.py
uv run --python 3.13 --extra dev pytest tests/test_change_aggregate.py tests/test_us_program_statistics.py
uv run --python 3.13 --extra dev ruff check tests/test_us_program_statistics.py tests/test_change_aggregate.py
Core logic is correct — variable name swaps, validation, and StopIteration → ValueError upgrade all check out. Two things to fix before merge.
Blocking
CI failures are caused by this PR. Adding medicare_cost to entity_variables["person"] makes the household calculator materialize it, and 4 snapshots in test_household_calculator_snapshot.py fail with new key: person[0].medicare_cost=14500.0. Need to regenerate the snapshot files.
Negative test can silently pass on the wrong exception.test_us_program_statistics_config_fails_before_simulation_run uses try/except + manual raise AssertionError. Use pytest.raises(ValueError, match="US program statistics config is invalid") instead.
Worth addressing
ChangeAggregate has no invalid-variable regression test.test_aggregate.py got one for Aggregate, but ChangeAggregate had the same bare next(...) pattern and isn't covered.
Entity change not flagged in the PR description.payroll_tax was entity="person"; employee_payroll_tax is entity="tax_unit". Correct, but it changes recipient counts in the program-statistics table — worth noting.
Verified correct
All three variable replacements match what policyengine-us actually exposes (employee_payroll_tax, medicare_cost, household_state_income_tax).
Validator runs cheap, before ensure() on either simulation.
Test arithmetic: employee_payroll_tax 1600, medicare_cost 300, state_income_tax 900 — all correct under the test weights.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #325
This draft PR addresses only the immediate causes of the US
economic_impact_analysisprogram-statistics breakage observed while following up on Vahid's JOSS PR (#264). It is not intended to solve the broader design problem of country package program variables changing over time; that follow-up is tracked in #326.Changes:
ProgramStatistics.program_nameas the direct model/output variable to aggregate; no separate alias field is introduced.policyengine-usvariables:payroll_tax->employee_payroll_taxmedicare->medicare_coststate_income_taxremainsstate_income_taxpayroll_taxwas configured as a person-level program, whileemployee_payroll_taxis a tax-unit-level variable. This means the payroll-tax program-statistics row now reports tax-unit recipient/winner/loser counts, using tax-unit weights, rather than person counts.medicare_costandstate_income_taxto the US output variables materialized by.py.medicare_costandstate_income_taxoutput keys.next(...)lookup failures with descriptiveValueErrors.Verification:
make formatuv run --python 3.13 --extra dev python -m pytest tests/test_household_calculator_snapshot.py::test_us_household_snapshot tests/test_errors.py tests/test_aggregate.py tests/test_change_aggregate.py tests/test_us_program_statistics.pyuv run --python 3.13 --extra dev pytest tests/test_change_aggregate.py tests/test_us_program_statistics.pyuv run --python 3.13 --extra dev ruff check tests/test_us_program_statistics.py tests/test_change_aggregate.pygit diff --check