Skip to content

Bug: Policy reforms not applied in economy-wide simulations #79

@anth-volk

Description

@anth-volk

Summary

Policy reforms are not being applied to economy-wide simulations. Both baseline and reform simulations produce identical results, making all policy impact calculations show zero difference.

Root Cause

The policyengine.core.Simulation class creates the underlying Microsimulation without the reform dict, then attempts to apply the policy after construction. Due to caching in the microsimulation framework, reforms applied after construction have no effect.

The problematic pattern in simulate_economy_us and simulate_economy_uk:

# This doesn't work - reforms applied AFTER Microsimulation construction
pe_sim = PESimulation(
    dataset=pe_dataset,
    tax_benefit_model_version=pe_model_version,
    policy=policy,  # Policy is not properly applied
    dynamic=dynamic,
)
pe_sim.ensure()

Reproduction

Created a test script (scripts/test_economy_simulation.py) that follows the exact code path from modal_app.py:

  1. Creates a small test dataset with 3 households (each with 2 adults + 2 children)
  2. Runs baseline simulation (no policy)
  3. Runs reform simulation (CTC fully refundable)
  4. Compares results

Expected: Different results between baseline and reform
Actual: Identical results - $0 change in CTC and income tax

Fix

Convert PEPolicy objects to reform dict format and pass the reform at Microsimulation construction time:

# Convert PEPolicy to reform dict
policy_reform = _pe_policy_to_reform_dict(policy)
dynamic_reform = _pe_policy_to_reform_dict(dynamic)
reform = _merge_reform_dicts(policy_reform, dynamic_reform)

# Run with reform at construction time
output = _run_us_economy_simulation(pe_dataset, reform, pe_model_version, simulation_id)

The helper functions create Microsimulation with the reform dict directly:

sim = Microsimulation(reform=reform)
# ... build simulation from dataset ...

Affected Functions

  • simulate_economy_us() - US economy simulations
  • simulate_economy_uk() - UK economy simulations

Note: Household calculations (_calculate_household_us and _calculate_household_uk) were fixed separately in a previous commit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions