-
Notifications
You must be signed in to change notification settings - Fork 121
Description
Describe the bug
The annotate_households step takes a very long time to complete (~25 minutes without sharrow in our implementation). The slowdown occurs after household data has been processed and disaggregate accessibilities are being calculated. More specifically, the nearest_skim function in find_nearest_accessibility_zone in disaggregate_accessibility.py is the bottleneck.
To Reproduce
Steps to reproduce the behavior:
- Copy input data from here and configs from here. (Note: if you want to truly benchmark the runtime slowness I will upload the full data instead of the test data linked here.)
- Clone Activitysim
- Open a command prompt, cd to Activitysim, then enter uv sync --locked to create the uv .venv.
- Enter .venv\Scripts\activate to activate the uv environment.
- Enter activitysim run -c path_to_configs -d path_to_data_test -o path_to_output
Expected behavior
This step takes about 2 minutes running with sharrow and I would expect it to run maybe a little slower (but way faster than 25 minutes) without sharrow.
Additional context
David Hensle worked with us to diagnose the problem and proposed a new method that works much faster. We also tested the centroids alternative method but did not yet get it working. However, the skims approach seems preferable to using MAZ centroids. Our implementation has 66k MAZs so this may not have surfaced as an obvious issue for implementations with fewer MAZs.