Skip to content

using skims with find_nearest_accessibility_zone is a bottleneck when working with large number of MAZs #1030

@bricegnichols

Description

@bricegnichols

Describe the bug
The annotate_households step takes a very long time to complete (~25 minutes without sharrow in our implementation). The slowdown occurs after household data has been processed and disaggregate accessibilities are being calculated. More specifically, the nearest_skim function in find_nearest_accessibility_zone in disaggregate_accessibility.py is the bottleneck.

To Reproduce
Steps to reproduce the behavior:

  1. Copy input data from here and configs from here. (Note: if you want to truly benchmark the runtime slowness I will upload the full data instead of the test data linked here.)
  2. Clone Activitysim
  3. Open a command prompt, cd to Activitysim, then enter uv sync --locked to create the uv .venv.
  4. Enter .venv\Scripts\activate to activate the uv environment.
  5. Enter activitysim run -c path_to_configs -d path_to_data_test -o path_to_output

Expected behavior
This step takes about 2 minutes running with sharrow and I would expect it to run maybe a little slower (but way faster than 25 minutes) without sharrow.

Additional context
David Hensle worked with us to diagnose the problem and proposed a new method that works much faster. We also tested the centroids alternative method but did not yet get it working. However, the skims approach seems preferable to using MAZ centroids. Our implementation has 66k MAZs so this may not have surfaced as an obvious issue for implementations with fewer MAZs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugSomething isn't working/bug f

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions