Skip to content

Neighborhoods: Support for non-exhausting iteration #2034

@triceo

Description

@triceo

An exhausting iteration is very expensive. For every bi join, we need to keep track of the not-yet-exhausted iterators on both sides; therefore for even mid-sized joins, the memory overhead of that grows beyond what's reasonable.

We want to have iterators which can possibly repeat the value they return. We do not want for the user to have to enable this feature. We think it may have to be enabled automatically in the following situations:

  • When we see that the join sizes are getting out of hand.
  • When the LS performs a lot of moves per step, because that means the joins would otherwise be building more and more of these expensive iterators.
  • For tri-picks and higher, as there we can automatically assume that the joins would be massive.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions