CROSS JOIN aliasing issue with certain Jobs #11
Labels
bug
Something isn't working
good first issue
Good for newcomers
mitigated
This issue is currently mitigated and a temp fix is in place.
What:
Certain algorithms operating on CROSS JOINS, e.g. in N-Body Problem sample, break under parallelization (i.e. when being run as a Job instead of a For) due to write destination aliasing.
Prelim Fix: (todo in 0.5.x)
Job can throw before operating over Stream Types with Wildcards.
This is trivial to implement in O(1) - in
Query<>.Job>
/Stream<>.Job()
, check if any Stream Type is a wildcard and boom. 💣This can even be paired with the below "wait for iteration WorkItems to finish" for a great halfway fix; the proper fix however mitigates the impact of Wildcards if the Archetype actually doesn't match them more than once - which is very common.
Proper Fix: (circa 0.7.x)
Join
can likely report not only that is it empty or populated, but also whether it has repeat enumerations of the same storages (a simple bool - I think this can be calculated in advance for the entire join before the firstSelect
/Iterate()
)Then,
Query.Job
(orStream.Job
) needs to not queue new WorkItems for the next Iteration before all previous Items have completed (but it can still queue all chunks of each individual iteration at once, because there's no possibility of aliasing between WorkItems in the sameSelect
!)The problem arises when
Join.Iterate()
basically swaps Storages around for repeat enumeration, and many patterns ofN:N
enumeration can be problematic here if multi-threaded.Related:
This may or may not become a different / simpler issue when other write aliasing protections are in place (e.g. Microscheduler for multiple parallel Jobs)
The text was updated successfully, but these errors were encountered: