Add motionhazard to the outer side of parallel aware join.(fix flaky incorrect results of agg) #284
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For a parallel aware join, we may partition outer data to batches if there is not enough memory.And if a worker has consumed all the data from subnode, it will arrive and wait for others to begin next phase: PHJ_BUILD_DONE.
PHJ_BUILD_DONE means that we has partitioned every thing from PHJ_BUILD_HASHING_OUTER phase and are ready to probe. It's true for Postgres, becuase that if a worker pulls no data from subnode and it leaves PHJ_BUILD_HASHING_OUTER phase, all subnode's data must be processed(ex: a Parallel Seqscan).
As all participants leaves PHJ_BUILD_HASHING_OUTER, the whole data should be pulled completely by these processes, or there may be other siblings who have never participated in phase PHJ_BUILD_HASHING_OUTER at all.
But it's not true for CBDB, a plan node may have Motions behind. Some parallel workers pull no data from subnode, it doesn't mean that other workers have no data to process at all. We need to wait for all parallel workers together instead of build barrier's participants arrived at the current phase before we move to the next phase.
Add an outer_motion_barrier to ensure all parallel workers finish their work of partition outer batches.
Authored-by: Zhang Mingli avamingli@gmail.com
closes: #ISSUE_Number
Change logs
Describe your change clearly, including what problem is being solved or what feature is being added.
If it has some breaking backward or forward compatibility, please clary.
Why are the changes needed?
Describe why the changes are necessary.
Does this PR introduce any user-facing change?
If yes, please clarify the previous behavior and the change this PR proposes.
How was this patch tested?
Please detail how the changes were tested, including manual tests and any relevant unit or integration tests.
Contributor's Checklist
Here are some reminders and checklists before/when submitting your pull request, please check them:
make installcheck
make -C src/test installcheck-cbdb-parallel