HybridGenerator: Asyncronous + parallel event generation#13788
Merged
sawenzel merged 1 commit intoAliceO2Group:devfrom Dec 12, 2024
Merged
HybridGenerator: Asyncronous + parallel event generation#13788sawenzel merged 1 commit intoAliceO2Group:devfrom
sawenzel merged 1 commit intoAliceO2Group:devfrom
Conversation
Contributor
|
REQUEST FOR PRODUCTION RELEASES: This will add The following labels are available |
sawenzel
force-pushed
the
swenzel/hybrid_async
branch
from
d02acbb to
0c1ff2a
Compare
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe)
sawenzel
force-pushed
the
swenzel/hybrid_async
branch
from
0c1ff2a to
526ae57
Compare
Collaborator
|
AliceO2Group/O2DPG#1845 updated tool for JSON template creation is proposed here related to these modifications. |
Collaborator
|
Error while checking build/O2/fullCI for 526ae57 at 2024-12-12 12:34: Full log here. |
alcaliva
pushed a commit
that referenced
this pull request
Jan 31, 2025
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe) (cherry picked from commit f4d9b9c)
alcaliva
pushed a commit
that referenced
this pull request
Jan 31, 2025
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe) (cherry picked from commit f4d9b9c)
alcaliva
pushed a commit
that referenced
this pull request
Jan 31, 2025
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe) (cherry picked from commit f4d9b9c)
alcaliva
pushed a commit
that referenced
this pull request
Jan 31, 2025
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe) (cherry picked from commit f4d9b9c)
alcaliva
pushed a commit
that referenced
this pull request
Feb 2, 2025
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe) (cherry picked from commit f4d9b9c)
alcaliva
pushed a commit
that referenced
this pull request
Feb 2, 2025
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe) (cherry picked from commit f4d9b9c)
alcaliva
pushed a commit
that referenced
this pull request
Feb 2, 2025
This introduces: * asyncronous event generation * possibility for parallel event generation This is useful for: * hiding latency (IO) of certain generators * decoupling the actual work from the call sequence into HybridGenerator * collaboration from multiple clones of the same generator to generate a certain number of events The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread. An example is added for parallel event gen. In addition, this commit has the following changes: - Small adjustments to seeding of Pythia8 in order to avoid same seeds in multiple parallel Pythia instances. - possibility to init external generator from an INI file (typically done or available in O2DPG) - use shared_ptr instead of unique_ptr to keep generators in order to avoid lifetime problems with async processing - preparion to run underlying generator Init() functions in async way (not active yet; needs checks if generators are thread safe) (cherry picked from commit f4d9b9c)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This introduces:
This is useful for:
The implementation relies on tbb::task_arena and input/output queues for decoupling the task_arena from the HybridGenerator thread.
Small adjustments to seeding of Pythia8 in order
to avoid same seeds in multiple parallel Pythia instances.