Buffer cleanup #455
Open
Buffer cleanup #455
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
Author
This shows the performance with/without IPC enabled on alps. The buffer cleanup branch improves things, turning off ipc is still a major improvement
edit: erratum -the graph says heffte-gpu-enabled, but in fact it is disabled here. |
Member
|
John very nice results! Can you also show this on Lumi and/or on Beverin? |
Member
|
with GPU off you mean: -D Heffte_DISABLE_GPU_AWARE_MPI=OFF ? |
Collaborator
Author
yes. exactly this |
Member
|
@biddisco Could you please pull and merge the latest master as it contains a fix for the Jülich's CI. I can then add the label and check if the unit tests are passing. |
To improve messaging performance, we may wish to replace simple kokkos:views in buffer handling with an alternative implementation that allows for better alignment and caching. Additionally, code may leak communication buffers via direct use of kokkos views that are passed around from send/rec code into/out of fields or attributes. Placing buffers behind an API reduces the chance of this happening.
…copies Always reset read/write pos on buffer when getBuffer is called
…memory Kokkos::views are aligned to 64 byte boundaries, but communications can benefit from stronger alignment. Support user defined (compile time) alignment of higher order and create kokkos views inside manually allocated cuda memory
biddisco
force-pushed
the
buffer-cleanup
branch
2 times, most recently
from
9cdd0f8 to
6f22411
Compare
Member
|
The last commit seams to break the compilation: https://ippl-bc4558.pages.jsc.fz-juelich.de/365538/allure-report/index.html#suites/0a150a27af0a5eb1faedcb6100255f56/782113b35712612a/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains cleanups to buffer management that should improve default performance
This PR also adds support for logging using the spdlog library and the fmt library for clean messages, this can be enabled or disabled at compile time using IPPL_LOG_LEVEL off, trace, debug, warn, error, critical etc etc by default it is off, to use it, one needs to find_package spdlog, fmt which are now provided in uenvs for alps