Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openPMD: avoid managed memory #2097

Closed
ax3l opened this issue Jul 16, 2021 · 2 comments · Fixed by #2859
Closed

openPMD: avoid managed memory #2097

ax3l opened this issue Jul 16, 2021 · 2 comments · Fixed by #2859
Assignees
Labels
backend: cuda Specific to CUDA execution (GPUs) backend: hip Specific to ROCm execution (GPUs) backend: sycl Specific to DPC++/SYCL execution (CPUs/GPUs) component: openPMD openPMD I/O Performance optimization workaround

Comments

@ax3l
Copy link
Member

ax3l commented Jul 16, 2021

Make sure that openPMD output can run with amrex.the_arena_is_managed=0 #2009

We need to explicitly copy D2H first, e.g. here
fields:

amrex::Real const * local_data = fab.dataPtr( icomp );
mesh_comp.storeChunk( openPMD::shareRaw(local_data),
chunk_offset, chunk_size );

particles:
curr.get()[i] = aos[i].pos(currDim);

This is how plotfiles do this:
fields: https://github.com/AMReX-Codes/amrex/blob/a0790de872d6b18b563712b9f1b18323b8d00072/Src/Base/AMReX_FArrayBox.cpp#L604
particles: https://github.com/AMReX-Codes/amrex/blob/6e27136f76280e565d30ff654031d1b1ac683171/Src/Particle/AMReX_ParticleIO.H#L608

@ax3l ax3l added backend: cuda Specific to CUDA execution (GPUs) Performance optimization component: openPMD openPMD I/O backend: hip Specific to ROCm execution (GPUs) backend: sycl Specific to DPC++/SYCL execution (CPUs/GPUs) labels Jul 16, 2021
@ax3l ax3l assigned ax3l and guj Jul 16, 2021
@ax3l
Copy link
Member Author

ax3l commented Jul 19, 2021

@ax3l
Copy link
Member Author

ax3l commented Feb 16, 2022

Since we filter the particles in a temporary, pinned memory buffer (#1632): no managed memory usage there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend: cuda Specific to CUDA execution (GPUs) backend: hip Specific to ROCm execution (GPUs) backend: sycl Specific to DPC++/SYCL execution (CPUs/GPUs) component: openPMD openPMD I/O Performance optimization workaround
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants