added env option OPENPMD_HDF5_ALIGNMENT

docs/source/backends/hdf5.rst

@@ -25,6 +25,7 @@ The following environment variables control HDF5 I/O behavior at runtime.
 environment variable                  default   description
 ===================================== ========= ====================================================================================
 ``OPENPMD_HDF5_INDEPENDENT``          ``ON``    Sets the MPI-parallel transfer mode to collective (``OFF``) or independent (``ON``).
+``OPENPMD_HDF5_ALIGNMENT``            ``1``     Tuning parameter for parallel I/O, choose an alignment which is a multiple of the disk block size.
 ``H5_COLL_API_SANITY_CHECK``          unset     Set to ``1`` to perform an ``MPI_Barrier`` inside each meta-data operation.
 ===================================== ========= ====================================================================================
 
@@ -34,6 +35,11 @@ Although we choose the default to be non-collective (independent) for ease of us
 For independent parallel I/O, potentially prefer using a modern version of the MPICH implementation (especially, use ROMIO instead of OpenMPI's ompio implementation).
 Please refer to the `HDF5 manual, function H5Pset_dxpl_mpio <https://support.hdfgroup.org/HDF5/doc/RM/H5P/H5Pset_dxpl_mpio.htm>`_ for more details.
 
+``OPENPMD_HDF5_ALIGNMENT`` This sets the alignment in Bytes for writes via the ``H5Pset_alignment`` function.
+According to the `HDF5 documentation <https://support.hdfgroup.org/HDF5/doc/RM/H5P/H5Pset_alignment.htm>`_:
+*For MPI IO and other parallel systems, choose an alignment which is a multiple of the disk block size.*
+On Lustre filesystems, according to the `NERSC documentation <https://www.nersc.gov/users/training/online-tutorials/introduction-to-scientific-i-o/?start=5>`_, it is advised to set this to the Lustre stripe size. In addition, ORNL Summit GPFS users are recommended to set the alignment value to 16777216(16MB).
+
 ``H5_COLL_API_SANITY_CHECK``: this is a HDF5 control option for debugging parallel I/O logic (API calls).
 Debugging a parallel program with that option enabled can help to spot bugs such as collective MPI-calls that are not called by all participating MPI ranks.
 Do not use in production, this will slow parallel I/O operations down.

src/IO/HDF5/ParallelHDF5IOHandler.cpp

@@ -74,6 +74,14 @@ ParallelHDF5IOHandlerImpl::ParallelHDF5IOHandlerImpl(AbstractIOHandler* handler,
     herr_t status;
     status = H5Pset_dxpl_mpio(m_datasetTransferProperty, xfer_mode);
 
+    auto const strByte = auxiliary::getEnvString( "OPENPMD_HDF5_ALIGNMENT", "1" );
+    std::stringstream sstream(strByte);
+    hsize_t bytes;
+    sstream >> bytes;
+
+    if ( bytes > 1 )
+         H5Pset_alignment(m_fileAccessProperty, 0, bytes);
+
     VERIFY(status >= 0, "[HDF5] Internal error: Failed to set HDF5 dataset transfer property");
     status = H5Pset_fapl_mpio(m_fileAccessProperty, m_mpiComm, m_mpiInfo);
     VERIFY(status >= 0, "[HDF5] Internal error: Failed to set HDF5 file access property");