PnetCDFBenchmark
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
parent directory.. | ||||
This benchmark is translated from the HDF5 benchmark. It store the plot file in NetCDF format using the PnetCDF library.
The usage and options is exactly the same as the HDF5 benchmark. Run the benchmark program with a input file containing the configuration. inputs is an example input file.
To build the benchmark:
First, edit the GNUmakefile:
set variable "PNETCDF_HOME" to the install directory of PnetCDF library
Then, simply run make
To run the benchmark:
./<benchmark> inputs
Note:
open MP in GNUmakefile should remain disabled since PnetCDF does not officialy support open MP.
Performance evaluation:
1024 processes on Cori @ NERSC
Lustre 64 stripe with 1 MiB stripe size
Input file (49 GiB total size):
ncells = 1024
max_grid_size = 32
nlevs = 1
ncomp = 6
nppc = 2
I/O time (sec) compared to HDF5 benchmark:
Round 1 2 3
HDF5 93 111 110
PnetCDF 17 16 28
Example output (Running on 1024 processes):
MPI initialized with 1024 MPI processes
AMReX (18.11-803-g43c833e1d9f0-dirty) initialized
Write_NCHEADER_time_1_2_0 = 0.4263730049 seconds.
#%$: ncmpi_create_time: 0.4263730049
Write_NCATT_time_1_3 = 0.006245136261 seconds.
#%$: file_init_att_time: 0.01249027252
Write_NCINIT_time_1 = 0.5565130711 seconds.
#%$: create_open_time: 0.5565130711
Write_NCATT_time_2 = 0.002128839493 seconds.
#%$: write_att_time: 0.002128839493
Write_NCATT_time_3 = 0.01520013809 seconds.
#%$: def_meta_var_dim_time: 0.01520013809
Write_NCVAR_time_4 = 0.2298190594 seconds.
#%$: write_meta_var_time: 0.2298190594
Write_NCATT_time = 0.8017010689 seconds.
#%$: write_all_meta_time: 0.8017010689
::---- calling NCPUT for the grid data on level 0
Write_NCATT_time_5 = 0.2226190567 seconds.
#%$: def_data_var_time: 0.2226190567
Write_NCVARPUT_time = 15.79316187 seconds.
Write_NCVARPUT_time_since = 1545594288
#%$: write_data_time: 15.79316187
Write_NC_time_7_closing = 0 seconds.
Write_PNETCDF_time = 16.72815895 seconds.
#%$: write_file_total_time: 16.72815895
TinyProfiler total time across processes [min...avg...max]: 26.11 ... 26.33 ... 27.63
----------------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
----------------------------------------------------------------------------------------------------
NCVarPutGrids 1 15.79 15.79 15.79 57.15%
VisMF::Write(FabArray) 1 0.2093 1.421 7.965 28.82%
VisMF::CalculateMinMax 1 0.0101 1.04 7.698 27.86%
ParticleContainer::Checkpoint() 1 0.08512 6.133 6.959 25.18%
WriteMultiLevelPlotfilePNETCDF 1 0.8486 0.8559 0.8599 3.11%
VisMF::WriteHeader 1 8.84e-07 0.0003416 0.3481 1.26%
DistributionMapping::LeastUsedCPUs() 1 0.05556 0.1135 0.2797 1.01%
NFI::ReadyToWrite:decider 0 0 0.0002229 0.2282 0.83%
WriteMultiLevelPlotfile() 1 0.0988 0.1212 0.1308 0.47%
WriteGenericPlotfileHeader() 0 0 0.0001129 0.1156 0.42%
FABio_binary::write_header 96 0.0001979 0.0007719 0.08167 0.30%
NCVarPut 1 0.07673 0.07985 0.08114 0.29%
DistributionMapping::SFCProcessorMapDoIt() 1 0.04754 0.05188 0.07602 0.28%
FabArray::setVal() 1 0.03905 0.05305 0.07502 0.27%
ParticleContainer::RedistributeMPI() 1 0.01621 0.04918 0.07067 0.26%
VisMF::FindOffsets 1 5.539e-06 0.002958 0.0501 0.18%
FABio::write_header 96 9.944e-05 0.0003072 0.04653 0.17%
ParticleContainer::RedistributeCPU() 1 0.008609 0.02038 0.04361 0.16%
ParticleContainer<NSR, NSI, NAR, NAI>::InitRandom() 1 0.0003943 0.001009 0.003208 0.01%
NFI::CleanUpMessages 1 1.041e-06 2.587e-06 5.681e-05 0.00%
VisMF::Header 1 1.715e-06 2.294e-06 1.671e-05 0.00%
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
----------------------------------------------------------------------------------------------------
WriteMultiLevelPlotfilePNETCDF 1 16.72 16.73 16.73 60.54%
NCVarPut 1 15.87 15.87 15.87 57.44%
NCVarPutGrids 1 15.79 15.79 15.79 57.15%
WriteMultiLevelPlotfile() 1 1.762 2.587 8.636 31.25%
VisMF::Write(FabArray) 1 1.569 2.466 8.514 30.81%
VisMF::CalculateMinMax 1 0.0101 1.04 7.698 27.86%
ParticleContainer::Checkpoint() 1 0.08512 6.133 6.959 25.18%
VisMF::WriteHeader 1 8.84e-07 0.0003416 0.3481 1.26%
DistributionMapping::SFCProcessorMapDoIt() 1 0.1315 0.1654 0.3377 1.22%
DistributionMapping::LeastUsedCPUs() 1 0.05556 0.1135 0.2797 1.01%
NFI::ReadyToWrite:decider 0 0 0.0002229 0.2282 0.83%
VisMF::FindOffsets 1 5.539e-06 0.003082 0.1779 0.64%
FABio_binary::write_header 96 0.0002976 0.001079 0.1282 0.46%
WriteGenericPlotfileHeader() 0 0 0.0001129 0.1156 0.42%
ParticleContainer<NSR, NSI, NAR, NAI>::InitRandom() 1 0.03218 0.07057 0.1145 0.41%
ParticleContainer::RedistributeCPU() 1 0.03102 0.06956 0.1115 0.40%
FabArray::setVal() 1 0.03905 0.05305 0.07502 0.27%
ParticleContainer::RedistributeMPI() 1 0.01621 0.04918 0.07067 0.26%
FABio::write_header 96 9.944e-05 0.0003072 0.04653 0.17%
NFI::CleanUpMessages 1 1.041e-06 2.587e-06 5.681e-05 0.00%
VisMF::Header 1 1.715e-06 2.294e-06 1.671e-05 0.00%
----------------------------------------------------------------------------------------------------
AMReX (18.11-803-g43c833e1d9f0-dirty) finalized