Snapshot computation

[tjwsch]

Partly implementation of snapshot features discussed in #89. This does not yet include the configuration of the parameter space at runtime and a callback interface to invoke postprocessing as discussed in SimTech project PN5-9(II).
Reading parameter information from an hdf5 file and writing snapshot outputs to hdf5 files and the project structure might need more work.

[tjwsch]

There are some open questions and restructuring of postprocessing is required.

[IshaanDesai]

Good work already 👏 My suggestions are fundamental and hence require some discussion before large implementation changes. Let us discuss here in the comments.

[tjwsch]

For large parameter spaces, we need to think about chunking. Everything I have found suggests that, within a dataset, all chunks must have the same size, which complicates matters. A poor choice of chunk size can lead, besides other effects, to files containing empty allocated space. Not knowing the dimensions of a snapshot also complicates finding an optimal chunk size. As subfiling is currently not supported in h5py, writing to one file from multiple processes is not the best idea, and activating MPI support in h5py requires additional steps, writing to a separate file from every process and merging the output files, seems to be the best option available with h5py at this time.
The first option would be to make the user provide the parameter space file in chunks This would allow processes to read only the chunks they require, but slightly complicate the parameter file creation process for the user, and if each file only reads whole chunks leads, to some load imbalances. If chunks are split up between processes this complicates the the decomposition. This would also pre-set a chunk size, which could have advantages and disadvantages.
The second option would be to read the parameter file as a whole and only have chunked output files. Again, the questions of optimal chunk sizes and how to minimize the empty file space are relevant. I think cases exist where each process would have empty allocated space, which would accumulate in the merged file unless the merging does not take place chunk-wise but snapshot-wise, and a new chunk size for the merged file is determined.

[uekerman]

make the user provide the parameter space file in chunks

This sounds like sth we should better hide from the user.

[IshaanDesai]

Looks good 👍 this is close to merging now. Lets resolve some last comments. Please also remember that documentation for snapshot computation needs to be added. As discussed earlier, we can do this in a separate pull request.