Merge pull request #2894 from ComputationalRadiationPhysics/release-0…

….4.3

Release 0.4.3: System Updates and Bug Fixes

.travis.yml

@@ -6,6 +6,7 @@ cache:
   apt: true
   directories:
     - $HOME/.cache/spack
+    - $HOME/.cache/cmake-3.10.0
   pip: true
 
 addons:
@@ -35,19 +36,37 @@ install:
   - if [ $SPACK_FOUND -ne 0 ]; then
       mkdir -p $SPACK_ROOT &&
       git clone --depth 50 https://github.com/spack/spack.git $SPACK_ROOT &&
-      echo -e "config:""\n  build_jobs:"" 2" > $SPACK_ROOT/etc/spack/config.yaml;
+      echo -e "config:""\n  build_jobs:"" 2" > $SPACK_ROOT/etc/spack/config.yaml &&
+      echo -e "packages:""\n  cmake:""\n    version:"" [3.10.0]""\n    paths:""\n      cmake@3.10.0:"" /home/travis/.cache/cmake-3.10.0""\n    buildable:"" False" > $SPACK_ROOT/etc/spack/packages.yaml;
     fi
   - spack compiler add
+  # required dependencies - CMake 3.10.0
+  - if [ "$TRAVIS_OS_NAME" == "linux" ]; then
+      if [ ! -f $HOME/.cache/cmake-3.10.0/bin/cmake ]; then
+        wget -O cmake.sh https://cmake.org/files/v3.10/cmake-3.10.0-Linux-x86_64.sh &&
+        sh cmake.sh --skip-license --exclude-subdir --prefix=$HOME/.cache/cmake-3.10.0 &&
+        rm cmake.sh;
+      fi;
+    elif [ "$TRAVIS_OS_NAME" == "osx" ]; then
+      if [ ! -d /Applications/CMake.app/Contents/ ]; then
+        curl -L -s -o cmake.dmg https://cmake.org/files/v3.10/cmake-3.10.0-Darwin-x86_64.dmg &&
+        yes | hdiutil mount cmake.dmg &&
+        sudo cp -R "/Volumes/cmake-3.10.0-Darwin-x86_64/CMake.app" /Applications &&
+        hdiutil detach /dev/disk1s1 &&
+        rm cmake.dmg;
+      fi;
+    fi
   - travis_wait spack install
-      cmake@3.10.0~openssl~ncurses
+      cmake
       $COMPILERSPEC
+  # required dependencies - Boost 1.62.0
   - travis_wait spack install
       boost@1.62.0~date_time~graph~iostreams~locale~log~random~thread~timer~wave
       $COMPILERSPEC
   - spack clean -a
   - source /etc/profile &&
     source $SPACK_ROOT/share/spack/setup-env.sh
-  - spack load cmake $COMPILERSPEC
+  - spack load cmake
   - spack load boost $COMPILERSPEC
 
 jobs:

CHANGELOG.md

@@ -1,9 +1,66 @@
 Changelog
 =========
 
+0.4.3
+-----
+
+**Date:** 2019-02-14
+
+System Updates and Bug Fixes
+
+This release adds updates and new HPC system templates. Important bug
+fixes include I/O work-arounds for issues in OpenMPI 2.0-4.0 (mainly
+with HDF5), guards for particle creation with user-defined
+profiles, a fixed binomial current smoothing, checks for the number
+of devices in grid distributions and container (Docker & Singularity)
+modernizations.
+
+Thanks to Axel Huebl, Alexander Debus, Igor Andriyash, Marco Garten,
+Sergei Bastrakov, Adam Simpson, Richard Pausch, Juncheng E,
+Klaus Steiniger, and René Widera for contributions to this release!
+
+### Changes to "0.4.2"
+
+**Bug Fixes:**
+- fix particle creation if density `<=` zero #2831
+- fix binomial current interpolation #2838
+- Docker & Singularity updates #2847
+- OpenMPI: use ROMIO for IO #2841 #2857
+- `--gridDist`: verify devices and blocks #2876
+- Phase space plugin: unit of colorbar in 2D3V #2878
+
+**Misc:**
+- `ionizer.param`: fix typo in "Aluminium" #2865
+- System Template Updates:
+  - Add system links #2818
+  - Taurus:
+    - add project #2819
+    - add Power9 V100 nodes #2856 
+  - add D.A.V.I.D.E (CINECA) #2821
+  - add JURECA (JSC) #2869
+  - add JUWELS (JSC) #2874
+  - Hypnos (HZDR): CMake update #2887
+  - Slurm systems: link `stdout` to `simOutput/output` #2839
+- Docs:
+  - Change link to CRP group @ HZDR #2814
+  - `FreeRng.def`: typo in example usage #2825
+  - More details on source builds #2828
+  - Dependencies: Blosc install #2829
+  - Ionization plot title linebreak #2867
+- plugins:
+  - ADIOS & phase space `-Wterminate` #2817
+  - Radiation: update documented options #2842
+- Update versions script: containers #2846
+- pyflakes: `str`/`bytes`/`int` compares #2866
+- Travis CI: Fix Spack CMake Install #2879
+- Contributor name typo in `LICENSE.md` #2880
+- Update mallocMC to 2.3.1crp #2893
+- CMake: Honor `_ROOT` Env Hints #2891 #2892 #2893
+
+
 0.4.2
 -----
-**Date:** 2018-11-TBA
+**Date:** 2018-11-19
 
 CPU Plugin Performance
 

INSTALL.rst

@@ -171,29 +171,44 @@ CUDA
 If you do not install the following libraries, you will not have the full amount of PIConGPU plugins.
 We recommend to install at least **pngwriter** and either **libSplash** (+ **HDF5**) or **ADIOS**.
 
+libpng
+""""""
+- 1.2.9+ (requires *zlib*)
+- *Debian/Ubuntu dependencies:* ``sudo apt-get install libpng-dev``
+- *Arch Linux dependencies:* ``sudo pacman --sync libpng``
+- *Spack:* ``spack install libpng``
+- *from source:*
+
+  - ``mkdir -p ~/src ~/lib``
+  - ``cd ~/src``
+  - ``curl -Lo libpng-1.6.34.tar.gz ftp://ftp-osl.osuosl.org/pub/libpng/src/libpng16/libpng-1.6.34.tar.gz``
+  - ``tar -xf libpng-1.6.34.tar.gz``
+  - ``cd libpng-1.6.34``
+  - ``CPPFLAGS=-I$HOME/lib/zlib/include LDFLAGS=-L$HOME/lib/zlib/lib ./configure --enable-static --enable-shared --prefix=$HOME/lib/libpng``
+  - ``make``
+  - ``make install``
+- *environment:* (assumes install from source in ``$HOME/lib/libpng``)
+
+  - ``export PNG_ROOT=$HOME/lib/libpng``
+  - ``export CMAKE_PREFIX_PATH=$PNG_ROOT:$CMAKE_PREFIX_PATH``
+  - ``export LD_LIBRARY_PATH=$PNG_ROOT/lib:$LD_LIBRARY_PATH``
+
 pngwriter
 """""""""
-- 0.7.0+
+- 0.7.0+ (requires *libpng*, *zlib*, and optional *freetype*)
 - *Spack:* ``spack install pngwriter``
 - *from source:*
 
-  - download from `github.com/pngwriter/pngwriter <https://github.com/pngwriter/pngwriter>`_
-  - Requires `libpng <http://www.libpng.org>`_
-
-    - *Debian/Ubuntu:* ``sudo apt-get install libpng-dev``
-    - *Arch Linux:* ``sudo pacman --sync libpng``
-  - example:
-
-    - ``mkdir -p ~/src ~/build ~/lib``
-    - ``git clone https://github.com/pngwriter/pngwriter.git ~/src/pngwriter/``
-    - ``cd ~/build``
-    - ``cmake -DCMAKE_INSTALL_PREFIX=$HOME/lib/pngwriter ~/src/pngwriter``
-    - ``make install``
+  - ``mkdir -p ~/src ~/build ~/lib``
+  - ``git clone https://github.com/pngwriter/pngwriter.git ~/src/pngwriter/``
+  - ``cd ~/build``
+  - ``cmake -DCMAKE_INSTALL_PREFIX=$HOME/lib/pngwriter ~/src/pngwriter``
+  - ``make install``
 
-  - *environment:* (assumes install from source in ``$HOME/lib/pngwriter``)
+- *environment:* (assumes install from source in ``$HOME/lib/pngwriter``)
 
-    - ``export CMAKE_PREFIX_PATH=$HOME/lib/pngwriter:$CMAKE_PREFIX_PATH``
-    - ``export LD_LIBRARY_PATH=$HOME/lib/pngwriter/lib:$LD_LIBRARY_PATH``
+  - ``export CMAKE_PREFIX_PATH=$HOME/lib/pngwriter:$CMAKE_PREFIX_PATH``
+  - ``export LD_LIBRARY_PATH=$HOME/lib/pngwriter/lib:$LD_LIBRARY_PATH``
 
 libSplash
 """""""""
@@ -205,7 +220,7 @@ libSplash
 
   - ``mkdir -p ~/src ~/build ~/lib``
   - ``git clone https://github.com/ComputationalRadiationPhysics/libSplash.git ~/src/splash/``
-  - ``cd ~/build``
+  - ``cd ~/build && rm -rf ../build/*``
   - ``cmake -DCMAKE_INSTALL_PREFIX=$HOME/lib/splash -DSplash_USE_MPI=ON -DSplash_USE_PARALLEL=ON ~/src/splash``
   - ``make install``
 
@@ -223,7 +238,7 @@ HDF5
 - *Spack:* ``spack install hdf5~fortran``
 - *from source:*
 
-  - ``mkdir -p ~/src ~/build ~/lib``
+  - ``mkdir -p ~/src ~/lib``
   - ``cd ~/src``
   - download hdf5 source code from `release list of the HDF5 group <https://www.hdfgroup.org/ftp/HDF5/releases/>`_, for example:
 
@@ -263,9 +278,31 @@ png2gas
 - converts png files to hdf5 files that can be used as an input for a species initial density profiles
 - compile and install exactly as *splash2txt* above
 
+c-blosc
+"""""""
+- general purpose compressor, used in ADIOS for in situ data reduction
+- *Debian/Ubuntu:* ``sudo apt-get install libblosc-dev``
+- *Arch Linux:* ``sudo pacman --sync blosc``
+- *Spack:* ``spack install c-blosc``
+- *from source:*
+
+  - ``mkdir -p ~/src ~/build ~/lib``
+  - ``cd ~/src``
+  - ``curl -Lo c-blosc-1.15.0.tar.gz https://github.com/Blosc/c-blosc/archive/v1.15.0.tar.gz``
+  - ``tar -xzf c-blosc-1.15.0.tar.gz``
+  - ``cd ~/build && rm -rf ../build/*``
+  - ``cmake -DCMAKE_INSTALL_PREFIX=$HOME/lib/c-blosc -DPREFER_EXTERNAL_ZLIB=ON ~/src/c-blosc-1.15.0/``
+  - ``make``
+  - ``make install``
+- *environment:* (assumes install from source in ``$HOME/lib/c-blosc``)
+
+  - ``export BLOSC_ROOT=$HOME/lib/c-blosc``
+  - ``export CMAKE_PREFIX_PATH=$BLOSC_ROOT:$CMAKE_PREFIX_PATH``
+  - ``export LD_LIBRARY_PATH=$BLOSC_ROOT/lib:$LD_LIBRARY_PATH``
+
 ADIOS
 """""
-- 1.13.1+ (requires *MPI* and *zlib*)
+- 1.13.1+ (requires *MPI*, *zlib* and *c-blosc*)
 - *Debian/Ubuntu:* ``sudo apt-get install libadios-dev libadios-bin``
 - *Arch Linux* using an `AUR helper <https://wiki.archlinux.org/index.php/AUR_helpers>`_: ``pacaur --sync libadios``
 - *Arch Linux* using the `AUR <https://wiki.archlinux.org/index.php/Arch_User_Repository>`_ manually:
@@ -277,12 +314,12 @@ ADIOS
 - *Spack:* ``spack install adios``
 - *from source:*
 
-  - ``mkdir -p ~/src ~/build ~/lib``
+  - ``mkdir -p ~/src ~/lib``
   - ``cd ~/src``
   - ``curl -Lo adios-1.13.1.tar.gz http://users.nccs.gov/~pnorbert/adios-1.13.1.tar.gz``
   - ``tar -xzf adios-1.13.1.tar.gz``
   - ``cd adios-1.13.1``
-  - ``CFLAGS="-fPIC" ./configure --enable-static --enable-shared --prefix=$HOME/lib/adios --with-mpi=$MPI_ROOT --with-zlib=/usr``
+  - ``CFLAGS="-fPIC" ./configure --enable-static --enable-shared --prefix=$HOME/lib/adios --with-mpi=$MPI_ROOT --with-zlib=$HOME/lib/zlib --with-blosc=$HOME/lib/c-blosc``
   - ``make``
   - ``make install``
 - *environment:* (assumes install from source in ``$HOME/lib/adios``)

LICENSE.md

@@ -7,7 +7,7 @@ Sergei Bastrakov, Florian Berninger, Heiko Burau, Michael Bussmann,
 Alexander Debus, Robert Dietrich, Carlchristian Eckert, Wen Fu, Marco Garten,
 Ilja Goethel, Alexander Grund, Sebastian Hahn, Anton Helm, Wolfgang Hoehnig,
 Axel Huebl, Jeffrey Kelling, Maximilian Knespel, Remi Lehe, Alexander Matthes,
-Richard Pausch, Rophie Rudat, Felix Schmitt, Conrad Schumann,
+Richard Pausch, Sophie Rudat, Felix Schmitt, Conrad Schumann,
 Benjamin Schneider, Joseph Schuchart, Sebastian Starke, Klaus Steiniger,
 Rene Widera, Benjamin Worpitz
 

README.md

@@ -73,7 +73,7 @@ PIConGPU was one of the **finalists** of the 2013
 [Gordon Bell Prize](http://sc13.supercomputing.org/content/acm-gordon-bell-prize).
 
 PIConGPU is developed and maintained by the
-[Computational Radiation Physics Group](http://www.hzdr.de/db/Cms?pNid=132&pOid=30354)
+[Computational Radiation Physics Group](https://www.hzdr.de/db/Cms?pNid=2097)
 at the [Institute for Radiation Physics](http://www.hzdr.de/db/Cms?pNid=132)
 at [HZDR](http://www.hzdr.de/) in close collaboration with the Center
 for Information Services and High Performance Computing

docs/source/conf.py

@@ -105,9 +105,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = u'0.4.2'
+version = u'0.4.3'
 # The full version, including alpha/beta/rc tags.
-release = u'0.4.2'
+release = u'0.4.3'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/source/install/instructions/source.rst

@@ -15,10 +15,10 @@ From Source
 
 Don't be afraid young physicist, self-compiling C/C++ projects is easy, fun and profitable!
 
-Compiling a project from source essentially requires three steps:
+Building a project from source essentially requires three steps:
 
     #. configure the project and find its dependencies
-    #. build the project
+    #. compile the project
     #. install the project
 
 All of the above steps can be performed without administrative rights ("root" or "superuser") as long as the install is not targeted at a system directory (such as ``/usr``) but inside a user-writable directory (such as ``$HOME`` or a project directory).
@@ -40,11 +40,19 @@ In order to compile projects from source, we assume you have individual director
 Note that on some supercomputing systems, you might need to install the final software outside of your home to make dependencies available during run-time (when the simulation runs).
 Use a different path for the last directory then.
 
-Step-by-Step
-^^^^^^^^^^^^
+What is Compiling?
+^^^^^^^^^^^^^^^^^^
+
+.. note::
+
+   This section is **not** yet the installation of PIConGPU from source.
+   It just introduces in general how one compiles projects.
+
+   If you like to skip this introduction, :ref:`jump straight to the dependency install section <install-dependencies>`.
 
 Compling can differ in two principle ways: building *inside* the source directory ("in-source") and in a *temporary directory* ("out-of-source").
 Modern projects prefer the latter and use a build system such as [CMake]_.
+
 An example could look like this
 
 .. code-block:: bash
@@ -62,7 +70,7 @@ Often, you want to pass further options to CMake with ``-DOPTION=VALUE`` or modi
 The second step which compiles the project can in many cases be parallelized by ``make -j``.
 In the final install step, you might need to prefix it with ``sudo`` in case ``CMAKE_INSTALL_PREFIX`` is pointing to a system directory.
 
-Some older projects still build *in-source* and use a build system called *autotools*.
+Some older projects often build *in-source* and use a build system called *autotools*.
 The syntax is still very similar:
 
 .. code-block:: bash
@@ -75,8 +83,16 @@ The syntax is still very similar:
    make
    make install
 
-That's all!
-Continue with the following section to build our dependencies.
+One can usually pass further options with ``--with-something=VALUE`` or ``--enable-thing`` to ``configure``.
+See ``configure --help`` when installing an *autotools* project.
+
+That is all on the theory of building projects from source!
+
+Now Start
+^^^^^^^^^
+
+You now know all the basics to install from source.
+Continue with the following section to :ref:`build our dependencies <install-dependencies>`.
 
 References
 ^^^^^^^^^^