MemoryHighWater

Tool Description

For a given Kokkos application program, the Memory High Water Mark tool prints, at the end of the run, the high water mark resident set size (RSS) of memory used, i.e., it prints the maximum of the system RAM which was consumed at any point during the run of the Kokkos program. The tool is located at: https://github.com/kokkos/kokkos-tools/tree/develop/profiling/memory-hwm

Compiling

Makefile

Simply type make inside the source directory for the memory high water mark. When compiling for specific platforms, modify the simple Makefile to use the correct compiler and compiler flags.

CMake

One can also use the cmake build system. Create a build directory and go to that directory. Type ccmake .. to ensure that kp_memory_hwm is in the list of profilers. Then, type cmake ..; make -j; sudo make install to build the profiler.

Using

This is a standard tool which does not yet support tool chaining. In Bash do:

export KOKKOS_TOOLS_LIBS={PATH_TO_TOOL_DIRECTORY}/kp_hwm.so
./application COMMANDS

This tool does not use any additional resources, and it is only called during initialization and finalization.

Output

The MemoryHighWaterMark tool will simply print out a high water mark for memory usage on each process.

Example Output

Consider the following code:

#include<Kokkos_Core.hpp>

int main(int argc, char* argv[]) {
  Kokkos::initialize(argc,argv);
  {
    int N = 100000000;
  
    Kokkos::View<double*> c("C",N);
    for(int i=0; i<5; i++) {
      Kokkos::View<double*> a("B",N);
      Kokkos::View<double*> b("C",N);
  
      Kokkos::parallel_for("Init", N, KOKKOS_LAMBDA (const int& i) {
        a(i) = 1.0*i;
        b(i) = 1.5*i;
      });

      Kokkos::parallel_for("Init", N, KOKKOS_LAMBDA (const int& i) {
        c(i) += a(i) + b(i);
      });
    }
    
    double dot;
    Kokkos::parallel_reduce("Dot", N, KOKKOS_LAMBDA (const int& i, double& lsum) {
      lsum += c(i)*c(i);
    },dot);

    printf("Result: %lf\n",dot);
  }
  Kokkos::finalize();
}

Running this code will print:

KokkosP: Library Loaded: /home/crtrott/Kokkos/kokkos-tools/src/tools/memory-hwm/kp_hwm.so
KokkosP: Example Library Initialized (sequence is 0, version: 20150628)
Result: 52083332552083570339872768.000000

KokkosP: Finalization of profiling library.
KokkosP: High water mark memory consumption: 2228096 kB