HPCToolKit

Import HPCToolKit Database

HPCToolKit

HPCToolKit provides a set of extremely versatile profiling tools. Among other things, it can be used to provide accurate loop or source line level code analysis. It is possible to import an HPCToolKit database with PerfAnalyzer, and create flamegraphs, custom events, and difference plots at the loop or source line level. The flamegraphs are dynamically linked to the source code and can be used to navigate the code base. The analysis can also be useful if some time is spent performing work on a GPU device, to produce flamegraphs incorporating the GPU library calls. There are several steps required to create a profile with HPCToolKit

Install HPCToolKit

The source can be downloaded from the HPCToolKit GitHub site and built from source.

Profile with HPCToolKit

To profile cycles and instructions at 200 samples per second, with maximum precision level of 3:

/path_to_HPCToolKit/bin/hpcrun -c f200 -p 3 -e CYCLES,INSTRUCTIONS -o job1_data /path_to_executable/exe exe-args

HPCToolKit structure analysis

The HPCToolKit structure analysis step can be used to create accurate loop or source line analysis, which can be extremely important for highly optimized code. The main executable and all shared objects should be analyzed by hpcstruct using a simple script, such as:

#!/bin/sh
cd /path_to_executable
/path_to_HPCToolKit/bin/hpcstruct -I ./path_to_source_code/src* ./path_to_executable/exe
for file in *so
do
/path_to_HPCToolKit/bin/hpcstruct -I ./path_to_source_code/src* $file
done

For statically linked libraries it is necessary to link with hpclink to get the required output.

Create HPCToolKit Database

The HPCToolKit Database can be created by hpcprof by providing the profile data and the structure analysis. This step can be performed with a simple script to create a database with thread metrics (-M thread option) such as:

#!/bin/sh
command="/path_to_HPCToolKit/bun/hpcprof -M thread --force-metric -I ./path_to_source_code/src* -o ./hpctoolkit-job1-database"
cd /path_to_executable
for file in *hpcstruct
do
command="$command -S $file"
done
command="$command /path_to_hpcrun_results/job1_data"
$command

Import HPCToolKit Database

Finally, the HPCToolKit database can be imported by perfAnalyzer from the Import Perf / HPCToolKit Profile Data menu. The type of analysis can be Function, Loop, or line. The directory of the HPCToolKit (/hpctoolkit-job1-database in the example), should be selected and the entire contents will be copied to the ./perf_analyzer/results folder and converted to the required format to be viewed.