Home
PerfAnalyzer is a lightweight tool for performance profiling of codes written in compiled languages such as C++ and Fortran. The tool originated from a need to analyze codes running on multiple processors and with multiple threads, whilst comparing the performance of different code designs. When combined with the Linux perf events interface, flamegraphs proved to be an extremely neat way of visualizing the necessary data. The automation of the job submission process, the data analysis, and the visualization within a web browser, led to the PerfAnalyzer tool. The main options are described below:
Jobs can be submitted using any combination of mpi, lsf, and ssh. This allows a great deal of flexibility to run locally, or to submit jobs to different hosts, clusters, or remote servers.
Jobs can be submitted to monitor an application using Linux perf events. By default, when multiple events are monitored, several runs may be performed. This is to prevent multiplexing of events, so that the resulting data
can be easily understood. Events can be sampled by setting either a sampling period or a sampling frequency. In the case of a sampling frequency, the kernel automatically scales the sampling period to obtain the requested
frequency. This makes it far simpler to control the number of samples collected, for events occurring at very different rates.
In the standard mode a profile job is submitted by attaching perf to each process of the application. All threads will be monitored for each process of the application, and profiling data is obtained for
each of the individual processes/threads. The permissions required to run perf correctly depend on the value set in /proc/sys/kernel/perf_event_paranoid. It is recommended to run with
/proc/sys/kernel/perf_event_paranoid equal to either -1 or 0 for application profiling (i.e. sudo sh -c 'echo -1 >/proc/sys/kernel/perf_event_paranoid'),
to avoid the need to run perf as root. To associate kernel function calls with the applications call sites the value of /proc/sys/kernel/kptr_restrict should be set to 0.
For system wide profiling perf is attached to one process on each node and monitors the activity on all cores of the node. In this case all of the threads running on each of the underlying cores are monitored, and profiling data is obtained for each individual core/thread.
To use system wide profiling it is recommended to set /proc/sys/kernel/perf_event_paranoid to either -1 or 0 (i.e. sudo sh -c 'echo -1 >/proc/sys/kernel/perf_event_paranoid'), to avoid the need to run perf as root.
Each time a job is submitted, the settings used are saved to the path_to_perf_profiler/data folder, with the file name job_name.settings. These settings files can be loaded to restore the exact settings and events of a previous job. Because configuring the initial settings
can be a bit laborious, this makes submitting and editing further jobs much simpler.
Results can be viewed by loading one or more of the results files from the location path_to_perf_profiler/data, with the file name job_name.results. For comparisons between jobs, it is often useful to load them together,
to allow automatic differencing of the profiles.
Results with a large number of processes/threads and events will increase the overhead of viewing the associated data.
Raw Perf data can be read directly and converted to the format required by PerfAnalyzer. The absolute path to the data must be provided, and perf script is invoked on each selected perf
file (.data or .perf) selected, to process the data.
A database created by HPCToolKit can also be converted for use with the PerfAnalyzer tool. This only works for standard profiles run with HPCToolKit, and not with trace profiles.
The base directory for the database should be selected, and the directory structure will be copied locally. If the profile was performed with a code structure analysis, the database
will contain accurate loop and source line analysis. The database can be converted using either Procedure, loop, or source level analysis, and the resulting flamegraphs and
charts are displayed at the corresponding level of detail. The flamegraph is also linked to a display of the source code from the database, to allow easy navigation between profiled
hotspots and the responsible code.
Progress of the jobs can be seen in the monitor job progress tab. Any job submission errors will be reported in detail, and the progress of each submitted job is polled regularly to report