hpctoolkit Overview

HPCToolkit is an integrated suite of tools for measurement and analysis of program performance on computers ranging from multicore desktop systems to the nation’s largest supercomputers. By using statistical sampling of timers and hardware performance counters, HPCToolkit collects accurate measurements of a program’s work, resource consumption, and inefficiency and attributes them to the full calling context in which they occur. HPCToolkit works with multilingual, fully optimized applications that are statically or dynamically linked. Since HPCToolkit uses sampling, measurement has low overhead (1-5%) and scales to large parallel systems. HPCToolkit’s presentation tools enable rapid analysis of a program’s execution costs, inefficiency, and scaling characteristics both within and across nodes of a parallel system. HPCToolkit supports measurement and analysis of serial codes, threaded codes (e.g. pthreads, OpenMP), MPI, and hybrid (MPI+threads) parallel codes.
Support

Usage

module load hpctoolkit
This will also load the java module for when you use HPCViewer.

Compiling

If you use shared libraries there is no need to recompile your application. Howver you should make sure the application is compiled with debug information ("-g")

Running

Add hpcrun to your execution. This will preload the instrumetation library. You can also set some environment variables to control the profiling frequency. Please refer to the manual for details. mpirun -np 2 hpcrun ./your-application

Analyzing and Viewing Data

When you run your code, it will result in some output files. These files are in a directory called something like hpctoolkit-your-application-measurements-jobid You can use hpcstruct and hpcprof to put the data in a format that you can then visualize using hpcviewer. First use hpcstruct on your code, for example: hpcstruct -I’/path/to/your/include/*’ ./your-application This will result in an output file called your-application.hpcstruct. (The asterrisk * represents recursive search of the subdirectories within the include directory. You can either escape the asterisk or put the whole directory name in quotes, just to protect it from the shell.) You can have multiple -I arguments. For more information, see the manpage or do hpcstruct –help. Then, to use hpcprof, you will use that file as input, along with the measurement outputs you obtained from running your job. Your command will look something like this: hpcprof -I’/path/to/your/include/*’ -S ./your-application.hpcstruct hpctoolkit-your-application-measurements-85800.nodename/ (see the manpage or hpcprof –help for more options) You can use hpcviewer to view your data. hpcviewer is available on yona but for remote users it may be prohibitively slow. You can download a version of it to run on your local machine, available at the HPCToolkit web page. For more details please see the HPCToolkit web page referenced above.
Builds

TITAN

  • hpctoolkit@5.4%gcc@5.3.0+mpi+papi
  • hpctoolkit@5.4%gcc@5.3.0+mpi+papi