Publishing Notice Prior to publishing work performed on Summit, please contact help@olcf.ornl.gov.

Compilers

Available Compilers

The following compilers are available on Summit:
  XL: IBM XL Compilers (loaded by default)
  LLVM: LLVM compiler infrastructure
  PGI: Portland Group compiler suite
  GNU: GNU Compiler Collection
  NVCC: CUDA C compiler

Upon login, the default versions of the XL compiler suite and Spectrum Message Passing Interface (MPI) are added to each user’s environment through the modules system. No changes to the environment are needed to make use of the defaults.

Multiple versions of each compiler family are provided, and can be inspected using the modules system:

summit$ module -t avail pgi
/sw/summit/modulefiles/site/linux-rhel7-ppc64le/Core:
pgi/17.10-patched
pgi/18.3
pgi/18.4
pgi/18.5
pgi/18.7

C compilation

Note: type char is unsigned by default
Vendor Module Compiler Version Enable C99 Enable C11 Default signed char Define macro
IBM xl xlc xlc_r 13.1.6 -std=gnu99 -std=gnu11 -qchar=signed -WF,-D
GNU system default gcc 4.8.5 -std=gnu99 -std=gnu11 -fsigned-char -D
GNU gcc gcc 6.4.0 -std=gnu99 -std=gnu11 -fsigned-char -D
LLVM llvm clang 3.8.0 default -std=gnu11 -fsigned-char -D
PGI pgi pgcc 17.10 -c99 -c11 -Mschar -D

C++ compilations

Note: type char is unsigned by default
Vendor Module Compiler Version Enable C++11 Enable C++14 Default signed char Define macro
IBM xl xlc++, xlc++_r 13.1.6 -std=gnu++11 -std=gnu++1y (PARTIAL) -qchar=signed -WF,-D
GNU system default g++ 4.8.5 -std=gnu++11 -std=gnu++1y -fsigned-char -D
GNU gcc g++ 6.4.0 -std=gnu++11 -std=gnu++1y -fsigned-char -D
LLVM llvm clang++ 3.8.0 -std=gnu++11 -std=gnu++1y -fsigned-char -D
PGI pgi pgc++ 17.10 -std=c++11 –gnu_extensions -std=c++14 –gnu_extensions -Mschar -D

Fortran compilation

Vendor Module Compiler Version Enable F90 Enable F2003 Enable F2008 Define macro
IBM xl xlf
xlf90
xlf95
xlf2003
xlf2008
15.1.6 -qlanglvl=90std -qlanglvl=2003std -qlanglvl=2008std -WF,-D
GNU system default gfortran 4.8.5, 6.4.0 -std=f90 -std=f2003 -std=f2008 -D
LLVM llvm xlflang 3.8.0 n/a n/a n/a -D
PGI pgi pgfortran 17.10 use .F90 source file suffix use .F03 source file suffix use .F08 source file suffix -D
Note: * The xlflang module currently conflicts with the clang module. This restriction is expected to be lifted in future releases.

MPI

MPI on Summit is provided by IBM Spectrum MPI. Spectrum MPI provides compiler wrappers that automatically choose the proper compiler to build your application.

The following compiler wrappers are available:
  C: mpicc
  C++: mpic++, mpiCC
  Fortran: mpifort, mpif77, mpif90

While these wrappers conveniently abstract away linking of Spectrum MPI, it’s sometimes helpful to see exactly what’s happening when invoked. The --showme flag will display the full link lines, without actually compiling:

summit$ mpicc --showme
/sw/summit/xl/16.1.1-beta6/xlC/16.1.1/bin/xlc -I/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20171006/linux-rhel7-ppc64le/xl-16.1.1-beta6/spectrum-mpi-10.2.0.7-20180830-eyo7zxm2piusmyffr3iytmgwdacl67ju/include -pthread -L/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20171006/linux-rhel7-ppc64le/xl-16.1.1-beta6/spectrum-mpi-10.2.0.7-20180830-eyo7zxm2piusmyffr3iytmgwdacl67ju/lib -lmpiprofilesupport -lmpi_ibm

OpenMP

Note: When using OpenMP with IBM XL compilers, the thread-safe compiler variant is required; These variants have the same name as the non-thread-safe compilers with an additional _r suffix. e.g. to compile OpenMPI C code one would use xlc_r
Note: OpenMP offloading support is still under active development. Performance and debugging capabilities in particular are expected to improve as the implementations mature.
Vendor 3.1 Support Enable OpenMP 4.x Support Enable OpenMP 4.x Offload
IBM FULL -qsmp=omp PARTIAL -qsmp=omp
-qoffload
GNU FULL -fopenmp PARTIAL -fopenmp
clang FULL -fopenmp PARTIAL -fopenmp
-fopenmp-targets=nvptx64-nvidia-cuda
–cuda-path=${OLCF_CUDA_ROOT}
xlflang FULL -fopenmp PARTIAL -fopenmp
-fopenmp-targets=nvptx64-nvidia-cuda
PGI FULL -mp NONE NONE

OpenACC

Vendor Module OpenACC Support Enable OpenACC
IBM xl NONE NONE
GNU system default NONE NONE
GNU gcc 2.5 -fopenacc
LLVM clang or xlflang NONE NONE
PGI pgi 2.5 -acc, -ta=nvidia:cc70

CUDA compilation

NVIDIA

CUDA C/C++ support is provided through the cuda module.
nvcc : Primary CUDA C/C++ compiler

Language support

-std=c++11 : provide C++11 support
–expt-extended-lambda : provide experimental host/device lambda support
–expt-relaxed-constexpr : provide experimental host/device constexpr support

Compiler support

NVCC currently supports XL, GCC, and PGI C++ backends.
–ccbin : set to host compiler location

CUDA Fortran compilation

IBM

The IBM compiler suite is made available through the default loaded xl module, the cuda module is also required.
xlcuf : primary Cuda fortran compiler, thread safe
Language support flags
 -qlanglvl=90std : provide Fortran90 support
-qlanglvl=95std : provide Fortran95 support
-qlanglvl=2003std : provide Fortran2003 support
-qlanglvl=2008std : provide Fortran2003 support

PGI

The PGI compiler suite is available through the pgi module.
pgfortran : Primary fortran compiler with CUDA Fortran support

Language support:

Files with .cuf suffix automatically compiled with cuda fortran support
Standard fortran suffixed source files determines the standard involved, see the man page for full details
-Mcuda : Enable CUDA Fortran on provided source file

Linking in Libraries

OLCF systems provide hundreds of software packages and scientific libraries pre-installed at the system-level for users to take advantage of. In order to link these libraries into an application, users must direct the compiler to their location. The module show command can be used to determine the location of a particular library. For example

summit$ module show essl
------------------------------------------------------------------------------------
   /sw/summit/modulefiles/core/essl/6.1.0-1:
------------------------------------------------------------------------------------
whatis("ESSL 6.1.0-1 ")
prepend_path("LD_LIBRARY_PATH","/sw/summit/essl/6.1.0-1/essl/6.1/lib64")
append_path("LD_LIBRARY_PATH","/sw/summit/xl/16.1.1-beta4/lib")
prepend_path("MANPATH","/sw/summit/essl/6.1.0-1/essl/6.1/man")
setenv("OLCF_ESSL_ROOT","/sw/summit/essl/6.1.0-1/essl/6.1")
help([[ESSL 6.1.0-1

]])

When this module is loaded, the $OLCF_ESSL_ROOT environment variable holds the path to the ESSL installation, which contains the lib64/ and include/ directories:

summit$ module load essl
summit$ echo $OLCF_ESSL_ROOT
/sw/summit/essl/6.1.0-1/essl/6.1
summit$ ls $OLCF_ESSL_ROOT
FFTW3  READMES  REDIST.txt  include  iso-swid  ivps  lap  lib64  man  msg

The following screencast shows an example of linking two libraries into a simple program on Summit.