titan

Up since 11/8/17 02:45 pm

eos

Up since 11/14/17 11:20 pm

rhea

Up since 10/17/17 05:40 pm

hpss

Up since 11/20/17 09:15 am

atlas1

Up since 11/15/17 07:25 am

atlas2

Up since 11/27/17 10:45 am
OLCF User Assistance Center

Can't find the information you need below? Need advice from a real person? We're here to help.

OLCF support consultants are available to respond to your emails and phone calls from 9:00 a.m. to 5:00 p.m. EST, Monday through Friday, exclusive of holidays. Emails received outside of regular support hours will be addressed the next business day.

Hyper Threading

See this article in context within the following user guides: Eos


 
 

Hyper Threading Overview

Eos includes Intel processors with Intel’s Hyper-Threading technology. With Hyper-Threading enabled, the operating system recognizes each physical core as two logical cores. Two independent processes or threads can run simultaneously on the same physical core, but because the two logical cores are sharing the same execution resources, the two streams may run at roughly half the speed of a single stream. If a process in a stream running on one of the logical cores stalls, the second stream on that core can use the stalled stream’s execution resources and possibly recoup cycles that would have been idle if the streams has been run with only one per physical core. Hyper Threading on Eos is supported by each of the available compilers — Intel, PGI, Cray and GNU.

Note: Hyper-Threading is enabled on Eos by default. The -j1 option to aprun explicitly disables Hyper Threading on Eos.
Hyper Threading for MPI Applications

For MPI applications, Hyper Threading can be utilized in a few different ways. One way is by running on half the nodes that you would typically need to allocate without Hyper Threading. The example below shows the code to do so and the resulting task layout on a node.

Code Example 1: (32) MPI tasks, (1) task per “core”, with Hyper Threading

# Request 1 node.
#PBS -l nodes=1# Tell aprun to use hyper threading.

# Implicitly using HT (default)
aprun -n 32 ./a.out

# Explicitly using HT
aprun -n 32 -j2 ./a.out

Compute Node 0
NUMA 0 NUMA 1
C0 C1 C2 C3 C4 C5 C6 C7 C0 C1 C2 C3 C4 C5 C6 C7
0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23
C8 C9 C10 C11 C12 C13 C14 C15 C8 C9 C10 C11 C12 C13 C14 C15
8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31

[1] ‘C[n]’ indicates ‘Core [n]’ for the NUMA node.

In contrast, the same example without Hyper Threading is shown below along with the resulting task layout on a node.

Code Example 2: (32) MPI tasks without Hyper Threading
# Request 2 nodes.
#PBS -l nodes=2# No aprun hyper threading.
aprun -n 32 -j1 ./a.out

Compute Node 0
NUMA 0 NUMA 1
C0 C1 C2 C3 C4 C5 C6 C7 C0 C1 C2 C3 C4 C5 C6 C7
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Compute Node 1
NUMA 0 NUMA 1
C0 C1 C2 C3 C4 C5 C6 C7 C0 C1 C2 C3 C4 C5 C6 C7
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

[1] ‘C[n]’ indicates ‘Core [n]’ for the NUMA node.

Hyper Threading for Threaded Codes

For threaded codes, Hyper Theading can allow you to run with double the number of threads per MPI task for a fixed number of nodes and tasks.

For example, an MPI/OpenMP code designed to run with (2) MPI tasks and (8) threads per task without hyper threading via:

 aprun -n2 -d8 -j1

Could be run with (16) threads per task with Hyper Threading:

 aprun -n2 -d16