CUDA: Atomics, Reductions, and Warp Shuffle

CUDA: Atomics, Reductions, and Warp Shuffle
Wednesday, May 13, 2020

NOTE: The format of this event has been changed to online only. NVIDIA will present remotely for the first ~1 hour and the remote connection will be left open for the hands-on session, where representatives from OLCF, NERSC, and NVIDIA will be available to support participants.
ORNL Remote Participation Only 1:00 PM – 3:00 PM (ET)
NERSC Remote Participation Only 10:00 AM – 12:00 PM (PT)

On Wednesday, May 13, 2020, NVIDIA will present part 5 of a 9-part CUDA Training Series titled “Atomics, Reductions, and Warp Shuffle”.

This CUDA programming model does not enforce any order of thread execution. This requires attention when performing operations like reductions on the GPU. This module introduces atomic operations, parallel reductions, and warp shuffle, which can help you in cases where exposing parallelism is not immediately obvious due to potential thread contention. After the presentation, there will be a session where participants can complete example exercises meant to reinforce the presented concepts.

Remote Participation
Remote participants can watch the presentations via web broadcast and will have access to the training exercises, but temporary access to the compute systems will be limited as follows:

  • Current NERSC users will have Cori-GPU access temporarily added to their accounts.
  • Temporary Summit access will not be available for remote participants.

NOTE: Registration is required. Please submit the form below.

If you have any questions, please contact Tom Papatheodore (

Zoom Connection Details:
Meeting ID: 510 486 5180
Call In: +1 669 900 6833
(slides | recording)
The recording will be made available here shortly after the event.
The example exercises for this module can be found in the exercises/hw5 folder of the following GitHub repo:
The exercise for this module of the training series is expected to take longer than the time allocated for the hands-on session, so NVIDIA has kindly offered to walk through the exercise during a separate session. Please see the details below.

Module 5 Exercise Walk-Through
May 19, 2020
3:30 – 4:30 PM (ET)

The recording of the exercise walk-through can be found here.

Zoom Connection Details:
Meeting ID: 510 486 5180
Call In: +1 669 900 6833


May 13 2020


Thomas Papatheodore
1 (865) 576-1244
QR Code

Comments are closed.