Using R on HPC Clusters Webinar
This Webinar tutorial helps users learn a basic workflow for how to use R on an HPC cluster. The tutorial will focus on parallel computing as a means to speed up R scripts on a cluster computer. Many packages in R offer some form of parallel computing yet they rely on a much smaller set of underlying approaches: multithreading in compiled code, the unix fork, and MPI. The tutorial will take a narrow path to focus on packages that directly engage the underlying approaches, yet are easy to use at a high-level.
- Learn how to work with GitHub in RStudio. Create a GitHub (or ORNL GitLab) account, create a repository and practice how to work with it from RStudio. Many tutorials are available on the web, for example by RStudio.
- Learn a few basic unix commands for listing files, creating a directory, removing files, etc. Lots of places to learn, for example Unix Shell Crash Course
We will run R as batch jobs on the clusters. The workflow will be:
Edit your code in RStudio -> push the code to GitHub/GitLab -> pull the code to the cluster and submit as batch -> look at your output and circle back to Edit.
This has the advantage of editing in a familiar environment and running in a common teaching environment. Other workflows are possible if you already know the tools.
Day 1: Friday, July 20, 9:00am-12:00pm
Parallel hardware and software overview and ways to use multiple cores on a single node: mclapply (fork), multithreaded BLAS
Day 2: Wednesday, July 27, 9:00am-12:00pm
Hardware review and using multiple nodes: MPI at high level via pbdMPI, matrix methods via kazaam and pbdDMAT
- Users who have access to and OLCF project may do the exercises on Andes. Participants who do not have an OLCF project may attend the lectures but will not have access to Andes.
- Users must have R and R studio installed on their laptops.
- Participants must have git installed on their laptops
- Participants must be able to ssh to a remote machine
- For Mac use Terminal
- For Windows user Putty
- See: https://github.com/olcf/foundational_hpc_skills/blob/master/intro_to_ssh/Intro_to_ssh_clients.pdf