Data Transfer with Rhea
Print this article
Data Transfer Methods for Rhea
Rhea mounts the User Home and Spider Atlas shared center-wide filesystems. Data transferred by any means to these filesystems will be available on Rhea. It is strongly recommended that large data transfers use
bbcp from a Data Transfer Node to transfer files to Rhea.
scp utility is useful when transferring small amounts of data. The following examples show how to transfer a file named
data.tar to a Rhea-mounted filesystem.
scp command was executed from a non-OLCF resource to copy a file to the
$MEMBERWORK directory on Atlas:
$ scp /path/to/data.tar [userid]@dtn.ccs.ornl.gov:/lustre/atlas/scratch/[userid]/[projid]/data-on-Atlas.tar
[projid] are replaced with your OLCF username and project. The directories under
$WORLDWORK can be written to using their fully-expanded paths
More information on the
scp utility can be found at the SFTP/SCP page.
For User Home areas, Rhea mounts the same
$HOME directories that the other OLCF systems mount, which means that the files you see in
/ccs/home/$USER on Titan or the Data Transfer Nodes are also present on Rhea at the same location.
For your work directories, a change is needed to run on Rhea. Let’s assume that your workflow for running your application involves copying files from your User Home area (i.e.
/ccs/home/$USER) to your Atlas Work area (i.e.
$MEMBERWORK). To do this on Rhea, you would need to copy those files to, for instance, your Member Work area at
[projid] is the identifier for one of the projects associated with your OLCF account.
Parallel File Transfer
The HPSS archival filesystem can be used to get and put files on Atlas. This is only an efficient option if the file you want to transfer is already on the HPSS. To move the file
Inital_conditions.tar from HPSS to Rhea, assuming that the current working directory is your Member Work area:
$ hsi get Initial_conditions.tar
More information on using
hsi can be found on the HSI page.