ParaView

ParaView is an open-source, multi-platform data analysis and visualization application. ParaView users can quickly build visualizations to analyze their data using qualitative and quantitative techniques. The data exploration can be done interactively in 3D or programmatically using ParaView’s batch processing capabilities.

ParaView was developed to analyze extremely large datasets using distributed memory computing resources. The OLCF provides ParaView server installs on Rhea to facilitate large scale distributed visualizations. The ParaView server running on Rhea may be used in a headless batch processing mode or be used to drive a ParaView GUI client running on your local machine.

ParaView client

A ParaView client instance is not available on Rhea. Interactive mode requires that your local machine have a version matched ParaView client installation and batch mode can benefit from a local installation as well to aid in script generation. Precompiled ParaView binaries for Windows, Macintosh, and Linux can be downloaded from Kitware.

Interactive mode

Although in a single machine setup both the ParaView client and server run on the same host, this need not be the case. It is possible to run a local ParaView client to display and interact with your data while the ParaView server runs in a Rhea batch job, allowing interactive analysis of very large data sets.

In interactive mode your local ParaView version number must match the ParaView version number available on Rhea. Please check the available ParaView versions using Rhea’s modules system.

Interactive Example

The following provides an example of launching the ParaView server on Rhea and connecting to it from a locally running ParaView client. Although several methods may be used the one described should work in most cases.

For Macintosh clients, it is necessary to install XQuartz (X11) to get a command prompt in which you will securely enter your OLCF credentials.
For Windows clients, it is necessary to install PuTTY to create an ssh connection in step 2.

Step 1: Launch ParaView on your Desktop and fetch a connection script for Rhea
Start ParaView and then select File/Connect to begin.

Next Select Fetch Servers

Next select the connection to Rhea for either windows or Mac/Linux and hit the “Import Selected” button.


You may now quit and restart ParaView in order to save connection setup in your preferences.

Step 2: Establish a connection to Rhea
Once restarted, and henceforth, simply select Rhea from the File->Connect dialog and click the “Connect” button.

A dialog box follows, in which you must enter in your username and project allocation, the number of nodes to reserve and a duration to reserve them for.


When you click OK, a windows command prompt or xterm pops up. In this window enter your credentials at the OLCF login prompt.


When your job reaches the top of the queue, the RenderView1 view window will return. At this point you are connected to Rhea and can open files that reside there and visualize them interactively.

VisIt

VisIt is a interactive, parallel analysis and visualization tool for scientific data. VisIt contains a rich set of visualization features so you can view your data in a variety of ways. It can be used to visualize scalar and vector fields defined on two- and three-dimensional (2D and 3D) structured and unstructured meshes.

Installing and Setting Up VisIt

VisIt uses a client-server architecture. You will obtain the best performance by running the VisIt client on your local computer and running the server on OLCF resources. VisIt for your local computer can be obtained here: VisIt Installation. Rhea currently has Remote Backend Version 2.13.0 available, so the local client version 2.13.x is recommended.

The first time you launch VisIt (after installing), you will be prompted for a remote host preference. Unfortunately, ORNL does not maintain this list and the ORNL entry is outdated. Click the “None” option instead. Restart VisIt, and go to Options->Host Profiles. Select “New Host”

  • For host nickname: Rhea (this is arbitrary)
  • Remote hostname: rhea.ccs.ornl.gov (required)
  • Host name aliases: rhea-login#g (required)
  • Maximum Nodes: unchecked, unless using the GPU partition on Rhea)
  • Maximum processors: unchecked (arbitrary but use fewer than cores available)
  • Path to VisIt Installation: /sw/rhea/visit (required)
  • Username: Your OLCF Username (required)
  • Tunnel data connections through SSH: Checked (required)

Under the “Launch Profiles” tab create a launch profile. Most of these values are arbitrary

  • Profile Name: No GPU, MPI, Multinode (arbitrary)
  • Timeout: 480 (arbitrary)
  • Number of threads per task: 0 (arbitrary, but not tested with OMP/pthread support)
  • Additional arguments: blank (arbitrary)

Under the “Parallel” Tab

  • Launch parallel engine: Checked (required)
  • Launch Tab:
    • Parallel launch method: qsub/mpirun (required)
    • Partition/Pool/Queue: batch (required)
    • Number of processors: 2 (arbitrary, but high number may lead to OOM errors)
    • Number of nodes: 2 (arbitrary)
    • Bank/Account: Your OLCF project to use (required)
    • Time Limit: 1:00:00 (arbitrary)
    • Machine file: Unchecked (required – Lets VisIt get the nodelist from the scheduler)
    • Constraints: unchecked
  • Advanced tab – All boxes unchecked
  • GPU Acceleration
    • Use cluster’s graphics cards: Unchecked

    Click “Apply”. Exit and re-launch VisIt.

    Usage

    Once you have VisIt installed and set up on your local computer:

    • Open VisIt on your local computer.
    • Go to: “File→Open file” or click the “Open” button on the GUI.
    • Click the “Host” dropdown menu on the “File open” window that popped up and choose “Rhea”.
    • This will prompt you for your OLCF passcode, and connect you to Rhea.
    • Navigate to the appropriate file.
    • Once you choose a file, you will be prompted for the number of nodes and processors you would like to use (remember that each node of Rhea contains 16 processors) and the Project ID, which VisIt calls a “Bank” as shown below.
    • Use the Bank field to enter your project ID

    • Once specified, the server side of VisIt will be launched, and you can interact with your data.

    Please do not run VisIt’s client from an OLCF machine. You will get much better performance if you install a client on your workstation and launch locally. You can directly connect to OLCF machines from inside VisIt and access your data remotely.

    For additional resources, please see the VisIt Wiki.

    Troubleshooting

    VisIt keeps asking for your password.

    If VisIt keeps asking for your “Password” in the dialog box below, and you are entering your correct PIN + RSA token code, you might need to select “Change username” and then enter your OLCF username when prompted.

    Screen Shot 2016-01-06 at 11.10.19 AM

    This will give you a new opportunity to enter your PIN + token code and your username will appear in login request box as shown below. If you want you OLCF username to be filled in by default, go to “Options→Host profiles” and enter it under “Username”. See the Modifying Host Profiles section below for more details.

    Screen Shot 2016-01-06 at 11.06.25 AM

    VisIt will not connect when you try to draw an image.

    If VisIt will not connect to Rhea when you try to draw an image, you should login to Rhea and enter “qstat” from the command line. Your VisIt job should appear in the queue. If you see it in a state marked “Q” you should wait a bit longer to see if it will start. If you do not see your job listed in the queue, check to make sure your project ID is entered in your VisIt host profile. See the Modifying Host Profiles section below for instructions.

    Modifying Host Profiles

    To make changes to an exiting host profile, do the following:

    • Go to “Options→Host Profiles”.
    • The window will display the known hosts on the left, with the settings for that host shown on the right in the “Host Settings” tab.
    • You can modify settings relevant to this host machine. For example, you can change the “Username” field if your OLCF username differs from your local computer username.
    • Once you have made your changes, press the “Apply” button, and then save the settings (Options/Save Settings).

    Each host can have several launch profiles. A launch profile specifies VisIt can be run on a given host computer. To make changes to a host’s launch profile, do the following:

    • Go to “Options→Host Profiles”.
    • Select the host in the left side of the window.
    • Select the “Launch Profiles” tab in the right side of the window. This will display the known launch profiles for this host.
    • Select a “Launch Profile” and the settings are displayed in the tabs below.
    • You can set your Project ID in the “Default Bank/Account” field in the “Parallel” tab.
    • You can change the queue used by modifying the “Partition/Pool/Queue” field in the “Parallel” tab.
    • Once you have made your changes, press the “Apply” button, and then save the settings (Options/Save Settings).

    Remote Visualization using VNC (non-GPU)

    In addition to the instructions below, Benjamin Hernandez of the OLCF Advanced Data and Workflows Group presented a related talk, GPU Rendering in Rhea and Titan, during the 2016 OLCF User Meeting.

    step 1 (local system)

    Install a vncviewer (turbovnc, tigervnc, etc.) on your local machine. When running vncviewer for the first time, it will ask to set a password for this and future vnc sessions.

    step 2 (terminal 1)

    From a Rhea connection launch a batch job and execute the below matlab-vnc.sh script to start the vncserver and run matlab within:

    1. localsytem: ssh -X @rhea.ccs.ornl.gov
    2. rhea: qsub -I -A abc123 -X -l nodes=1,walltime=01:00:00
    3. rhea: ./matlab-vnc.sh
    ./matlab-vnc.sh
    New 'rhea6:1 (userA)' desktop is rhea6:1
     
    Starting applications specified in /ccs/home/userA/.vnc/xstartup
    Log file is /ccs/home/userA/.vnc/rhea6:1.log
      
    **************************************************************************
    Instructions
     
    In a new terminal, open a tunneling connection with rhea6 and port 5901
    example:
             userid@rhea.ccs.ornl.gov -L 5901:rhea6:5901
     **************************************************************************
      
    MATLAB is selecting SOFTWARE OPENGL rendering.
    

    step 3 (terminal 2)

    In a second terminal on your local system open a tunneling connection following the instructions given by the vnc start-up script:

    • localsystem: ssh @rhea.ccs.ornl.gov -L 5901:rhea99:5901

    step 4 (local system)

    Launch the vncviewer. When you launch the vncviewer that you downloaded you will need to specify ‘localhost:5901’. You will also set a passoword for the initial connection or enter the created password for subsequent connections.

    matlab-vnc.sh (non-GPU rendering)

    #!/bin/sh
    
    what()
    {
    	hostname
    }
    echo "Starting vncserver"
    
    vncserver :1 -geometry 1920x1080  -depth 24
    
    echo
    echo
    echo "**************************************************************************"
    echo "Instructions"
    echo 
    echo "In a new terminal, open a tunneling connection with $(what) and port 5901"
    echo "example:"
    echo  "         userid@rhea.ccs.ornl.gov -L 5901:$(what):5901 "
    echo
    echo "**************************************************************************"
    echo
    echo
    export DISPLAY=:1
    . /sw/rhea/environment-modules/3.2.10/rhel6.7_gnu4.4.7/init/sh
    module load matlab/R2016b
    matlab
    vncserver -kill :1
    

    Remote Visualization using VNC (GPU nodes)

    Available April 29

    step 1 (local system)

    Install a vncviewer (turbovnc, tigervnc, etc.) on your local machine. When running vncviewer for the first time, it will ask to set a password for this and future vnc sessions.

    step 2 (terminal 1)

    From a Rhea connection launch a batch job and execute the below matlab-vnc.sh script to start the vncserver and run matlab within:

    1. localsytem: ssh -X @rhea.ccs.ornl.gov
    2. rhea: qsub -I -A abc123 -X -l nodes=1,walltime=01:00:00 -lpartition=gpu
    3. rhea: ./matlab-vnc.sh
    ./matlab-vnc.sh
    New 'rhea6:1 (userA)' desktop is rhea6:1
     
    Starting applications specified in /ccs/home/userA/.vnc/xstartup
    Log file is /ccs/home/userA/.vnc/rhea6:1.log
      
    **************************************************************************
    Instructions
     
    In a new terminal, open a tunneling connection with rhea6 and port 5901
    example:
             userid@rhea.ccs.ornl.gov -L 5901:rhea6:5901
     **************************************************************************
      
    MATLAB is selecting SOFTWARE OPENGL rendering.
    

    step 3 (terminal 2)

    In a second terminal on your local system open a tunneling connection following the instructions given by the vnc start-up script:

    • localsystem: ssh @rhea.ccs.ornl.gov -L 5901:rhea99:5901

    step 4 (local system)

    Launch the vncviewer. When you launch the vncviewer that you downloaded you will need to specify ‘localhost:5901’. You will also set a passoword for the initial connection or enter the created password for subsequent connections.

    vmd-vgl.sh (GPU rendering)

    #!/bin/sh
    
    what()
    {
    	hostname
    }
    echo
    echo "Starting X"
    xinit &
    sleep 5
    echo "Starting vncserver"
    
    vncserver :1 -geometry 1920x1080  -depth 24
    
    echo
    echo
    echo "**************************************************************************"
    echo "Instructions"
    echo 
    echo "In a new terminal, open a tunneling connection with $(what) and port 5901"
    echo "example:"
    echo  "         userid@rhea.ccs.ornl.gov -L 5901:$(what):5901 "
    echo
    echo "**************************************************************************"
    echo
    echo
    export DISPLAY=:1
    module load vmd/1.9.3
    vglrun vmd
    vncserver -kill :1
    

    Remote Visualization using Nice DCV (GPU nodes only)

    step 1 (terminal 1)

    Launch an interactive job:

     qsub -I -A projectID   -l nodes=1 -l walltime=00:30:00 -l partition=gpu
    
    As of April 29, the dcv feature will be required:

     qsub -I -A projectID   -l nodes=1 -l walltime=00:30:00 -l partition=gpu -l feature=dcv
    

    Run the following commands:

    xinit &
    export DISPLAY=:0    
    dcv create-session --gl-display :0 mySessionName 
    hostname  // will be used to open a tunneling connection with this node
    

    step 1 (terminal 2)

    Open a tunneling connection with gpu node N, given by hostname:

    ssh user@rhea.ccs.ornl.gov -L 8443:rhea-gpuN:8443
    

    Open your web browser using the following link and use your credentials to access OLCF systems: https://localhost:8443

    When finished, kill the dcv session in first terminal:

    dcv close-session mySessionName
    kill %1