titan

Up since 11/8/17 02:45 pm

eos

Up since 11/14/17 11:20 pm

rhea

Up since 10/17/17 05:40 pm

hpss

Up since 11/20/17 09:15 am

atlas1

Up since 11/15/17 07:25 am

atlas2

Up since 11/14/17 09:20 am
OLCF User Assistance Center

Can't find the information you need below? Need advice from a real person? We're here to help.

OLCF support consultants are available to respond to your emails and phone calls from 9:00 a.m. to 5:00 p.m. EST, Monday through Friday, exclusive of holidays. Emails received outside of regular support hours will be addressed the next business day.

Loading Events

Processing and Analysis of Very Large Data Sets

Details

Start:
August 6, 2013 @ 8:00 am
End:
August 8, 2013 @ 5:00 pm

Venue

Hilton Hotel Knoxville
501 West Church Avenue
Knoxville, TN 37902 United States
+ Google Map
Phone:
(865) 251-2573

Organizer

Fernanda Foertter
Phone:
(865) 576-9391
Email:
foertterfs@ornl.gov

This workshop focuses on the whole lifetime of large datasets. From job prep, to jobs, to analysis, this workshop will help you better deal with large data from acquisition to publication.  Planned topics include:

– How do I know if I have BIG data?
– What you should use for large data prep and analysis
– Why shuffling data during a job kills performance and how you can improve it
– Libraries: better ways to do parallel I/O
– Are all file formats the same?
– How do I begin to visualize enormous datasets?
– In-situ analysis: a how-to
– Sharing your massive data with friends and strangers
– Future outlook on a growing data problem

Hands on tutorials are also planned for various scripting languages, parallel I/O libraries, and viz and analysis tools.

Tentative Agenda

August 6th, 2013
08:30 AM Breakfast
09:00 AM Welcome and Intro: The Era of Data Intensive Science Fernanda Foertter, HPC User Assistance Specialist @ORNL
09:15 AM ATLAS – Big Data and the Higgs Alexei Klimentov@BNL
09:45 AM I/O Challenges in Bioinformatics at the Joint Genome Institute Kjiersten Fagnan, Bioinformatics Consultant @LBL
10:15 AM Apache Hadoop in Spider Filesystem-Lim Seung-Hwan Lim, @ORNL
10:45 AM OLCF-Data-Intro-IO-Gerber-FINAL Richard Gerber, Senior Science Advisor @NERSC
11:15 AM Globus_Online-Ananthakrishnan Rachana Ananthakrishnan, Sr. Engagement Manager/Solutions Architect, University of Chicago
11:45 AM Working Lunch: Training account setup and system intro
01:00 PM ADIOS_Tutorial-Podhorski Norbert Podhorszki, Computer Scientist @ORNL
02:00 PM Hands on Tutorials: ADIOS Read/Write API
1:Write data with ADIOS
2:Tools
3:Read data with ADIOS
Part I
03:15 PM Break
03:30 PM Hands on Tutorials: ADIOS Write API (Non-XML version)
4:Multi-block writing (like what AMR codes do)
5:Spatial aggregation
6:Staging example
7:Visualization of ADIOS data
8:I/O skeleton generation with Skel
Part II
05:00 PM Closing Remarks Fernanda Foertter, HPC User Assistance Specialist @ORNL
August 7th, 2013
08:30 AM Breakfast
09:00 AM Intro and Preview Fernanda Foertter, HPC User Assistance Specialist @ORNL
09:15 AM Data Analysis and Visualization of Big Data from HPC-Ahern Sean Ahern, Chief Data Officer @UT/JICS
09:45 AM Developing and modifying ParaView to create a plugin for very large transparent datasets John Biddiscombe, Computational Scientist @CSCS
10:15 AM Data Management and Analysis in Support of DOE Climate Science-Shipman Galen Shipman, Director of Compute and Data Environment for Science @ORNL
10:45 AM Analyzing Big Data with
ParaView: Going Beyond the Builtin Server
Marcus D. Hanwell, Technical Leader, Scientific Computing Group, @Kitware
11:15 AM Large Scale Spatiotemporal Data Mining-Vatsavai Raju Vatsavai, Senior Research Scientist @ORNL
11:45 AM Working Lunch: Viz tutorial setup and prep
01:00 PM Visualization of Extremely Large Datasets using VisIt Dave Pugmire, Computer Scientist @ORNL
01:30 PM Hands on Tutorials: Scientific Visualization
VisIt, Dave Pugmire, Computer Scientist @ORNL
Sean Ahern, Chief Data Officer @UT/JICS
Part I
03:45 PM Break
04:00 PM Hands on Tutorials: Scientific Visualization
Paraview, Marcus D. Hanwell, Technical Leader, Scientific Computing Group, @Kitware
Part II
06:00 PM Closing Remarks Fernanda Foertter, HPC User Assistance Specialist @ORNL
07:00 PM Networking Event Sponsored by Data Direct Networks
August 8th, 2013
08:30 AM Breakfast
08:50 AM Intro and Preview Fernanda Foertter, HPC User Assistance Specialist @ORNL
09:00 AM Toward Real-Time Analysis in Neutron Science-Shipman Galen Shipman, Director of Compute and Data Environment for Science @ORNL
09:30 AM A Survey of the State-of-the-art in Checkpointing-Vazhkudai Sudharshan Vazhkudai, Group Leader Technology Integration, @ORNL
10:00 AM OLCF Data Visualization Laboratory Jamison Daniel, Scientific Visualization Staff, @ORNL
Panel: Scientific Data Management: The Road Ahead
10:30 AM Data @ OLCF: Current Policies and Future Opportunities Jack Wells, Director of Science @ORNL
10:45AM Data Management Resources for Future Data Life Cycles Sudharshan Vazhkudai, Group Leader Technology Integration, @ORNL
11:00 AM Data Complexity, Heterogeneity and Metadata: A Climate Science Perspective Valentine Anantharaj, Computational Climate Scientist, @ORNL
11:15 AM Data-Driven Science at NERSC Richard Gerber, Senior Science Advisor, @LBNL
11:30 AM Panel Q&A
12:00 PM Working Lunch: Tools tutorial setup
12:45 PM PBD-R: Programming with big data in R George Ostrouchov, Senior Researcher in the Scientific Data Group @ORNL
01:15 PM Hands on Tutorials: ToolsPBD-R, Drew Schmidt, Research Associate @UTK Part I
03:00 PM Break
03:15 PM Hands on Tutorials: Tools03:15 PM Python tools overview, Arnold Tharrington, Computational Scientist @OLCF03:45 PM iPython, Robert French, HPC User Assistance Specialist @OLCF

04:15 PM Using Globus Online, Suzanne Parete-Koon, HPC User Assistance Specialist @OLCF

04:45 PM Visualization using JavaScript libraries, Omar ElTayeby, @NICS

Part II
05:15 PM Closing Remarks Fernanda Foertter, HPC User Assistance Specialist @ORNL

Registration

!!On-site Registration closes July 26th!!

Registration is now closed.

  Hotel info

We have arranged for a room block at the Hilton Hotel Knoxville at the special group rate of $86.00  Each individual will be responsible for his/her own reservation.  Reservations may be made by the following:

Room Group code:   PAVL 
HOTEL:   865-523-2300 (8:00 AM – 4:00 PM)
DIRECT: 865-251-2578 (8:00 AM – 4:00 PM)
TOLL-FREE:         800-HILTONS
WEBSITE:           www.hilton.com

All reservations must be guarantee by either a credit card or a first night’s deposit.

Hilton Hotel Knoxville – $$$
501 West Church Avenue, Knoxville, TN 37902
Rated High
Upscale, full-service, downtown Knoxville hotel
0.8 mile from UTK University of Tennessee Knoxville
Map of hotel and UTK University of Tennessee Knoxville
Pet-friendly hotel

TripAdvisor Traveler Rating: 4.0 of 5 stars
Based on 268 reviews Read Recent Reviews