Processing and Analysis of Very Large Data Sets
This workshop focuses on the whole lifetime of large datasets. From job prep, to jobs, to analysis, this workshop will help you better deal with large data from acquisition to publication. Planned topics include:
– How do I know if I have BIG data?
– What you should use for large data prep and analysis
– Why shuffling data during a job kills performance and how you can improve it
– Libraries: better ways to do parallel I/O
– Are all file formats the same?
– How do I begin to visualize enormous datasets?
– In-situ analysis: a how-to
– Sharing your massive data with friends and strangers
– Future outlook on a growing data problem
Hands on tutorials are also planned for various scripting languages, parallel I/O libraries, and viz and analysis tools.
[tab: Tentative Agenda]| August 6th, 2013 | ||
|---|---|---|
| 08:30 AM | Breakfast | |
| 09:00 AM | Welcome and Intro: The Era of Data Intensive Science | Fernanda Foertter, HPC User Assistance Specialist @ORNL |
| 09:15 AM | ATLAS – Big Data and the Higgs | Alexei Klimentov@BNL |
| 09:45 AM | I/O Challenges in Bioinformatics at the Joint Genome Institute | Kjiersten Fagnan, Bioinformatics Consultant @LBL |
| 10:15 AM | Apache Hadoop in Spider Filesystem-Lim | Seung-Hwan Lim, @ORNL |
| 10:45 AM | OLCF-Data-Intro-IO-Gerber-FINAL | Richard Gerber, Senior Science Advisor @NERSC |
| 11:15 AM | Globus_Online-Ananthakrishnan | Rachana Ananthakrishnan, Sr. Engagement Manager/Solutions Architect, University of Chicago |
| 11:45 AM | Working Lunch: Training account setup and system intro | |
| 01:00 PM | ADIOS_Tutorial-Podhorski | Norbert Podhorszki, Computer Scientist @ORNL |
| 02:00 PM | Hands on Tutorials: ADIOS Read/Write API 1:Write data with ADIOS 2:Tools 3:Read data with ADIOS |
Part I |
| 03:15 PM | Break | |
| 03:30 PM | Hands on Tutorials: ADIOS Write API (Non-XML version) 4:Multi-block writing (like what AMR codes do) 5:Spatial aggregation 6:Staging example 7:Visualization of ADIOS data 8:I/O skeleton generation with Skel |
Part II |
| 05:00 PM | Closing Remarks | Fernanda Foertter, HPC User Assistance Specialist @ORNL |
| August 7th, 2013 | ||
|---|---|---|
| 08:30 AM | Breakfast | |
| 09:00 AM | Intro and Preview | Fernanda Foertter, HPC User Assistance Specialist @ORNL |
| 09:15 AM | Data Analysis and Visualization of Big Data from HPC-Ahern | Sean Ahern, Chief Data Officer @UT/JICS |
| 09:45 AM | Developing and modifying ParaView to create a plugin for very large transparent datasets | John Biddiscombe, Computational Scientist @CSCS |
| 10:15 AM | Data Management and Analysis in Support of DOE Climate Science-Shipman | Galen Shipman, Director of Compute and Data Environment for Science @ORNL |
| 10:45 AM | Analyzing Big Data with ParaView: Going Beyond the Builtin Server |
Marcus D. Hanwell, Technical Leader, Scientific Computing Group, @Kitware |
| 11:15 AM | Large Scale Spatiotemporal Data Mining-Vatsavai | Raju Vatsavai, Senior Research Scientist @ORNL |
| 11:45 AM | Working Lunch: Viz tutorial setup and prep | |
| 01:00 PM | Visualization of Extremely Large Datasets using VisIt | Dave Pugmire, Computer Scientist @ORNL |
| 01:30 PM | Hands on Tutorials: Scientific Visualization VisIt, Dave Pugmire, Computer Scientist @ORNL Sean Ahern, Chief Data Officer @UT/JICS |
Part I |
| 03:45 PM | Break | |
| 04:00 PM | Hands on Tutorials: Scientific Visualization Paraview, Marcus D. Hanwell, Technical Leader, Scientific Computing Group, @Kitware |
Part II |
| 06:00 PM | Closing Remarks | Fernanda Foertter, HPC User Assistance Specialist @ORNL |
| 07:00 PM | Networking Event | Sponsored by Data Direct Networks |
| August 8th, 2013 | ||
|---|---|---|
| 08:30 AM | Breakfast | |
| 08:50 AM | Intro and Preview | Fernanda Foertter, HPC User Assistance Specialist @ORNL |
| 09:00 AM | Toward Real-Time Analysis in Neutron Science-Shipman | Galen Shipman, Director of Compute and Data Environment for Science @ORNL |
| 09:30 AM | A Survey of the State-of-the-art in Checkpointing-Vazhkudai | Sudharshan Vazhkudai, Group Leader Technology Integration, @ORNL |
| 10:00 AM | OLCF Data Visualization Laboratory | Jamison Daniel, Scientific Visualization Staff, @ORNL |
| Panel: Scientific Data Management: The Road Ahead | ||
| 10:30 AM | Data @ OLCF: Current Policies and Future Opportunities | Jack Wells, Director of Science @ORNL |
| 10:45AM | Data Management Resources for Future Data Life Cycles | Sudharshan Vazhkudai, Group Leader Technology Integration, @ORNL |
| 11:00 AM | Data Complexity, Heterogeneity and Metadata: A Climate Science Perspective | Valentine Anantharaj, Computational Climate Scientist, @ORNL |
| 11:15 AM | Data-Driven Science at NERSC | Richard Gerber, Senior Science Advisor, @LBNL |
| 11:30 AM | Panel Q&A | |
| 12:00 PM | Working Lunch: Tools tutorial setup | |
| 12:45 PM | PBD-R: Programming with big data in R | George Ostrouchov, Senior Researcher in the Scientific Data Group @ORNL |
| 01:15 PM | Hands on Tutorials: ToolsPBD-R, Drew Schmidt, Research Associate @UTK | Part I |
| 03:00 PM | Break | |
| 03:15 PM | Hands on Tutorials: Tools03:15 PM Python tools overview, Arnold Tharrington, Computational Scientist @OLCF03:45 PM iPython, Robert French, HPC User Assistance Specialist @OLCF
04:15 PM Using Globus Online, Suzanne Parete-Koon, HPC User Assistance Specialist @OLCF 04:45 PM Visualization using JavaScript libraries, Omar ElTayeby, @NICS |
Part II |
| 05:15 PM | Closing Remarks | Fernanda Foertter, HPC User Assistance Specialist @ORNL |
!!On-site Registration closes July 26th!!
[tab: Hotel info]We have arranged for a room block at the Hilton Hotel Knoxville at the special group rate of $86.00 Each individual will be responsible for his/her own reservation. Reservations may be made by the following:
Room Group code: PAVL
HOTEL: 865-523-2300 (8:00 AM – 4:00 PM)
DIRECT: 865-251-2578 (8:00 AM – 4:00 PM)
TOLL-FREE: 800-HILTONS
WEBSITE: www.hilton.com
All reservations must be guarantee by either a credit card or a first night’s deposit.
501 West Church Avenue, Knoxville, TN 37902
 |
Rated High
Upscale, full-service, downtown Knoxville hotel 0.8 mile from UTK University of Tennessee Knoxville Map of hotel and UTK University of Tennessee Knoxville Pet-friendly hotel
|
