Note: this page is still live for archive purposes. The projects selected for the Summit Early Science program are listed here: Summit Early Science.
The OLCF is now accepting proposals for Early Science projects on Summit. The goals of the Early Science program are threefold. First, this will be an opportunity to realize early scientific achievements on what may well be the largest supercomputer for open science. Second, the Early Science projects will demonstrate the scalability and performance of applications ported to the Summit architecture. Third, the OLCF will benefit from the “hardening” of both the hardware and software environment using production-ready codes at scale in important mission-relevant scientific computationally challenging projects. The Early Science projects will have access to Summit from the time it has finished the acceptance testing period until the system is transitioning to the INCITE and ALCC user programs. While this will be a relatively short period of time, the capability of Summit is such that each project will have sufficient computational resources to carry out a challenging science campaign.
The Oak Ridge Leadership Computing Facility (OLCF) will deploy its next, pre-exascale system Summit in 2018. Summit is an IBM/NVIDIA system capable of over 200 petaflops delivered by ~4600 nodes with two IBM POWER9 processors and six NVIDIA V100 accelerators per node. Each node will have 512 GB of DDR4 memory, 96 GB of HBM2 memory connected to the GPUs, and 1600 GB of non-volatile memory, resulting in over 10 PB of total system memory. The node interconnect is a dual-rail EDR InfiniBand interconnect from Mellanox. Disk storage is provided by a 250 PB IBM GPFS filesystem.
| Feature | Titan | Summit |
|---|---|---|
| Application Performance | Baseline | 5-10x Titan |
| Number of Nodes | 18,688 | ~4,600 |
| Node Performance | 1.4 TF | > 40 TF |
| Memory per Node | 32 GB DDR3 + 6 GB GDDR5 | 512 GB DDR4 + 96 GB HBM2 |
| NV memory per Node | 0 | 1600 GB |
| Total System Memory | 710 TB | >10 PB DDR4 + HBM2 + Non-volatile |
| System Interconnect (node injection bandwidth) | Gemini (6.4 GB/s) | Dual Rail EDR-IB (25 GB/s) |
| Interconnect Topology | 3D Torus | Non-blocking Fat Tree |
| Processors | 1 AMD Opteron™ 1 NVIDIA Kepler™ | 2 IBM POWER9™ 6 NVIDIA Volta™ |
| File System | 32 PB, 1 TB/s, Lustre® | 250 PB, 2.5 TB/s GPFS™ |
| Peak power consumption | 9 MW | 15 MW |
The goals of the Early Science program are threefold. First, this will be an opportunity to realize early scientific achievements on what may well be the largest supercomputer for open science. Second, the Early Science projects will demonstrate the scalability and performance of applications ported to the Summit architecture. Third, the OLCF will benefit from the “hardening” of both the hardware and software environment using production-ready codes at scale in important mission-relevant scientific computationally challenging projects.
The Early Science projects will have access to Summit from the time it has finished the acceptance testing period until the system is transitioning to the INCITE and ALCC user programs. While this will be a relatively short period of time, the capability of Summit is such that each project will have sufficient computational resources to carry out a challenging science campaign.
Contact Information
Please direct any questions about this call for proposals by email to: [email protected]
Timeline for Summit Early Science
The anticipated timeline for Summit Early Science is as follows. The actual timeline will vary depending upon the realities and timing of the Summit deployment.
- December 15, 2017: The call for Summit Early Science proposals opens;
- December 31, 2017: Due date for letters of intent to respond to the Summit Early Science CFP;
- Early 2018: As the first phase of Summit becomes available, proposing teams will be provided limited access for benchmarking runs required for preparation of the Summit Early Science proposal;
- Proposals are encouraged to be submitted as soon as the necessary benchmarking results are available.
- June 1, 2018: Due date for the proposals to the Summit Early Science CFP;
- September 2018: Early Science projects selected;
- October 1, 2018: Early Science projects have access to Summit. This date may change depending on the actual acceptance date of Summit;
- January 1, 2019: Early Science project access to Summit reduces to 20% of Summit, as the INCITE user program transitions to Summit;
- June 30, 2019: Early Science projects end;
- Late 2019 (actual date TBD): Early Science project reports submission due;
Outcomes and Expectations of Early Science Projects
The Summit Early Science program is expected to deliver early scientific accomplishments in the form of peer-reviewed publications, reports, and highlights. This program is not for application development.
Support for Summit Early Science Projects
Each Summit Early Science project will be assigned an OLCF staff member as liaison. This liaison will devote a fraction of his or her time to collaborating with the Early Science project. Other technical experts will be involved as needed. These experts may include staff from the IBM/NVIDIA OLCF Center of Excellence or other ORNL research staff. Summit Early Science project teams will be invited to participate in Summit training events and workshops.
Summit Early Science Letter of Intent
A letter of intent to submit a Summit Early Science proposal is required, and due on December 31, 2017. This will enable the OLCF to arrange for reviewers, plan for the access to Summit needed to carry out the required benchmark calculations for the proposals, and plan for the allocation of Summit resources in the Early Science program period. These letters should include a list of members of the team and their affiliation, a paragraph with the science goal of the proposed project, a paragraph with a description of the scientific software to be used, including a brief statement on its current parallel scalability, accelerator performance, and the effort to make this application ready for Summit.
Early Science Proposal Submission
- Submission deadline: June 1, 2018
- Proposals will be accepted from the date of this call until the submission deadline. It is recommended to submit early, as proposals will be evaluated and awarded as they come in.
- Proposals should be submitted by email to: [email protected]
- Prepare your proposal using the instructions below
- Submit as a single PDF document
Evaluation of Proposals
A team of internal ORNL staff and external science-domain experts will evaluate proposals on the following criteria.
- Potential scientific impact of the proposed Early Science project
- Demonstration of the need for Summit resources for the technical work
- Demonstration of the computational readiness of the code, with respect scalability and accelerated performance
- Expertise and commitment of the research team
- Diversity of science domains and algorithms employed across the Early Science Program
Proposals will be evaluated as soon as possible after submission, with rolling acceptance into the Early Science program. Award decisions will be made by the OLCF-4 Project Team.
Proposal Instructions
Please create your proposal document with a project title, and the section headings noted below.
Section 1: Project Team
1a. Principal Investigator (PI) Information
- Last Name, First Name, Title (Dr., Mr., Ms., etc.)
- Institution
- Street address
- Email address
- Funding source for the proposed work
1b. For each team member
- Last Name, First Name, Title (Dr., Mr., Ms., etc.)
- Institution
- Street address
- Email address
- Funding source for the proposed work
Section 2: Project Description
2a. Executive Summary (1/2 page)
Provide an Executive Summary describing the scientific objectives and the impact of the proposed research. (1/2 page)
2b. Impact Statement (50 words)
Provide a two-sentence project summary that can be used to describe the impact of your project to the public.
2c. Scientific Campaign (2 pages)
Provide a technical description of the proposed project. Important aspects of this section are the scientific objectives, the scientific methodology, the relevance of the research, the need for resources provided by Summit, and the benefit of the project outcomes to the scientific community. A timeline and milestone table should be included.
Section 3: Computational Readiness (2-3 pages)
Provide a description of the computational readiness of the code that will be used in the science campaign. The overall objective is to demonstrate that the scientific campaign will utilize the Summit architecture efficiently, and that from a computational readiness perspective a proposal based on this code to the INCITE program would be competitive. For the Early Science proposals, computational readiness is, therefore, defined by a scalability and a performance metric as follows.
Scalability: Applications should demonstrate reduced time to solution (for strong scaling benchmarks) or time to solution divided by the number of nodes used (for weak scaling benchmarks) to 20% (i.e. 920 nodes) or more of the full Summit machine, N20. Timings should be obtained for several runs with node-counts below 920 nodes, one run at 920 nodes, and at least one run above 1000 nodes, in which both CPU’s and all GPU’s on the node are used.
Performance: Applications should demonstrate a performance improvement of a factor of two or better by using all six GPUs per node, compared to using both CPUs per node only, with jobs that runs on 20% (i.e. 920 nodes) of the full Summit machine. A description needs to be provided on the strategy used to optimize both cases.
The scalability and performance of the scientific code should be determined using runs that are representative of the calculations that will be carried out as part of the science campaign of the proposed project, and should be presented in a table with benchmark results, as well as in double logarithmic plots as follows.
The scalability and performance data need to be derived from benchmark calculations carried out on Summit. The OLCF will provide limited access to Summit during the time that the call is open.
Section 4: Resource Request (1 page)
Provide an estimate of the resources needed to meet the milestones described in section 2. Provide a detailed description of the number of runs, the number of nodes that will be used, and provide resource estimates based on the benchmark calculations described in section 3.
Section 5: Commitments
Please confirm that, should your proposal be awarded as a Summit Early Science project, you will commit to meeting the following requirements:
- Follow all OLCF policies and procedures
- Describe project progress during periodic conference calls with OLCF staff
- Submit a detailed project report at the end of the Early Science project period, to be received no later than July 31, 2019
