Skip to main content

Training

Monthly Weekly Daily List Grid Tile
June 2025

June 2025 OLCF User Conference Call: Dask on HPC Systems

Scientific workflows automate research tasks that include data pre-processing, model training, running simulations, analyzing output, and producing visualizations. The use of scientific workflow tools is especially important in high-performance computing (HPC) environments to reduce errors, automate complex tasks, and otherwise increase productivity. While numerous workflow management systems exist, they often impose rigid, top-down architectures that limit flexibility and integration with evolving research code. In this talk, I present a bottom-up approach to scientific workflow design using Python and Dask, a flexible parallel computing library. By constructing task graphs directly from domain-specific code, researchers can retain full control over data movement, computation, and parallelism. I will demonstrate how Dask's dynamic execution model enables the creation of scalable, custom workflows tailored to HPC systems and discuss real-world case studies from my own research. This session is intended for scientific practitioners and developers seeking expressive and adaptable strategies for managing large-scale computational pipelines.
25 Jun
12:00 pm - 1:00 pm
Zoom
July 2025
August 2025
No event found!