Training

Summer 2025 Virtual Hands-On HPC Crash Course

Scientific workflows automate research tasks that include data pre-processing, model training, running simulations, analyzing output, and producing visualizations. The use of scientific workflow tools is especially important in high-performance computing (HPC) environments to reduce errors, automate complex tasks, and otherwise increase productivity. While numerous workflow management systems exist, they often impose rigid, top-down architectures that limit flexibility and integration with evolving research code. In this talk, I present a bottom-up approach to scientific workflow design using Python and Dask, a flexible parallel computing library. By constructing task graphs directly from domain-specific code, researchers can retain full control over data movement, computation, and parallelism. I will demonstrate how Dask's dynamic execution model enables the creation of scalable, custom workflows tailored to HPC systems and discuss real-world case studies from my own research. This session is intended for scientific practitioners and developers seeking expressive and adaptable strategies for managing large-scale computational pipelines.

Abstract TBD