Interactivity in Supercomputing
Authors: Dr. Peter Messmer (Nvidia Corporation)
Abstract: The HPC landscape has undergone a lot of changes in the past years. In addition to more heterogeneous system architectures, a wealth of novel workloads specifically from the data sciences have emerged. With these novel workloads, a broader user base with different workflows and expectations about interactivity have arrived. In addition, traditional HPC applications have embraced more interactive workflows. The goal of this BoF is to bring together domain scientists, tool developers and HPC center administrators to identify the scientific impact and technical challenges of highly interactive access to HPC resources.
Long Description: The goal of this BOF is to form a community of domain scientists, tool developers and center operators interested in the interactive use of supercomputing systems, to gain a better understanding of the respective needs and requirements, and to form a common vision for interactive HPC resources.
The traditional scientific computing workflow is driven by the external constraints, like time available on a shared resource or physical distance between user and HPC center, rather than by the requirements of the scientist’s agenda. Intuition forming exploratory parameter scans, “what-if” scenarios or investigations of extreme cases are at the core of the scientific discovery process. These computationally highly demanding scenarios could all benefit from supercomputing resources, but are often hard to realize in the traditional, batch oriented workflow.
Many attempts of interactive use of HPC resources have been undertaken in the past, and yet widespread adoption in the scientific computing workflow has not occurred. Possible reasons are lack of perceived benefit to the domain scientist, tool complexities and challenges to resource access.
The cross-disciplinary and cross-organizational challenges faced in preparing these demos highlighted the need for a community consisting of tools developers, center operators and domain scientists to define the right policies, develop the necessary tools and provide the scientific framework to optimally benefit from interactive supercomputing.
Being one of the few events that attracts a global audience of members from all these groups, SC offers a unique venue to form such a community. The informal nature of a BOF is the ideal format to foster the cross-disciplinary exchange and to kick-start joint activities.
The short presentations and the following discussion should help domain scientists to better understand the available tools and the perspective of center operators. Tool developers should get a broader perspective of the needs of domain scientists and challenges faced by center operators, whereas HPC center operators should get an understanding of the different use case and their impact on the scientific users.
We hope to summarize the discussion and insight gained at his BOF into a report for the greater community and expand upon these activities at future SC conference.
This BOF has been previously held two years ago at SC15. The session was well attended (~100 participants) and lead to a lively discussion among the center operators, the domain scientists and the middleware developers. Some of these results are summarized on the BOF site at https://sites.google.com/view/interative-hpc/home. This year, we are planning to survey the community to better understand the needs of the individual groups and improve the interactive experience of HPC systems.
A request, it will be great if this BoF is not scheduled when the Top 500 BoF, or In-situ visualization BoFs are happening at SC17.
Conference Presentation: pdf
Birds of a Feather Index