SC17 Denver, CO

P94: Fully Hierarchical Scheduling: Paving the Way to Exascale Workloads

Authors: Stephen Herbein (University of Delaware), Tapasya Patki (Lawrence Livermore National Laboratory), Dong H. Ahn (Lawrence Livermore National Laboratory), Don Lipari (Lawrence Livermore National Laboratory), Tamara Dahlgren (Lawrence Livermore National Laboratory), David Domyancic (Lawrence Livermore National Laboratory), Michela Taufer (University of Delaware)

Abstract: Exascale workloads, such as uncertainty quantification (UQ), represent an order of magnitude increase in both scheduling scale and complexity. Batch schedulers with their decades-old, centralized scheduling model will fail to address the needs of these new workloads. To address these upcoming challenges, we claim that HPC schedulers must transition from the centralized to the fully hierarchical scheduling model. In this work, we assess the impact of the fully hierarchical model on both a synthetic stress test and a real-world UQ workload. We observe over a 100x increase in scheduler scalability on the synthetic stress test and a 37% decrease in the runtime of real-world UQ workloads under the fully hierarchical model. Our empirical results demonstrate that the fully hierarchical scheduling model can overcome the limitations of existing schedulers to meet the needs of UQ and other exascale workloads.
Award: Best Poster Finalist (BP): yes

Poster: pdf
Two-page extended abstract: pdf

Poster Index