Why Is MPI So Slow? Analyzing the Fundamental Limits in Implementing MPI-3.1
SessionOptimizing MPI
Authors
Event Type
Paper
Communication Optimization
OS and Runtime Systems
Parallel Programming Languages, Libraries, Models and Notations
Programming Systems
TimeThursday, November 16th3:30pm - 4pm
Location405-406-407
DescriptionThis paper provides an in-depth analysis of the software overheads in the MPI performance-critical path and exposes mandatory performance overheads that are unavoidable based on the MPI-3.1 specification. We first present a highly optimized implementation of the MPI-3.1 standard where the communication stack---all the way from the application to the low-level network communication API---takes only a few tens of instructions. We then carefully study these instructions and analyze the root cause of these overheads based on specific requirements from the MPI standard that are unavoidable under the current MPI standard. We finally recommend potential changes to the MPI standard that can minimize these overheads. Our experimental results on a variety of network architectures and applications demonstrate significant benefits from our proposed changes.
Download PDF: here
Authors
