SC17 Denver, CO

OpenStack For HPC: Best Practices for Optimizing Software-Defined Infrastructure

Authors: Mr. Blair Bethwaite (Monash University)

Abstract: OpenStack has become an ideal solution for managing HPC infrastructure and workloads. However, performance overheads, network integration, and system complexity pose daunting architectural challenges. High performance parallel file systems and the integration of HPC resources into on-premise hybrid clouds allow organizations to increase agility and reduce cost. Using OpenStack technology to control large pools of compute, storage and networking resources throughout the data center allows IT organizations to reduce storage cost while maintaining performance and utilization as it scales. This BoF is aimed at architects, administrators, software engineers, and scientists interested in designing and deploying OpenStack infrastructure for HPC.

Long Description: Cloud Computing represents one of the most significant shifts in IT, and the group of projects that comprise OpenStack are the de-facto standards for putting cloud technologies and methodologies within reach. The level of interest in the application of OpenStack in the HPC and Research Computing space reflects the already strong representation of scientific OpenStack deployments amongst the research community.

In response to the high representation of research computing deployments the OpenStack Foundation created a Scientific Working Group. Over seventy people attended the inaugural working group meeting, in April 2016, and actively participated in collaborative note taking. There was strong representation from SC stalwarts including several US computing centers, current NSF projects, national labs and prominent educational and research institutions, as well as their European and APJ counterparts.

It is the intent of this BoF to provide the broader HPC community an overview of the challenges of supporting HPC workloads with OpenStack and best practices adopted by members of the OpenStack community. Dealing with the complexity of Neutron networking, particularly with Neutron’s many component technologies like SR-IOV/VXLAN/VLAN, Distributed Virtual Routers, SNAT, etc, will be an important topic. The need to access parallel filesystems from within OpenStack tenant networks is a popular refrain, so another topic will be the use of storage systems such as Ceph, GPFS, and Lustre for shared filesystems, virtual machine root disks, virtual machine images, and scratch space. Other HPC-centric topics revolve around accounting and scheduling, including practical resource allocation approaches with the on-demand IaaS model. Through an open and thoughtful exchange, we intend to begin developing a shared understanding and vision of how open cloud computing solutions can best support existing and emerging uses in a range of research disciplines.

The sponsors of this BoF represent a range of those leading the charge in the OpenStack community. They cover a wide range of architectural approaches, including: bare-metal and virtualisation (both machine and OS); along with integrated high-performance interconnects, storage and computational accelerators. This group is well placed to set the scene for HPC on OpenStack and answer and discuss audience questions, including what does and does not work, and where challenges remain.

This BoF was previously held with great success (standing room only) at SC16 (Etherpad here: At SC15, a BoF entitled "Virtualization and Clouds in HPC" was also widely attended, filling the room to capacity, with many more potential attendees refused entry due to lack of space. The tangible outcomes of this BoF will be: 1) A public Etherpad of collaborative notes and chat from during and after the session 2) Raising awareness of OpenStack’s Scientific Working Group and encouraging new participants 3) An updated FAQ added to the OpenStack wiki which summarizes the major contributions and limitations of OpenStack in HPC 4) The emergence (hopefully) of opportunities for the OpenStack Scientific Working Group to work towards addressing these challenges - the community building aspect of events such as this BoF is vital to the health of these Open Source endeavours.

Conference Presentation: pdf

Birds of a Feather Index