Lustre Community BoF: Lustre Deployments for the Next 5 Years
Authors: Frank Baetke (Hewlett Packard, European Open File System)
BP
Abstract: Lustre is the leading open source file system for HPC and is widely used and in more mission-critical installations. Lustre currently supports many HPC infrastructures beyond its traditional stronghold of scientific research including financial services, oil and gas, advanced manufacturing, and visual effects. At this year’s Community BoF, the worldwide community of developers, administrators, and solution providers will gather to discuss the current status of the Lustre community, OpenSFS and EOFS organizations, and also discuss recent feature developments, new challenges and corresponding opportunities for the next 5 years of Lustre. Summaries of LUG2017 and LAD 2017 will be presented.
Long Description: Lustre is the leading open source file system for HPC and is widely used and in mission-critical installations. Lustre currently supports many HPC infrastructures. At this year’s BOF the worldwide community of developers, administrators, and solution providers will gather to discuss the current status of the Lustre community, OpenSFS and EOFS organizations, also discuss new challenges and corresponding opportunities for the next 5 years of Lustre. Short summaries of LUG2017 and LAD 2017 will also be presented.
Used on 9 out of 10 of the top 10 systems and over 75% of the TOP100 systems on the June 2017 TOP500 list, Lustre is one of the most widely adopted technologies in HPC. From the Tianhe-2 system at the National Supercomputer Center in, China, the Titan system at Oak Ridge National Laboratory, the Sequoia system at Lawrence Livermore National Laboratory, to the K computer at RIKEN Advanced Institute for Computer Science in Kobe, Japan, and more. Lustre is also widely used across mid and small-scale HPC systems with continued adoption attributable to the stability of the Lustre file system as it has matured.
Vital to the technology is the community that continues to drive Lustre forward. As development has evolved into a true open source model, the end users, developers, and solutions providers have come together through the worldwide OpenSFS and EOFS communities. This community development model has resulted in significant new features, improved stability and broader adoption.
At the 2017 Lustre community BOF we will focus on the recent feature developments and discuss how they will shape the next 5 years of Lustre deployment around the world. Progressive file layouts (PFL), File Level Redundancy (FLR) and the ability to scale metadata servers (Distributed Namespace) are among such game-changing features. From financial services, oil and gas, advanced manufacturing, to basic sciences, Lustre is at the heart of many HPC infrastructures and must continue to evolve to support new use cases. We will explore these use cases and discuss the roadmap for meeting the requirements they present.
The session will include a town hall meeting open to all BOF participants. OpenSFS and EOFS will provide initial discussion topics focusing on a brief set of challenges. The list of challenges will be compiled by the OpenSFS and EOFS by polling the community before the BOF. These challenges will then be posed to the attendees by the BOF session leaders to initiate an open floor discussion for their view of how Lustre needs to evolve to meet these requirements.
The Lustre Community BOF has been held at previous SC events and has been well attended and enthusiastically received. SC presents a rare opportunity for the worldwide community of Lustre users, administrators, developers, and solution providers to meet and discuss how to make Lustre more successful. Attendance is usually very good with 100 – 120 attendees.
A written report summarizing the Lustre Community BOF will be developed and will be made available to the Lustre community at lustre.opensfs.org
Conference Presentation: pdf
Birds of a Feather Index