SC17 Denver, CO

The HDF5 Dataverse

Authors: Mr. David Pearah (HDF Group)

Abstract: The HDF Group will provide a forum for its diverse user community to share ideas and discuss current initiatives. David Pearah will present the latest HDF5 roadmap, including upcoming releases and soliciting input on the future roadmap. Quicey Koziol will moderate a panel with representatives from research, commercial, and government organizations who will present case studies on how they leverage HDF technologies to solve their big compute problems as well as discuss the challenges of using HDF5.

Long Description: HDF5 is a unique, open-source, high-performance technology suite that consists of an abstract data model, library, and file format used for storing and managing extremely large and/or complex data collections. The technology is used worldwide by government, industry, and academia in a wide range of science, engineering, and business disciplines.

There are over ~900 projects on Github utilizing HDF5 due to its (1) versatile self-describing data model that can represent very complex data objects, relationships between objects along with the users’ metadata; (2) completely portable binary file format with no limit on the number or size of data objects; (3) software library optimized for high-speed and compact reads/writes; and (4) tools for managing, manipulating, viewing, and analyzing HDF5 data. In addition, the HDF5 suite is included by every major HPC vendor as part of their core software because of its broad adoption and its ability to improve I/O performance and data organization within HPC Environments.

The HDF Group is excited to discuss its latest HDF5 technology roadmap with the SC17 attendees and share how its scientific, government, and industry users utilize HDF5 technologies to solve real-world problems including work performed to support The Exascale Computing Project (ECP). The HDF Group encourages participants to discuss challenges when using HDF5 to help The HDF Group to prioritize items on the HDF5 roadmap. The HDF Group will also encourage discussion on how the community can contribute to the maintenance and future development of HDF5.

Conference Presentation: pdf

Birds of a Feather Index