SC17 Denver, CO

P45: Campaign Storage: Erasure Coding with GPUs

Authors: Walker Haddock (University of Alabama, Birmingham), Matthew L. Curry (Sandia National Laboratories), Purushotham Bangalore (University of Alabama, Birmingham), Anthony Skjellum (University of Tennessee, Chattanooga)

Abstract: Cloud computing has developed high capacity, reliable and economical storage systems based on object technology. Los Alamos National Labs has designed the storage systems for Trinity to include a cloud type object storage system as a layer between the Parallel File System (PFS). This pre-archive system has been dubbed "Campaign Storage" with the purpose of storing data products to be quickly accessible during the life of a research project. Data stored on the Campaign Storage can be pulled into the PFS or moved to archive after the data has been curated. Campaign Storage reduces the capacity requirements for PFS storage and reduces the data transfer bandwidth requirements for the archive storage.

We make these contributions to the pre-archive storage layer: * GPU assisted erasure coding * Demonstrating erasure on File Transfer Agents * Reducing erasure recovery costs with "lazy recovery" * Enabling larger erasure coded disk pools

Award: Best Poster Finalist (BP): no

Poster: pdf
Two-page extended abstract: pdf

Poster Index