P58: Wharf: Sharing Docker Images across Hosts from a Distributed Filesystem
SessionPoster Reception
Authors
Event Type
ACM Student Research Competition
Poster
Reception
TimeTuesday, November 14th5:15pm - 7pm
LocationFour Seasons Ballroom
DescriptionDue to their portability and less overhead compared to traditional virtual machines, containers are becoming an attractive solution for running HPC workloads. Docker is a popular toolset which enables convenient provisioning and management of containers and their corresponding images. However, Docker does not natively support running on shared storage, a crucial requirement in large-scale HPC clusters which are often diskless or access data via a shared burst buffer layer. This lack of distributed storage support can lead to overhead when running containerized HPC applications. In this work, we explore how Docker images can be served efficiently from a shared distributed storage layer. We implement a distributed layer on top of Docker that allows multiple Docker daemons to access container images from a shared filesystem such as IBM Spectrum Scale or NFS. Our design is independent of the underlying storage layer and minimizes the synchronization overhead between different daemons.
