A07: Scalable Parallel Scripting in the Cloud
SessionPoster Reception
Author
Event Type
ACM Student Research Competition
Poster
Reception
TimeTuesday, November 14th5:15pm - 7pm
LocationFour Seasons Ballroom
DescriptionIt’s often complicated, time consuming, but frequently necessary to successfully port complex workflows to multiple high-performance environments. Parsl is a Python-based parallel scripting library that provides a simple model for describing and executing dataflow-based scripts over arbitrary execution resources such as clouds, campus clusters, and high-performance systems. Parsl’s execution layer abstracts the differences between providers enabling provisioning and management of compute nodes for use with a pilot system. In this poster, we describe the development of a new execution provider designed to support Amazon Web Services (AWS) and Microsoft’s Azure. This provider supports the transparent execution of implicitly parallel Python-based scripts using elastic cloud resources. We demonstrate that Parsl is capable of executing thousands of applications per second over this elastic execution fabric.
