Effective Programming Models for Deep Learning at Scale
Workshop: ESPM2 2017: Third International Workshop on Extreme Scale Programming Models and Middleware
Authors: Daniel Holmes (University of Edinburgh), Michael Houston (Nvidia Corporation), Prabhat M (Lawrence Berkeley National Laboratory), Jeffrey M. Squyres (Cisco Systems), Rick Stevens (Argonne National Laboratory)
Abstract: Artificial intelligence (AI) has been an interesting research topic for many decades but has struggled to enter mainstream use. Deep Learning (DL) is one form of AI that has recently become more practicable and useful because of dramatic increases in the computational power and in the amount of training data available. Research labs are already using Deep Learning to progress scientific investigations in numerous fields. Commercial enterprises are starting to make product development and marketing decisions based on machine learning models. However, there is a worrying skills gap between the hype and the reality of getting business benefit from Deep Learning. To address this, we need to answer some urgent questions. What practical programming techniques (specifically, programming models and middleware options) should we be teaching new recruits into this area? What existing knowledge and experience (from HPC or elsewhere) should existing practitioners be leveraging? Do traditional big-iron supercomputers and HPC software techniques (including MPI or PGAS) have a place in this vibrant new sphere or is all about high-level scripting, complex workflows, and elastic cloud resources?
Workshop Index