SC17 Denver, CO

Optimizing Word2Vec Performance on Multicore Systems

Workshop: IA^3 2017 - 7th Workshop on Irregular Applications: Architectures and Algorithms
Authors: Vasudevan Rengasamy (Pennsylvania State University)

Abstract: The Skip-gram with negative sampling (SGNS) method of Word2Vec is an unsupervised approach to map words in a text corpus to low dimensional real vectors. The learned vectors capture semantic relationships between co-occurring words and can be used as inputs to many natural language processing and machine learning tasks. There are several high-performance implementations of the Word2Vec SGNS method. In this paper, we introduce a new optimization called context combining to further boost SGNS performance on multicore systems. For processing the One Billion Word benchmark dataset on a 16-core platform, we show that our approach is 3.53X faster than the original multithreaded Word2Vec implementation and 1.28X faster than a recent parallel Word2Vec implementation. We also show that our accuracy on benchmark queries is comparable to state-of-the-art implementations.

Workshop Index