Improving Numerical Computation with Practical Tools and Novel Computer Arithmetic
Authors: Dr. John Gustafson (National University of Singapore; A*STAR Computational Resource Center, Singapore)
BP
Abstract: Efficient use of computer arithmetic is crucial to HPC. Unfortunately, few software analysis tools exist to analyze floating-point accuracy or to recommend robust mixed-precision modifications for full-sized applications. At the same time, next-generation arithmetic formats such as Gustafson’s latest Type III unums (posits and valids) offer the possibility of better accuracy, performance, energy efficiency, and reproducibility, but their implementations and tool ecosystems are still young. The goal of this BoF is to connect researchers and industry practitioners interested in improving the state of the art in this area, and to begin to identify concrete ways to work together.
Long Description: The impact of the next generation arithmetic is similar to that of a turn of Moore’s law, doubling computing capability… but without having to shrink transistors. As current floating-point numbers (“floats”) were standardised in the 1980s when transistors were millions of times more expensive than they are now; floats make compromises that are overdue for update. The ability to train neural networks with less bits, e.g. 8 bits, provides an enormous performance boost for artificial intelligence (AI), presently the most important computing focus and challenge. Novel arithmetic is a game changer for the computing industry. The game in High Performance Computing (HPC) presently is to achieve “exascale” computing, which most practitioners implicitly assume means 1018 double-precision floating-point operations per second and meeting the critical power envelope challenge of 20-30 MW. The ability of the new generation arithmetic to use lesser bits for the same/higher computation accuracy will facilitate meeting both the performance and power challenges of exascale computing.
Unum arithmetic, in particular the newest type of unums (posits and valids), is a recent breakthrough by John Gustafson that is superior to the IEEE 754 standard. Posits have higher accuracy and a larger dynamic range using the same number of bits, or what is perhaps more disruptive, sufficient accuracy using fewer bits, thereby saving storage, bandwidth, energy and power. Posits produce bitwise-identical results across diverse computer systems; current floats cannot even guarantee identical results on successive program runs. Posits can serve as drop-in replacements for floats, and are actually simpler than floats, which will result in faster circuits requiring less chip area.
Changing arithmetic however affects the entire hardware-software stack from chip design to the application. The Next Generation Arithmetic BOF session thus provides an excellent opportunity for computer arithmetic enthusiasts to gather, interact and exchange ideas on not only what the next generation arithmetic should be, but also the news and updates on the latest developments of breakthroughs with next generation data formats and their corresponding hardware, tools, applications, services, etc.
Conference Presentation: pdf
Birds of a Feather Index