Speaker
Description
Deep learning methods are increasingly deployed using low-precision arithmetic, primarily driven by memory, energy, and throughput constraints. At the same time, deep neural networks are highly compositional systems, a structure that naturally raises concerns about the amplification and accumulation of numerical errors across layers and operations. Nonetheless, such models are being applied at an increasing scale to problems in applied mathematics and engineering, where reliability and high numerical accuracy are essential. In this talk, we provide a broad overview of the challenges posed by finite-precision arithmetic in deep learning, identify scenarios in which reduced precision can lead to problematic behavior, and discuss strategies for mitigating associated risks.