Review on Recent Matrix Multiplication Optimization Using Deep Learning

The use of deep learning techniques with matrix multiplication has transformed several fields, leading to notable breakthroughs and innovative uses. In this paper, we summarize the state-of-the-art and explain the many applications, approaches, and advancements resulting from the combination of matrix multiplication with deep learning. Examining a wide range of articles released since 2019, this evaluation focuses on the creative methods, strategies, and improvements that scholars in many domains have used. The review encapsulates significant strides in matrix multiplication optimization through deep learning methodologies, unveiling a spectrum of innovative approaches and their consequential impacts. These advancements span dynamic precision neural network inference on silicon photonic processors, adaptable templates for matrix multiplication across various architectures, and coherent analog computing engines for optical matrix multiplication. Furthermore, the exploration extends to unconventional accelerators like AI Engine tiles and low-power multi-layer perceptron accelerators, showcasing remarkable efficiency gains. The integration of machine learning frameworks, such as WISE, into sparse matrix operations further underscores the potential for substantial computational efficiency improvements. Collectively, these findings underscore the transformative potential of deep learning-driven optimization techniques in revolutionizing matrix multiplication performance, offering pathways to enhanced computational efficiency and energy savings across diverse application domains.