Low-Complexity NN Technology: Model and Precision Search, Acceleration Circuit, and Applications

Wednesday, April 24, 2024

Quantization represents a popular NN complexity-reduction technology that leverages the (im)precision-tolerant nature of neural network training and inference. Our ongoing efforts have delivered main-stream NNs with only 1-bit weights, e.g., binary-weighted CNNs and transformers, while maintaining satisfactory accuracy performance. A variety of applications have been used to demonstrate the power of our proposed simple and effective Neural Architecture Search scheme (TPC-NAS). They include CNNs and transformers for Imagenet and object detection, transformers for natural language processing applications, and CNNs for vision processing.
The low-complexity NN technology has also been applied to build a PE-based CNN/transformer hardware accelerator in Xilinx FPGA SoC. Corresponding training and inference software framework has been developed based on PyTorch, effectively making the proposed deep learning accelerator a solid foundation for an easy-to-deploy DL implementation platform. Lastly, the presentation will showcase a few real-time neural network tasks accomplished through the proposed DL platform.

Speaker/s

Tzi-Dar Chiueh received B.S. and Ph.D. degrees in electrical engineering from National Taiwan University and California Institute of Technology in 1983 and 1989, respectively. He is now a Distinguished Professor in the Department of Electrical Engineering and Graduate Institute of Electronics Engineering and Dean of the Graduate School of Advanced Technology at National Taiwan University. His research interests include algorithms, architecture, and integrated circuits for baseband communication systems and neural networks. Dr. Chiueh was the recipient of the Outstanding Research Award from the National Science Council, Taiwan, in 2004-2007. In 2009, he received the Outstanding Industry Contribution Award from the Ministry of Economic Affairs, Taiwan. In 2017, he also received the Outstanding Technology Transfer Contribution Award from the Ministry of Science and Technology, Taiwan. He was awarded the Himax Chair Professorship and the Macronix Chair Professorship at NTU in 2006 and 2021, respectively. Prof. Chiueh is an IEEE Fellow.

Related