In deep neural networks, the number of trainable parameters is quite huge, which raises big challenges in both practical training and theoretical analysis. An essential question is, is the intrinsic freedom in training really very high? If not, can we find a subspace such that training in the subspace can also lead to satisfactory performance. In this talk, we will discuss this low-dimension hypothesis and design efficient methods to find such tiny subspaces. The answer is quite positive: we indeed can train deep neural networks in tiny subspaces, which almost naturally benefits training efficiency, generalization capability, and robustness.
Post Talk Link: Click Here
Passcode: %&4*9WUW
Xiaolin Huang, Associate Professor, Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China. He received the PhD degree from Tsinghua University, and then worked as a postdoc with ESAT-STADIUS, KU Leuven, Belgium, and as a Humboldt fellow with Pattern Recognition Lab, the Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany. In 2017, he was awarded by “1000-Talent Plan” (Young Program) and joined Shanghai Jiao Tong University. His research is focusing on machine learning in the view of optimization, which specifically includes topics on how to expand kernel learning capability; how to improve (adversarial) robustness; and how to speed up the training process. In recent years, he has published more than 30 papers in top journal/conferences like JMLR, TPAMI, ACHA, TOG, TIP, TMI, NeurIPS, ICLR, CVPR, AISTATS, etc. He has also published a review on piecewise linear neural networks in Nature Reviews Methods Premiers, and co-authored one textbook on AI Foundation (in Chinese). He is now a Senior Member of IEEE and is serving as AE (in optimization area) for Frontiers in Applied Mathematics and Statistics, AC for ICCV, etc.
Read More
Read More