Time：Friday, May 20th, 2022, 10:00-11:00
Location：Tecent Meeting
Link:https://meeting.tencent.com/dm/N9BlY0oqqrMp
Room Number: 778-1386-4492
Speaker: Yaoyu Zhang, Shanghai Jiao Tong University
Abstract: Understanding the structure of loss landscape of deep neural networks (DNNs) is obviously important. In this talk, I will present the embedding principle that the loss landscape of a DNN contains all the critical points of all the narrower DNNs. Empirically, we find that a wide DNN is often attracted by highly-degenerate critical points that are embedded from narrow DNNs. The embedding principle provides a new perspective to study the general easy optimization of wide DNNs and unravels a potential implicit low-complexity regularization during the training, by which a more exact and comprehensive understanding can be anticipated in the near future.