Rahim Entezari: “The role of permutation invariance in linear mode connectivity of neural networks”
Apr 25, 2022—Apr 29, 2022
- This event has passed.
Event Navigation
Rahim Entezari will present research pursued with Behnam Neyshabur, Hanie Sedghi and Olga Saukh at the Tenth International Conference on Learning Representations ICLR 2022, which takes place online from April 25–29, 2022.
Abstract:
In this work, we conjecture that if the permutation invariance of neural networks is taken into account, SGD solutions will likely have no barrier in the linear interpolation between them. Although it is a bold conjecture, we show how extensive empirical attempts fall short of refuting it.
We further provide a preliminary theoretical result to support our conjecture. Our conjecture has implications for the lottery ticket hypothesis, distributed training, and ensemble methods.
About the Conference:
The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning.
ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.
Participants at ICLR span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.