Past Activity
Neuron Shapley: Discovering the Responsible Neurons
Time: 1.30 - 3.00 pm, Mar 16, 2021
Speaker: Ning Zhang
Zoom Link: 757 261 8450
https://cuhksz.zoom.com.cn/j/7572618450
Short Abstract:
Reference:
The Implicit Bias of Gradient Descent on Separable Data
Time: 1.30 - 3.00 pm, Mar 8, 2021
Speaker: Ziniu Li
Zoom Link: 757 261 8450
https://cuhksz.zoom.com.cn/j/7572618450
Short Abstract:
Reference:
Predicting What You Already Know Helps: Provable Self-Supervised Learning
Time: 1.30 - 2.30 pm, Mar 2, 2021
Speaker: Lixian Zeng
Zoom Link: 757 261 8450
https://cuhksz.zoom.com.cn/j/7572618450
Short Abstract:
Reference:
On the Convergence of Deep Networks with Sample Quadratic Overparameterization
Time: 1.30 - 2.30 pm, Feb 9, 2021
Speaker: Yushun Zhang
Zoom Link: 757 261 8450
https://cuhksz.zoom.com.cn/j/7572618450
Short Abstract:
The remarkable ability of deep neural networks to perfectly fit training data when optimized by gradient-based algorithms is yet to be fully explained theoretically. Explanations by recent theoretical works rely on the networks to be wider by orders of magnitude than the ones used in practice. In this work, we take a step towards closing the gap between theory and practice. We show that a randomly initialized deep neural network with ReLU activation converges to a global minimum in a logarithmic number of gradient-descent iterations, under a considerably milder condition on its width. Our analysis is based on a novel technique of training a network with fixed activation patterns. We study the unique properties of the technique that allow an improved convergence, and can be transformed at any time to an equivalent ReLU network of a reasonable size.