Connections between Stochasticity of SGD and Generalizability
Presented at Microsoft Research Lab - India, Bengaluru, India, 2019
This is an attempt to understand how stochasticity in an optimization algorithm affects the generalization properties of a neural network. Current work includes understanding the distribution of Stochastic Gradient Noise as it directly correlates to generalization under most empirical settings. There has been a recent interest in understanding the effects of distributional properties of SGD as it affects the optimization dynamics in finding good local minima (which have superior generalization properties). With the use of hypothesis testing, we observe that for batch sizes
The current results have been presented at the workshop Science meets Engineering of Deep Learning at Neural Information Processing Systems 2019.
The paper can be found here.
