Selected Publications

See my complete publication record on Google Scholar or DBLP.

Non-Gaussianity of Stochastic Gradient Noise

Abhishek Panigrahi, Raghav Somani, Navin Goyal & Praneeth Netrapalli

Published in: Science meets Engineering of Deep Learning (SEDL) workshop, Neural Information Processing Systems (NeurIPS), 2019

We study the distribution of stochastic gradient noise during training and observe that, for batch sizes \(256\) and above, it is best described as Gaussian, at least in the early phases of training. Read more

* Equal contribution