# Posts by Collection

## Montreal, Canada during NeurIPS 2018

Posted on:

Visited Montreal, Canada with Microsoft Research Labmates to attend and present at NeurIPS 2018 Read more

## Melbourne, Australia during WSDM 2019

Posted on:

Visited Melbourne, Australia to attend and present at WSDM 2019 Read more

## Vancouver, Canada during NeurIPS 2019

Posted on:

Visited Vancouver, Canada to attend NeurIPS 2019 and present at SEDL 2019 Read more

## Some Approaches of Building Recommendation Systems

Posted on:

The project aims at using different recommendation methods for different kinds of real world data like rating matrices, images and text, using Deep Learning and Optimization. Read more

## Sparse Regression and Support Recovery bounds for Orthogonal Matching Pursuit

Posted on:

We study the problem of sparse regression where the goal is to learn a sparse vector that best optimizes a given objective function. Under the assumption that the objective function satisfies restricted strong convexity (RSC), we analyze Orthogonal Matching Pursuit (OMP) and obtain support recovery result as well as a tight generalization error bound for OMP. Furthermore, we obtain lower bounds for OMP, showing that both our results on support recovery and generalization error are tight up to logarithmic factors. To the best of our knowledge, these support recovery and generalization bounds are the first such matching upper and lower bounds (up to logarithmic factors) for any sparse regression algorithm under the RSC assumption. Read more

## Universality Patterns in the Training of Neural Networks

Posted on:

This work proposes and demonstrates a surprising pattern in the training of neural networks: there is a one to one relation between the values of any pair of losses (such as cross entropy, mean squared error, $0/1$ error etc.) evaluated for a model arising at (any point of) a training run. This pattern is universal in the sense that this one to one relationship is identical across architectures (such as VGG, Resnet, Densenet etc.), algorithms (SGD and SGD with momentum) and training loss functions (cross entropy and mean squared error). Read more

## Connections between Stochasticity of SGD and Generalizability

Posted on:

This is an attempt to understand how stochasticity in an optimization algorithm affect generalization properties of a Neural Network. Read more

## Robust Mixed Linear Regression using heterogeneous batches

Posted on:

For the problem of learning Mixed Linear Regression, this work introduces a spectral approach that is simultaneously robust under both data scarcity and outlier tasks. Read more

Posted on:

Understanding scaling limits of gradient flow processes on large unlabeled graphs. This problem is motivated by the problem of optimizing permutation invariant risk functions of (single layer and deep) Neural Networks. Theoretical aspects stem from the original theory of gradient flows on the Wasserstein space, which have been used to understand scaling limits of (stochstic) gradient descent ((S)GD) processes in the case of single hidden layer neural networks. There are also other related questions that are specific to the qualitative nature of the stochasticity (sub-gaissian vs heavy tailed) in the SGD process. Read more

## Clustered Monotone Transforms for Rating Factorization

Raghav Somani*, Gaurush Hiranandani*, Sanmi Koyejo & Sreangsu Acharyya
Published at: Web Search and Data Mining (WSDM), 2019

The paper has been accepted for an oral persentation (84/511 submissions ≈ 16% Acceptance Rate). Read more

[paper] [arXiv] [bib] [code] [video]

## Support Recovery for Orthogonal Matching Pursuit: Upper and Lower bounds

Raghav Somani*, Chirag Gupta*, Prateek Jain & Praneeth Netrapalli
Published at: Neural Information Processing Systems (NeurIPS), 2018

The paper has been accepted for Spotlight presentation. Read more

[paper] [bib] [video]

## Non-Gaussianity of Stochastic Gradient Noise

Abhishek Panigrahi, Raghav Somani, Navin Goyal & Praneeth Netrapalli
Published at: Science meets Engineering of Deep Learning (SEDL) workshop, Neural Information Processing Systems (NeurIPS), 2019

We study the distribution of the Stochastic Gradient Noise during the training and observe that for batch sizes $256$ and above, the distribution is best described as Gaussian at-least in the early phases of training. Read more

[arXiv] [bib]

## Meta-learning for Mixed Linear Regression

Weihao Kong, Raghav Somani, Zhao Song, Sham Kakade, Sewoong Oh
Published at: International Conference on Machine Learning (ICML), 2020

The paper has been accepted for a presentation. Read more

[paper] [arXiv] [bib] [code] [video]

## Robust Meta-learning for Mixed Linear Regression with Small Batches

Weihao Kong, Raghav Somani, Sham Kakade, Sewoong Oh
Published at: Neural Information Processing Systems (NeurIPS), 2020

The paper has been accepted for a poster. Read more

[paper] [arXiv] [bib] [code] [video]

## Gradient flows on graphons: existence, convergence, continuity equations

Sewoong Oh, Soumik Pal, Raghav Somani & Raghav Tripathi
Published at: Optimal Transport and Machine Learning (OTML) workshop, Neural Information Processing Systems (NeurIPS), 2021

The paper has been accepted for a poster. Read more

[arXiv] [bib]