Raghav Somani

Raghav Somani

4th year Ph.D. student at
Paul G. Allen School of Computer Science and Engineering

Research Projects

Scaling laws of optimization algorithms for Deep Learning - the Graphon perspective

May 01, 2021

Research project, University of Washington, Seattle, WA, USA

Understanding scaling limits of discrete Euclidean optimization algorithms on large unlabeled graphs. This problem is motivated by the problem of optimizing permutation invariant risk functions of (single layer and deep) Neural Networks (NNs). Theoretical aspects stem from the original theory of gradient flows on the Wasserstein space, which have been used to understand scaling limits of (stochstic) gradient descent ((S)GD) processes in the case of single hidden layer neural networks. There are also other related questions that are specific to the qualitative nature of the stochasticity in the SGD process, and the role of depth in NNs.

Robust Mixed Linear Regression using heterogeneous batches

December 08, 2020

Research project, University of Washington, Seattle, WA, USA

For the problem of learning Mixed Linear Regression, this work introduces a spectral approach that is simultaneously robust under both data scarcity and outlier tasks.

Universality Patterns in the Training of Neural Networks

June 05, 2019

Research project, Microsoft Research Lab - India, Bengaluru, India

This work proposes and demonstrates a surprising pattern in the training of neural networks: there is a one to one relation between the values of any pair of losses (such as cross entropy, mean squared error, \(0/1\) error etc.) evaluated for a model arising at (any point of) a training run. This pattern is universal in the sense that this one to one relationship is identical across architectures (such as VGG, Resnet, Densenet etc.), algorithms (SGD and SGD with momentum) and training loss functions (cross entropy and mean squared error).

Sparse Regression and Support Recovery bounds for Orthogonal Matching Pursuit

October 05, 2018

Research project, Microsoft Research Lab - India, Bengaluru, India

We study the problem of sparse regression where the goal is to learn a sparse vector that best optimizes a given objective function. Under the assumption that the objective function satisfies restricted strong convexity (RSC), we analyze Orthogonal Matching Pursuit (OMP) and obtain support recovery result as well as a tight generalization error bound for OMP. Furthermore, we obtain lower bounds for OMP, showing that both our results on support recovery and generalization error are tight up to logarithmic factors. To the best of our knowledge, these support recovery and generalization bounds are the first such matching upper and lower bounds (up to logarithmic factors) for any sparse regression algorithm under the RSC assumption.

Some Approaches of Building Recommendation Systems

May 02, 2017

Bachelor Thesis Project, Department of Mathematics, IIT Guwahati, Indian Institute of Technology Guwahati

The project aims at using different recommendation methods for different kinds of real world data like rating matrices, images and text, using Deep Learning and Optimization.