# Projects

## Connections between Stochasticity of SGD and Generalizability

December 08, 2019

Research project, Microsoft Research Lab - India, Bengaluru, India

This is an attempt to understand how stochasticity in an optimization algorithm affect generalization properties of a Neural Network.

## Universality Patterns in the Training of Neural Networks

June 05, 2019

Research project, Microsoft Research Lab - India, Bengaluru, India

This work proposes and demonstrates a surprising pattern in the training of neural networks: there is a one to one relation between the values of any pair of losses (such as cross entropy, mean squared error, $0/1$ error etc.) evaluated for a model arising at (any point of) a training run. This pattern is universal in the sense that this one to one relationship is identical across architectures (such as VGG, Resnet, Densenet etc.), algorithms (SGD and SGD with momentum) and training loss functions (cross entropy and mean squared error).

## Sparse Regression and Support Recovery bounds for Orthogonal Matching Pursuit

October 05, 2018

Research project, Microsoft Research Lab - India, Bengaluru, India

We study the problem of sparse regression where the goal is to learn a sparse vector that best optimizes a given objective function. Under the assumption that the objective function satisfies restricted strong convexity (RSC), we analyze Orthogonal Matching Pursuit (OMP) and obtain support recovery result as well as a tight generalization error bound for OMP. Furthermore, we obtain lower bounds for OMP, showing that both our results on support recovery and generalization error are tight up to logarithmic factors. To the best of our knowledge, these support recovery and generalization bounds are the first such matching upper and lower bounds (up to logarithmic factors) for any sparse regression algorithm under the RSC assumption.

## Clustered Monotone Transforms for Rating Factorization

August 16, 2018

Continued Intern Project, Microsoft Research Lab - India, Bengaluru, India

We propose Clustered Monotone Transforms for Rating Factorization (CMTRF), a novel approach to perform regression up to unknown monotonic transforms over unknown population segments. For recommendation systems, the technique searches for monotonic transformations of the rating scales resulting in a better fit. This is combined with an underlying matrix factorization regression model that couples the user-wise ratings to exploit shared low dimensional structure. The rating scale transformations can be generated for each user (N-CMTRF), for a cluster of users (CMTRF), or for all the users at once (1-CMTRF), forming the basis of three simple and efficient algorithms proposed, all of which alternate between transformation of the rating scales and matrix factorization regression. Despite the non-convexity, CMTRF is theoretically shown to recover a unique solution under mild conditions.

## A case study of Empirical Bayes in Recommendation system

July 11, 2017

Academic Project, Department of Mathematics, IIT Guwahati, Indian Institute of Technology Guwahati

We provide a formulation of empirical bayes described by Atchadé (2011) to tune the hyperparameters of priors used in Bayesian set up of collaborative filter.

## Some Approaches of Building Recommendation Systems

May 02, 2017

Bachelor Thesis Project, Department of Mathematics, IIT Guwahati, Indian Institute of Technology Guwahati

The project aims at using different recommendation methods for different kinds of real world data like rating matrices, images and text, using Deep Learning and Optimization.

## Modelling Economic Policy Uncertainty Index using Text Classification

July 20, 2015

Intern Project, Centre of Advanced Financial Research And Learning (CAFRAL), Researve Bank of India, Mumbai

Using Soft Margin Kernel Support Vector Machine to classify newspaper articles to model an Economic Policy Uncertainty Index for India.

## Mini Search Engine

April 20, 2015

Lab Project, Department of Mathematics, IIT Guwahati, Indian Institute of Technology Guwahati

We used data structures like Hash Tables, Balanced Trees in order to design a text search engine that gives the frequency of the searched word in a given folder of files.