Posts by Tags

Non-asymptotic rate for Random Shuffling for Quadratic functions

Posted on:

This article is in continuation of my previous blog, and discusses about a section of the work by Jeffery Z. HaoChen and Suvrit Sra 2018, in which the authors come up with a non-asymptotic rate of $\mathcal{O}\left(\frac{1}{T^2} + \frac{n^3}{T^3} \right)$ for Random Shuffling Stochastic algorithm which is strictly better than that of SGD. Read more

Posted on:

Nesterov’s Acceleration

Posted on:

This post contains a summary and survey of the Nesterov’s accelerated gradient descent method and some insightful implications that can be derived from it. We analyze the simple convex quadratic case and have a close look at the dynamics of the error vectors. Read more

A survey on Large Scale Optimization

Posted on:

This post contains a summary and survey of the theoretical understandings of Large Scale Optimization by referring some talks, papers, and lectures that I have come across in the recent. Read more

Non-asymptotic rate for Random Shuffling for Quadratic functions

Posted on:

This article is in continuation of my previous blog, and discusses about a section of the work by Jeffery Z. HaoChen and Suvrit Sra 2018, in which the authors come up with a non-asymptotic rate of $\mathcal{O}\left(\frac{1}{T^2} + \frac{n^3}{T^3} \right)$ for Random Shuffling Stochastic algorithm which is strictly better than that of SGD. Read more

Posted on:

Posted on:

Nesterov’s Acceleration

Posted on:

This post contains a summary and survey of the Nesterov’s accelerated gradient descent method and some insightful implications that can be derived from it. We analyze the simple convex quadratic case and have a close look at the dynamics of the error vectors. Read more