A Novel Method for Machine Learning Problems Using StochAstic Recursive grAdient algoritHm

讲座通知

 

2018年4月25日;

下午 15:00 ~ 16:00

 


信息管理与工程学院308室

上海财经大学(第三教学楼西侧)

上海市杨浦区武东路100号

主题

A Novel Method for Machine Learning Problems Using StochAstic Recursive grAdient algoritHm

主讲人

刘杰

Lehigh University

刘杰博士本科毕业于南开大学数学与应用数学系。现美国Lehigh University工业与系统工程系五年级博士生,研究方向为大规模优化算法,包括其在机器学习,电力系统,并行和分布式计算以及统计学习里面的应用。他是Lehigh大学Rossin College院长奖学金和Gotshall奖学金获得者,2017-2018年度IBM全球博士奖学金获得者。曾在美国阿贡国家实验室,西门子美国研究所,IBM Research欧洲爱尔兰研究所,三菱电机研究所(MERL)以及腾讯AI Lab进行科研实习和研究访问。

讲座简介

We propose a StochAstic Recursive grAdient algoritHm (SARAH), as well as its practical variant SARAH+, as a novel approach to the finite-sum minimization problems. Different from the vanilla SGD and other modern stochastic methods such as SVRG, S2GD, SAG and SAGA, SARAH admits a simple recursive framework for updating stochastic gradient estimates; when comparing to SAG/SAGA, SARAH does not require a storage of past gradients. The linear convergence rate of SARAH is proven under strong convexity assumption and SARAH has sublinear convergence rates for convex and nonconvex functions. Numerical experiments demonstrate the efficiency of our algorithm.