Harbin Institute of Technology Advanced Communications Technologies Forum 2019

Title：

Understanding Optimization and Generalization in Deep Learning: A Trajectory-based Analysis

Date and Time: 9:00-11:00, January 4th, 2019

Location：ROOM 1011, BUILDING 2A, NO.2 YIKUANG STREET,

HARBIN, HEILONGJIANG, CHINA

Abstract:

This presentation will present recent progress on understanding deep neural networks by analyzing the trajectory of the gradient descent algorithm. Using this analysis technique, we are able to explain:

1) Why gradient descent finds a global minimum of the training loss even though the objective function is highly non-convex, and

2) Why a neural network can generalize even the number of parameters in the neural network is more than the number of training data.

Biography:

Simon Shaolei Du , Ph.D Department of Computer Science, Carnegie Mellon University, whose tutors are Prof.  Aarti Singh and Prof. Barnabás Póczos. His research interests include theoretical machine learning and statistical topics such as in-depth learning, matrix decomposition, convex/nonconvex optimization, transfer learning, reinforcement learning, nonparametric statistics, and robust statistics. In 2015, he received a double degree in Electrical Engineering and Computer, Engineering Mathematics and Statistics from the University of California at Berkeley. He is a Berkeley EECS winner, a member of Etta Kappa Nu, and a member of Phi Beta Kappa. As a leading young scholar in international machine learning, he is the 2018 NeurIPS top conference Best paper winner, the 2018 NeurIPS top conference NVIDIA Pioneer Award winner, and 19 computer top conference papers. He has also worked in research labs at Microsoft and Facebook.