An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians

Thursday, April 14, 2022

In this talk, I will discuss our recent work on stochastic optimization with equality constraints. We consider solving nonlinear optimization problems with stochastic objective and deterministic equality constraints. We propose a stochastic algorithm based on sequential quadratic programming (SQP) that uses a differentiable exact augmented Lagrangian as the merit function. The design of the algorithm is motivated by an old SQP method (Lucidi, 1990) developed for solving deterministic problems. I will first explain how to handle stochastic objectives when the stepsizes are deterministic and prespecified. Next, I will explain how to adaptively select the random stepsizes by adapting the stochastic line search procedure of Paquette and Scheinberg (2020) that was developed for unconstrained problems. We established the global “almost sure” convergence for the SQP method. If time permits, I will also discuss recent progress on solving problems with inequality constraints.

Speaker/s

Mladen Kolar is Associate Professor of Econometrics and Statistics at the University of Chicago Booth School of Business. Kolar's research is focused on high-dimensional statistical methods, probabilistic graphical models, and scalable optimization methods, driven by the need to uncover interesting, and scientifically meaningful structures from observational data. His research appears in journals such as the Journal of Machine Learning Research, the Annals of Statistics, the Journal of the Royal Statistical Society, the Journal of the American Statistical Association, Biometrika, and other outlets. Kolar also regularly presents his research at the top machine learning conferences, including Advances in Neural Information Processing Systems (NeurIPS) and the International Conference of Machine Learning (ICML). Kolar currently serves as associate editor for the Journal of Machine Learning Research, the Journal of Computational and Graphical Statistics, and the New England Journal of Statistics in Data Science. Kolar was awarded a prestigious Facebook Fellowship in 2010 for his work on machine learning and network models. He spent a summer with Facebook’s ads optimization team working on a large-scale system for click-through rate prediction. Kolar earned his Ph.D. in Machine Learning in 2013 from Carnegie Mellon University (CMU), as well as a Diploma in Computer Engineering from the University of Zagreb. For his Ph.D. thesis work on “Uncovering Structure in High-Dimensions: Networks and Multi-task Learning Problems,” Kolar received from 2014 SIGKDD Dissertation Award honorable mention.

Related