Interface Programme Session  Stochastic Optimization for LargeScale Systems
From Monday 4th November 2019 to Friday 8th November 2019
DOCUMENTARY RESOURCES
Publications by Dimitri BERTSEKAS around Dynamic Programming and Optimal Control
Publications by Pierre CARPENTIER, JeanPhilippe CHANCELIER, Guy COHEN, Michel DE LARA around Stochastic Optimization
Publications by R. Tyrrell ROCKAFELLAR around Optimization und uncertainty
Publications by Joseph Frédéric BONNANS around Stochastic Programming
Publications by Alexander SHAPIRO on Stochastic Programming
Publications by Dimitri BERTSEKAS around Dynamic Programming and Optimal Control
Publications by Pierre CARPENTIER, JeanPhilippe CHANCELIER, Guy COHEN, Michel DE LARA around Stochastic Optimization
Publications by R. Tyrrell ROCKAFELLAR around Optimization und uncertainty
Publications by Joseph Frédéric BONNANS around Stochastic Programming
Publications by Alexander SHAPIRO on Stochastic Programming
AUDIOVISUAL RESOURCES

An introduction to BSDE
By Peter IMKELLER Backward stochastic differential equations have been a very successful and active tool for stochastic finance and insurance for some decades. More generally they serve as a central method in applications of control theory in many areas. We introduce BSDE by looking at a simple utility optimization problem in financial stochastics. We shall derive an important class of BSDE by applying the martingale optimality principle to solve an optimal investment problem for a financial agent whose income is partly affected by market external risk. We then present the basics of existence and uniqueness theory for solutions to BSDE the coefficients of which satisfy global Lipschitz conditions.


LargeScale Machine Learning and Convex Optimization (1/2  2/2)
By Francis BACH Many machine learning and signal processing problems are traditionally cast as convex optimization problems. A common difficulty in solving these problems is the size of the data, where there are many observations ("large n") and each of these is large ("large p"). In this setting, online algorithms such as stochastic gradient descent which pass over the data only once, are usually preferred over batch algorithms, which require multiple passes over the data. Given n observations/iterations, the optimal convergence rates of these algorithms are O(1/n‾√)O(1/n) for general convex functions and reaches O(1/n)O(1/n)for stronglyconvex functions. In this tutorial, I will first present the classical results in stochastic approximation and relate them to classical optimizationand statistics results. I will then show how the smoothness of loss functions may be used to design novel algorithms with improved behavior, both in theory and practice: in the ideal infinitedata setting, an efficient novel Newtonbased stochastic approximation algorithm leads to a convergence rate of O(1/n)O(1/n)without strong convexity assumptions, while in the practical finitedata setting, an appropriate combination of batch and online algorithms leads to unexpected behaviors, such as a linear convergence rate for strongly convex problems, with an iteration cost similar to stochastic gradient descent.
