From: 2023-02-23 15:30 to 16:30
Place: Seminar Room KC 3N27 at Dept. of Automatic Control
Contact: venkatraman [dot] renganathan [at] control [dot] lth [dot] se

When: Feb. 23, 15:30-16:30
Where: Seminar Room KC 3N27
Speaker: Ingvar Ziemann
Title: On the Fundamental Limits of Learning to Control

Abstract: Learning algorithms are set to play an ever increasing part in modern, often safety-critical, applications that have traditionally been within the purview of controls.  However, the emergence of learning algorithms in safety-critical systems in not without problems  and their failure modes remain poorly understood. In this talk we will discuss when---and perhaps more importantly---when we cannot hope to successfully apply learning algorithms. We will begin by briefly reviewing the role of stability and controllability in estimating a dynamical system. Using the linear quadratic regulator (LQR) as a case study, we then proceed to discuss when learning to control is difficult. In the adaptive (or online) control setting, focusing on regret minimization, we will demonstrate that ill-conditioned systems are hard to learn to control. More precisely we show that: i) LQR optimal closed-loop controllers that are near marginally stable lead to arbitrarily large regret; ii) poorly controllable systems are hard to learn---regret can be exponential in the controllability index of the system; iii) poorly observable are hard to learn. Our proof approach rests on a reduction showing that regret minimization is at least as a difficult as an identification problem with an experiment design constraint on the Fisher information. Time permitting, we will also discuss extensions to the episodic setting and/or policy gradient methods where similar results hold.

Bio: Ingvar Ziemann received his PhD in November 2022 from the Division of Decision and Control Systems at The Royal Institute of Technology (KTH) under the supervision of Henrik Sandberg. His research is centered on using statistical and information theoretic tools to study learning-enabled control methods, with a current interest in studying how learning algorithms generalize in the context of dynamical systems. Prior to starting his Ph.D., he obtained two sets of Master's and Bachelor's degrees in Mathematics (SU/KTH) and in Economics and Finance from the Stockholm School of Economics (SSE). Ingvar is the recipient of a Swedish Research Council International Postdoc Grant, the IEEE CDC 2022 Best Student Paper Award, and the 2017 Stockholm Mathematics Center Excellent Master Thesis Award.