A Decoupling Principle in Stochastic Optimal Control and Its Implications

February 28, 2020 – 11:15 AM
TSRB auditorium

Abstract

The problem of Stochastic Optimal Control is ubiquitous in Robotics and Control since it is the fundamental formulation for decision-making under uncertainty. The answer to the problem can be computed by solving an associated Dynamic Programming (DP) problem. Unfortunately, the DP paradigm is also synonymous with the infamous “Curse of Dimensionality (COD),” a phrase coined by the discoverer of the Dynamic Programming paradigm, Richard Bellman, nearly 60 years ago, to capture the fact that the computational complexity of solving a DP problem grows exponentially in the dimension of the state space of the problem.

In this talk, we will introduce a newly discovered paradigm in stochastic optimal control, called “Decoupling,” that allows us to separate the design of the open and closed loops of a stochastic optimal control problem with continuous control space. This Decoupled solution allows us to break the COD inherent in DP problems, while remaining near-optimal, to third order, to the true stochastic control. The implications of the Decoupled design are examined in the context of Model Predictive Control (MPC) and Reinforcement Learning (RL). We shall introduce two algorithms, called the Trajectory Optimized Perturbation Feedback Control (T-PFC), and the Decoupled Data based Control(D2C), for the MPC and RL problems respectively. We shall also examine the consequences of the decoupling principle in partially observed/ belief space planning problems and present the Trajectory optimized Linear Quadratic Gaussian (T-LQG) algorithm.

Biography

Suman Chakravorty obtained his B.Tech in Mechanical Engineering in 1997 from the Indian Institute of Technology, Madras and his PhD in Aerospace Engineering from the University of Michigan, Ann Arbor in 2004. From August 2004- August 2010, he was an Assistant Professor with the Aerospace Engineering Department at Texas A&M University, College Station and since August 2010, he has been an Associate Professor in the department. Dr. Chakravorty’s broad research interests lie in the estimation and control of stochastic dynamical systems with application to autonomous, distributed robotic mapping and planning, and situational awareness problems. He is a member of AIAA, ASME and IEEE. He is an Associate Editor for the ASME Journal on Dynamical Systems, Measurement and Control and the IEEE Robotics and Automation Letters.