Online Optimization and Control using Black-Box Predictions

April 19, 2022 11:00 am – 12:00 pm

Location: Instructional Center 105

Also live streamed at

Zoom Meeting ID: 968 1345 6832

Adam Wierman




Making use of modern black-box AI tools is potentially transformational for online optimization and control. However, such machine-learned algorithms typically do not have formal guarantees on their worst-case performance, stability, or safety. So, while their performance may improve upon traditional approaches in “typical” cases, they may perform arbitrarily worse in scenarios where the training examples are not representative due to, e.g., distribution shift or unrepresentative training data. This represents a significant drawback when considering the use of AI tools for energy systems and autonomous cities, which are safety-critical. A challenging open question is thus: Is it possible to provide guarantees that allow black-box AI tools to be used in safety-critical applications? In this talk, I will introduce recent work that aims to develop algorithms that make use of black-box AI tools to provide good performance in the typical case while integrating the “untrusted advice” from these algorithms into traditional algorithms to ensure formal worst-case guarantees. Specifically, we will discuss the use of black-box untrusted advice in the context of online convex body chasing, online non-convex optimization, and linear quadratic control, identifying both novel algorithms and fundamental limits in each case.


Adam Wierman is a Professor in the Department of Computing and Mathematical Sciences at Caltech. He received his Ph.D., M.Sc., and B.Sc. in Computer Science from Carnegie Mellon University and has been a faculty at Caltech since 2007. Adam’s research strives to make the networked systems that govern our world sustainable and resilient. He is best known for his work spearheading the design of algorithms for sustainable data centers and his co-authored book on “The Fundamentals of Heavy-tails”. He is a recipient of multiple awards, including the ACM Sigmetrics Rising Star award, the ACM Sigmetrics Test of Time award, the IEEE Communications Society William R. Bennett Prize, multiple teaching awards, and is a co-author of papers that have received “best paper” awards at a wide variety of conferences across computer science, power engineering, and operations research.

Control and estimation of ensembles via optimal transport: Structured multi-marginal optimal transport and efficient computations

April 8, 2022 11:00am – 12:00pm

Location: 254 Classroom Skiles

Dr. Johan Karlsson

Associate Professor

Department of Mathematics

KTH Royal Institute of Technology


The optimal mass transport problem is a classical problem in mathematics, and dates back to 1781 and work by G. Monge where he formulated an optimization problem for minimizing the cost of transporting soil for construction of forts and roads. Historically the optimal mass transport problem has been widely used in economics in, e.g., planning and logistics, and was at the heart of the 1975 Nobel Memorial Prize in Economic Sciences. In the last two decades there has been a rapid development of theory and methods for optimal mass transport and the ideas have attracted considerable attention in several economic and engineering fields. These developments have led to a mature framework for optimal mass transport with computationally efficient algorithms that can be used to address problems in the areas of systems, control, and estimation.

In this talk, I will give an overview of the optimal mass transport framework and show how it can be applied to solve problems in state estimation and ensemble control. In particular, I will present a version of the optimal transport problem where the cost function is adapted to take into account the dynamics of the underlying systems. Also duality results between control and estimation will be considered together with illustrative examples. The approach is non-parameteric and can be applied to problems ranging from a multi-agent systems to a continuous flow of systems. This problem can also be formulated as a multi-marginal optimal transport problem and we show how several common problems, e.g., barycenter and tracking problems, can be seen as special cases of this. This naturally leads to consider structured optimal transport problems, which both can be used to model a rich set of problems and can be solved efficiently using customized methods inspired by the Sinkhorn iterations. This also connects to the Schrödinger bridge problem and ensemble hidden Markov models.


Johan Karlsson received an MSc degree in Engineering Physics from KTH in 2003 and a PhD in Optimization and Systems Theory from KTH in 2008. From 2009 to 2011, he was with Sirius International, Stockholm. From 2011 to 2013 he was working as a postdoctoral associate at the Department of Computer and Electrical Engineering, University of Florida. From 2013 he joined the Department of Mathematics, KTH, as an assistant professor and since 2017 he is working as an associate professor. His current research interests include inverse problems, methods for large scale optimization, and model reduction, for applications in remote sensing, signal processing, and control theory.

Space Debris Propagation, Prediction, and Removal

March 11, 2022 11:00am

Zoom Meeting:

Meeting ID: 928 6895 7681

Passcode: 841672

Dr. Xiaoli Bai

Associate Professor

Department of Mechanical and Aerospace Engineering

Rutgers, The State University of New Jersey


Since the launch of the first satellite (Sputnik 1) in 1957, humans have created a lot of objects in orbit around Earth. The estimated number of space objects larger than 10 cm is presently approaching 37,000, 1000000 between 1 and 10cm, and for objects smaller than 1cm the number exceeds 330 million. Both the number of space objects and the number of conflicts between these objects are increasing exponentially.

This talk overviews the research we have been pursuing on to address the challenges posed by the growth of space debris. We will first introduce the Modified Chebyshev-Picard Iteration (MCPI) Methods, which are a set of parallel-structured methods for solution of initial value problems and boundary value problems. The MCPI methods have been recommended as the “promising and parallelizable method for orbit propagation” by the National Research Council. The talk will then highlight our recent results to develop a physics-based learning approach to predict space objects’ trajectories with higher accuracy and higher reliability than those of the current methods. Last, we will present our research in autonomous, performance-driven, and online trajectory planning and tracking of space robotics for space debris removal with the goal to solve the problem in real time.


Dr. Xiaoli Bai is an Associate Professor in the department of Mechanical and Aerospace Engineering at Rutgers, The State University of New Jersey. She obtained her PhD degree of Aerospace Engineering from Texas A&M University. Her current research interests include astrodynamics and Space Situational Awareness; spacecraft guidance, control, and space robotics; and Unmanned Aerial Vehicle navigation and control. She was an Associate Fellow for the Class of 2021 in the American Institute of Aeronautics and Astronautics (AIAA), a recipient of the 2019 NASA Early Career Faculty award, The 2016 Air Force Office of Scientific Research Young Investigator Research Program award, Outstanding Young Aerospace Engineer Award from Texas A&M University in 2018, A. Water Tyson Assistant Professor Award from Rutgers in 2018, and Amelia Earhart Fellowship.

When is altruism good in distributed decision-making?

February 25, 2022 11:00am

Location: Technology Square Research Building (TSRB) 509


The web of interconnections between today’s technology and society is upending many traditional ways of doing things: the internet of things, bitcoin, the sharing economy, and connected autonomous vehicles are increasingly in the public mind. As such, computer scientists and engineers must be increasingly conscious of the interplay between the technical performance of their systems and the personal objectives of users, customers, and adversaries. I will present our recent work studying the design tradeoffs faced by a planner who wishes to influence and optimize the behavior of a group of self-interested individuals. A key goal is to balance the potential benefits of implementing a given behavior-influencing scheme with its potential costs; we seek to systematically avoid schemes which are likely to create perverse incentives. We will explore these concepts in two contexts: routing of autonomous vehicles in mixed-autonomy transportation networks and decision design for communication-denied distributed multiagent systems. We will ask “when is altruism good?” through the lens of the Price of Anarchy and a new conceptual framework that we term the “Perversity Index,” which captures the potential harm that an incentive scheme may cause. Ultimately, we seek to develop foundations for a theory of robust socially-networked systems which leverages the decision processes of automated components to enhance overall system reliability and performance.


Philip Brown is an Assistant Professor in the Department of Computer Science at the University of Colorado at Colorado Springs. He received the PhD in Electrical and Computer Engineering from the University of California, Santa Barbara under the supervision of Jason Marden. He received the Master and Bachelor of Science in Electrical Engineering from the University of Colorado at Boulder and Georgia Tech (respectively), between which he developed process control technology for the biofuels industry. Philip is interested in the impact of human social behavior on the performance of large-scale infrastructure and software systems, and studies this by combining concepts from game theory and feedback control of distributed systems. Philip was a finalist for best student paper at IEEE CDC in 2016 and 2017, received the 2018 CCDC Best PhD Thesis award from UCSB, and the Best Paper Award from GameNets 2021.

Cyberattack Detection through Dynamic Watermarking

November, 16, 2021, 11:00 am – 12:00 pm

Location: Instructional Center 215

Also live streamed at


Dynamic watermarking, as an active intrusion detection technique, can potentially detect replay attacks, spoofing attacks, and deception attacks in the feedback channel for control systems. In this talk, we will discuss our recent work on a novel dynamic watermarking algorithm for finite-state finite-action Markov decision processes and present bounds on the mean time between false alarms, and the mean delay between the time an attack occurs and when it is detected. We further compute the sensitivity of the performance of the control system as a function of the watermark. We demonstrate the effectiveness of the proposed dynamic watermarking algorithm by detecting a spoofing attack in a sensor network system.


Abhishek Gupta is an assistant professor at Electrical and Computer Engineering at The Ohio State University. He completed his Ph.D. in Aerospace Engineering (2014), MS in Applied Mathematics (2012), and MS in Aerospace Engineering (2011), all from University of Illinois at Urbana-Champaign (UIUC). He completed his undergraduate in Aerospace Engineering from Indian Institute of Technology, Bombay, India (2005-09). His research develops new theory and algorithms for stochastic control problems, games, and optimization problems, with applications to secure cyberphysical systems and develop market mechanisms for deep renewable integration. He is a recipient of Kenneth Lee Herrick Memorial Award at UIUC and Lumley Research Award at OSU.

Enabling Bipedal Locomotion with Robotic Assistive Devices through Learning and Control

October 29, 2021 11:00am-12:30pm

Location: Pettit Microelectronics Building, 102A&B Conference Room


Lower-body exoskeletons and prostheses have the potential of restoring autonomy to millions of individuals with ambulatory disabilities, and thereby improving quality of life. However, achieving locomotive stability is challenging enough from a robotics and control perspective, let alone addressing the added complexity of satisfying subjective gait preferences of a human user. Thus, the goal of this talk is to discuss how theoretic approaches to bipedal locomotion, based upon nonlinear controllers with formal guarantees of stability, can be coupled with learning to achieve user-preferred stable locomotion.

In my talk, I will first discuss the theoretical underpinnings of achieving provably stable locomotion via nonlinear control theory, and apply this methodology to various dynamic robotic platforms experimentally. Then, I explore the unification of preference-based learning with this formal approach to locomotion to explicitly optimize user preference and comfort. Finally, I experimentally demonstrate the combined framework on a full lower-body exoskeleton, with both non-disabled subjects and subjects with complete motor paraplegia. This work, therefore, demonstrates the utility of coupling preference-based learning with control theory in a structured fashion with a view towards practical application. Ultimately, this result provides a formal approach for achieving locomotive autonomy with robotic assistive devices that has the potential to accelerate clinical implementation and enable the use of these devices in everyday life.


Maegan Tucker is currently a PhD Candidate in the Mechanical and Civil Engineering Department at the California Institute of Technology (Caltech). She received her Bachelor of Science in Mechanical Engineering from Georgia Tech in 2017. Her research is centered around developing systematic methods of achieving stable, robust, and natural bipedal locomotion on lower-body assistive devices, as well as developing human-in-the-loop methods to customize the experimental locomotion based on subjective user feedback.

Real-time Distributed Decision Making in Networked Systems

September 17, 2021 10-11am


Na Li

Gordon McKay professor

Electrical Engineering and Applied Mathematics

Harvard University


Recent revolutions in sensing, computation, communication, and actuation technologies have been boosting the development and implementation of data-driven decision making, greatly advancing the monitoring and control of complex network systems. In this talk, we will focus on real-time distributed decision-making algorithms for networked systems. The first part will be on the  scalable multiagent reinforcement learning algorithms and the second part will be on the model free control methods for power systems based on continuous time zeroth-order optimization methods. We will show that exploiting network structure or underlying physical dynamics will facilitate the design of scalable real-time learning and control methods.


Na Li is a Gordon McKay professor in Electrical Engineering and Applied Mathematics at Harvard University.  She received her Bachelor degree in Mathematics from Zhejiang University in 2007 and Ph.D. degree in Control and Dynamical systems from California Institute of Technology in 2013. She was a postdoctoral associate at Massachusetts Institute of Technology 2013-2014.  Her research lies in control, learning, and optimization of networked systems, including theory development, algorithm design, and applications to real-world cyber-physical societal system. She received NSF career award (2016), AFSOR Young Investigator Award (2017), ONR Young Investigator Award(2019),  Donald P. Eckman Award (2019), McDonald Mentoring Award (2020), along with some other awards.

The Rational Selection of Goal Operations and the Integration of Search Strategies with Goal-Driven Marine Autonomy

September 9, 2021 1-2pm in Marcus Nanotechnology room 1117 – 1118

Video Recording:

Michael T. Cox

Research Professor

Department of Computer Science and Engineering

Wright State University, Dayton


Intelligent physical systems as embodied cognitive systems must perform high-level reasoning while concurrently managing an underlying control architecture. The link between cognition and control must manage the problem of converting continuous values from the real world to symbolic representations (and back). To generate effective behaviors, reasoning must include a capacity to replan, acquire and update new information, detect and respond to anomalies, and perform various operations on system goals. But, these processes are not independent and need further exploration. This paper examines an agent’s choices when multiple goal operations co-occur and interact, and it establishes a method of choosing between them. We demonstrate the benefits and discuss the trade offs involved with this and show positive results in a dynamic marine search task.


Michael T. Cox is a Research Professor in the Department of Computer Science and Engineering at Wright State University, Dayton and was the founding Director of Autonomy Research at Parallax Advanced Research. Dr. Cox is a former DARPA/I2O program manager and has held senior research positions at Raytheon BBN Technologies, the CMU School of Computer Science, and the University of Maryland Institute for Advanced Computer Studies. He has interests in autonomous systems, mixed-initiative planning, computational metacognition, and case-based reasoning. His research group developed the Metacognitive Integrated Dual-Cycle Architecture (MIDCA) and was instrumental in the development of a high-level approach to autonomy called goal-driven autonomy. He graduated summa cum laude in Computer Science (1986) from Georgia Tech and holds a PhD in Computer Science (1996) from the same.

Objective Learning for Autonomous Systems

April 9, 2021 – 2:00 pm in Bluejeans:

Shaoshuai Mou

Purdue University


Autonomous systems especially those driven under optimal control are usually associated with objective functions to describe their goals/tasks in specific missions. Since they are usually unknown in practice especially for complicated missions, learning such objective functions is significant to autonomous systems especially in their imitation learning and teaming with human. In this talk we will introduce our recent progress in objective learning based on inverse optimal control and inverse optimization, especially their applications in human motion segmentation, learning from sparse demonstrations, and learning with directional corrections. We will also present an end-to-end learning framework based on Pontryagin Principle, feedbacks and optimal control, which is able to treat solving inverse optimization, system identification, and some control/planning tasks as its special modes.


Dr. Shaoshuai Mou is an Assistant Professor in the School of Aeronautics and Astronautics at Purdue University. Before joining Purdue, he received a Ph.D. in Electrical Engineering at Yale University in 2014 and worked as a postdoc researcher at MIT for a year after that. His research interests include multi-agent autonomy and learning, distributed algorithms for control and optimization, human-machine teaming, resilience & cybersecurity, and also experimental research involving autonomous air and ground vehicles. Dr. Mou co-direct Purdue’s new Center for Innovation in Control, Optimization and Networks (ICON), which consists of more than 50 faculty and aims to integrate classical theories in control/optimization/networks with recent advance in machine learning/AI/data science to address fundamental challenges in autonomous and connected systems. For more information, please refer to

Fastest Identification in Linear Systems

March 19, 2021 – 09:00 am

Alexandre Proutiere

KTH, Stockholm, Sweden


Abstract: We report recent results on two classical inference problems in linear systems. (i) In the first problem, we wish to identify the dynamics of a canonical linear time-invariant systems $x_{t+1}=Ax_t+\eta_{t+1}$ from an observed trajectory. We provide system-specific sample complexity lower bound satisfied by any $(\epsilon, \delta)$-PAC algorithm, i.e., yielding an estimation error less than $\epsilon$ with probability at least $1-\delta$. We further establish that the Ordinary Least Squares estimator achieves this fundamental limit. (ii) In the second inference problem, we aim at identifying the best arm in bandit optimization with linear rewards. We derive instance-specific sample complexity lower bounds for any $\delta$-PAC algorithm, and devise a simple track-and-stop algorithm achieving this lower bound. In both inference problems, the analysis relies on novel concentration results for the spectrum of the covariates matrix.


Alexandre Proutiere is professor in the Decision and Control System division at KTH, Stockholm Sweden since 2011. Before joining KTH he was esearcher at Microsoft Research (Cambridge) from 2007 to 2011,research engineer at France Telecom R&D from 2000 to 2006, Invited lecturer and researcher at the computer science department ENS Paris from 2004 to 2006. He received a PhD in Applied Mathematics from Ecole Polytechnique, graduated in Mathematiques from Ecole Normale Superieure. He also received an engineering degree from Telecom Paris, and is an engineer from Corps des Mines. He won the ACM Sigmetrics rising star award in 2009, ACM best papers awards at Sigmetrics 2004 and 2010, and Mobihoc 2009. His research interests are in probability and their applications, and more specifically today in learning in dynamical systems.