#### Bridging Machine Learning and Controls for Cyber-Physical Systems

Machine learning and control theory are two foundational but disjoint communities. For example, machine learning requires data to produce models, and control theory requires models to provide stability and performance guarantees. The challenge now, with using data-driven approaches, is to close the loop for real-time control and decision making.

We present novel data-driven approaches to synthesize control-oriented models that bridge machine learning and controls. While there are many areas like building control, process control, autonomous systems etc. where this finds application, our current focus is on its application to building control, in particular demand response and energy management.

##### Application – Demand Response and Energy Management

"Essentially, all models are wrong, but some are useful."- George E. P. Box

In January 2014, the east coast (PJM) electricity grid experienced an 86X increase in the price of electricity from $31/MWh to$2,680/MWh in a matter of 10 minutes. This extreme price volatility has become the new norm in our electric grids. Building additional peak generation capacity is not environmentally or economically sustainable. Thus, the focus has shifted from energy efficiency to energy flexibility.

Our AI platform provides energy and cost savings by strategically shifting loads, shaving peak demands and automatic climate control while tracking volatility in the price of electricity. It learns from historical energy usage patterns to make recommendations on how to best choose equipment settings across thousands of controllers to reduce power consumption while ensuring custom comfort conditions.

We generate predictive models using Random Forests and Gaussian Processes for finite-time receding horizon control - where we can not only predict the state of the building, but also generate control strategies using only historical weather, schedule, set-points and electricity consumption data.

##### Learning and Control using Gaussian Processes

Building physics-based models of complex physical systems like buildings and chemical plants is extremely cost and time prohibitive for applications such as real-time optimal control, production planning and supply chain logistics. Machine learning algorithms can reduce this cost and time complexity, and are, consequently, more scalable for large-scale physical systems. However, there are many practical challenges that must be addressed before employing machine learning for closed-loop control.

To this end, we propose the use of Gaussian Processes (GP) for learning control-oriented models. We present data-driven methods

• for optimal experiment design (OED) of functional tests to learn dynamics of a real building subject to stringent operational constraints,
• to synthesize control-oriented models for receding horizon control, and
• to continuously improve the learned model in closed-loop with a real-time controller.

Read our paper or listen to my 20 min talk from ICCPS 2018 to learn about our approach to above three challenges and why GPs form an appropriate choice.

##### Learning and Control using Random Forests

The central idea behind data-driven MPC is to obtain control-oriented models using machine learning or black-box modeling, and formulate the control problem in a way that receding horizon control (RHC) can still be applied and the optimization problem can be solved efficiently.

Consider a black-box model (of a dynamical system) given by $x_{k+1}=f(x_k,u_k,d_k)$, where $x,u,d$ represent states, inputs and disturbances, respectively. Depending upon the learning algorithm, $f$ is typically nonlinear, nonconvex and sometimes nondifferentiable (as is the case with regression trees and random forests) with no closed-form expression. Such functional representations learned through black-box modeling may not be directly suitable for control and optimization as the optimization problem can be computationally intractable, or due to nondifferentiabilities we may have to settle with a sub-optimal solution using evolutionary algorithms. These problems can be eliminated by decomposing $f(x_k,u_k,d_k)= g(d_k,x_k,h(u_k)),$ where both $g$ and $h$ are learned using the data, and $h(u_k)$ is convex and differentiable, and thus suitable for optimization. Our control algorithm with Random Forests (below) exploits this functional decomposition or separation of variables to overcome the aforementioned challenges with black-box optimization.

Checkout my talk at Microsoft Research Redmond to know the mathematical details. Also refer to our papers published in ACC 2017, CDC 2017, and Applied Energy 2018 .

#### Interactive Analytics

Today’s energy dashboards are static and only process historic data or provide analyses that are baked in. For example, they are used for simple data analytics, monitoring, visualization or anomaly detection. These analyses are formulated using recommended guidelines, experience and best practices. What building operators are interested in knowing is what will happen in the future.

Interactive Analytics or IAX is an energy analytics engine that learns from past building usage patterns to answer queries about prediction and control set point recommendations. It uses Amazon Alexa, a cloud service which allows for natural language interaction, to procedurally generate dashboards in response to user queries. Using our AI platform, it can not only predict the state of the building but also generate control strategies using only historical weather, schedule, set-points and electricity consumption data.

IAX is as a Siri for querying buildings’ energy usage. It provides an easy way to increase financial rewards and reduce participation risk in Demand Response programs. It predicts power consumption and generates optimal curtailment strategies with confidence.

#### Local Interpretability

More than the accuracy of synthesizing control strategies using a black-box model, the building operators are interested in solutions that are also interpretable and trustworthy. Thus, the control recommendations should have traceability so they can be verified to be stable and safe.

There is no true interpretation of anything; interpretation is a vehicle in the service of human comprehension. - Andreas Buja

There is always a trade-off between the accuracy and interpretability of the black-box models trained using machine learning. For example, random forests or neural networks, however accurate, cannot explain why a particular prediction should be trusted. On the other hand, decision trees are highly interpretable because of the structure of the algorithm but they tend to overfit very easily and thus result in a poor accuracy. The goal here is to investigate possibility of explaining predictions from any training method like random forests and neural networks.