Safety and Optimality Monitors for Learning-enabled Systems using Conformal Prediction

Abstract: The use of machine learning to create data-driven plant models and controllers has led to an increased need for safety and optimality monitors for model-based systems. System plant models are subject to uncertainty due to learning constraints, such as unseen data and overfitting, or physical constraints, such as unknown dynamics and noise. This uncertainty is detrimental to safety-critical systems and must be properly regulated. To curb this uncertainty, we create prediction sets using the guarantees provided by Conformal Prediction. With a user-specified high probability, these prediction sets contain the true plant system states for an entire prediction horizon, which we verify theoretically and empirically. Additionally, we use the same conformal guarantees to create prediction sets for a Neural Network representation of a Model Predictive Controller, which utilizes large amounts of data to create a map of states to outputs without the normal computation required by an MPC controller. These guarantees ensure that, with a user-specified high probability, our prediction set will contain our true optimal MPC controller. As a result, we create optimality and safety monitors for our Neural Network MPC controller using Conformal Prediction. These case studies on model uncertainty and optimality can also be extended beyond the Neural Network MPC controller and provide guarantees on any multi-step predictive control strategy, making us successful in creating a general framework for monitoring safety and optimality on any multi-step system subject to uncertainty.