Data privacy and coding for machine learning: From federated learning to differentially private online learning

Abstract: With the widespread availability of personal data to machine learning models and the adoption of a multitude of data protection regulations, data privacy has become one of the key concerns in designing machine learning systems.  I will discuss different aspects and notions of data privacy that are suitable for different types of data and learning purposes. First, I will discuss unlearning of clustering models in federated settings, where data privacy is required in two ways:  within the unlearning paradigm that involves the removal of training data points as well as their influence on the learning model and the federated learning paradigm that is concerned with aggregating locally trained models without leaking information. I will present a federated framework that integrates a computation-efficient unlearning mechanism and a communication-efficient algorithm for secure aggregation of sparse local models based on novel ideas borrowed from coding theory. Second, I will discuss classification in hyperbolic spaces in a federated setting and present an algorithm that involves secure aggregation of local models under the presence of label-switching issues that are common in federated settings. Label switching is resolved via the use of Bh codes. If time permits, I will also discuss a recent problem in differentially private online learning.