High-dimensional Information Processing: Optimality and Universality

Abstract: The proliferation of large-scale datasets has driven extensive applications of high-dimensional information processing algorithms in various fields. These algorithms, designed for distilling useful information from complex data, need to be grounded in rigorous understanding to ensure they are reliably and efficiently implemented. At the core of such comprehension are two pivotal questions: (1) How to design these algorithms in a principled way to achieve the optimal performance? (2) Can the theoretical analysis done in some idealized settings reflect the algorithms’ effectiveness in practical scenarios?

In this talk, I will present our work investigating these two questions by harnessing some unique and powerful properties emerging in high dimensions. In the first part of the talk, I will focus on signal recovery in the high-dimensional linear models and show how we can design the regularization function of SLOPE algorithm to achieve optimal statistical accuracy. In the second part, I will talk about a universality phenomenon in non-linear machine learning models that allows us to rigorously analyze the training and test errors, using results obtained in the linear models. Throughout the discussions, I will illustrate how the high-dimensional nature of data can be a blessing, enabling precise characterizations and optimal designs of information processing algorithms in practice.

High-dimensional Information Processing: Optimality and Universality

Abstract: The proliferation of large-scale datasets has driven extensive applications of high-dimensional information processing algorithms in various fields. These algorithms, designed for distilling useful information from complex data, need to be grounded in rigorous understanding to ensure they are reliably and efficiently implemented. At the core of such comprehension are two pivotal questions: (1) How to design these algorithms in a principled way to achieve the optimal performance? (2) Can the theoretical analysis done in some idealized settings reflect the algorithms’ effectiveness in practical scenarios?

In this talk, I will present our work investigating these two questions by harnessing some unique and powerful properties emerging in high dimensions. In the first part of the talk, I will focus on signal recovery in the high-dimensional linear models and show how we can design the regularization function of SLOPE algorithm to achieve optimal statistical accuracy. In the second part, I will talk about a universality phenomenon in non-linear machine learning models that allows us to rigorously analyze the training and test errors, using results obtained in the linear models. Throughout the discussions, I will illustrate how the high-dimensional nature of data can be a blessing, enabling precise characterizations and optimal designs of information processing algorithms in practice.