Transcript – Trade-Offs Between Approximation and Generalization in Learning Systems

Hi, this is Shahin Shahrampour from the industrial and systems engineering department. I present our progress on trade between approximation and generalization in learning systems. Today, I report on a joint work with my Ph.D. student in Yinsong Wang, where we have focused on kernel approximation using random features. Random features significantly improve the computational cost of kernel learning. But the performance may not be on par with kernel methods. Therefore, to improve the approximation accuracy, a number of recent studies have explored the idea of data dependent sampling of random features, which modifies this forecasting oracle from which random features are sampled. The proposed techniques improve the approximations, but their suitability is often verified on a single machine learning task. In this work, we propose a task, a specific scoring rules for selecting random features which can be employed for different applications. With some adjustments, we restrict our attention to canonical correlation analysis, and we provide a novel principle guide for finding this core function that maximizes the canonical correlations with theoretical proof that this method can outperform kernel can only correlations and expectations sets. And we also provide numerical experiments on benchmark datasets that support our theory. Together with another member of the Triad, Dr. Simon Foucart in the math department, we are currently investigating learning problems from an optimal recovery perspective. This framework considers learning with non-random data over generalization and statistical, says Winola, is no longer applicable. We have looked at the notion of worst-case error in Hubert’s spaces and showed that optimal recovery provides a formula that is User-Friendly. From an algorithmic point of view, our study is still ongoing and we are looking at several other important research questions in this area. At the end, I would like to gratefully acknowledge the support of the T3 program, which has helped us pursue this multidisciplinary research project.