Collaborative Research: CIF: Medium: Foundations of Robust Deep Learning via Data Geometry and Dyadic Structure

Deep learning is a subfield of machine learning that uses artificial neural networks to learn patterns from data. Deep learning methods currently achieve the best performance in important problems such as facial recognition, image processing, speech recognition, and machine translation and have also been used in scientific problems in space weather, molecular biology, chemistry, and the health sciences. Despite this success, deep-learning methods are still widely viewed as black boxes, where their decisions are difficult to interpret. Furthermore, there is little mathematical understanding about how to design an effective deep neural network. This project will advance this understanding by investigating how the geometry of the data can be used to better understand and improve deep-learning design strategies. These advances will lead to improved performance in deep learning in scientific fields with large societal impact such as medicine, public health, molecular biology, and chemistry. The research project also involves the mentoring of doctoral students and the material generated from the project will be incorporated into multiple courses on data science. In this project, the team of researchers will develop a comprehensive theoretical framework for harnessing data geometry, graphs, and invariance to enhance performance and interpretability for a wide scope of deep learning problems. This will include investigating the effects of neural network transformations on the data geometry, which is captured by applying manifold learning at different layers in the neural network. New metrics will be designed to empirically test invariance relative to a symmetry group at different layers. These metrics will be used to drive a data-augmentation process that maximizes the learned invariance of the neural network during training. Methods for detecting hidden invariances in data will also be developed that can be used to analyze the neural network training process. By investigating these problems, this project will provide deep-neural-network design guidelines that will lead to a new generation of deep learning that moves beyond heuristics and ensures that a neural network is designed to ensure accuracy, robustness, and interpretability for the learning task. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.