Deep learning is the study and use of artificial neural networks, which are finite dimensional spaces of non-linear functions. Although neural nets have been around since the 1940’s, they have recently achieved state-of-the-art in a variety of machine learning tasks, ranging from machine vision to natural language processing, and reinforcement learning. The fact that they work at all is a bit surprising and leads to a number of interesting mathematical and statistical questions.
I will begin by defining neural networks and giving a sense of how they are used in practice. I will then focus on what I think are several important problems related to approximation theory, random matrix theory, and optimization. In each case, I will explain a bit of what is known.
Deep Learning for Mathematicians