Lecturer: Emily King
Fields: Mathematical methods, data analysis, machine learning, image processing, harmonic analysis
Content
Are you curious about how to extract important information from a data set? Very likely, you will be rewarded if you use some sort of low complexity model in your analysis and processing. A low complexity model is a representation of data which is in some sense much simpler than what the original format of the data would suggest. For example, every time you take a picture with a phone, about 80% of the data is discarded when the image is saved as a JPEG file. The JPEG compression algorithm works due to the fact that discrete cosine functions yield a low complexity model for natural images that tricks human perception. As another example, linear bottlenecks, pooling, pruning, and dropout are all examples of enforcing a low complexity model on neural networks to prevent overfitting. Some benefits of low complexity models include:
- Approximating data via a low complexity model often highlights overall structure of the data set or key features.
- Appropriately reducing the complexity of data as a pre-processing step can speed up algorithms without drastically affecting the outcome.
- Reducing the complexity of a system during a training task can prevent overfitting.
The course will begin with an introduction to applied harmonic analysis, touching on pertinent topics from linear algebra, Fourier analysis, time-frequency analysis, and wavelet/shearlet analysis. Then an overview of low complexity models will be given, followed by specific discussions of
- Linear dimensionality reduction (principal component analysis, Johnson-Lindenstrauss embeddings)
- Sparsity and low rank assumptions (LASSO, l^p norms, k-means clustering, dictionary learning)
- Nonlinear dimensionality reduction / manifold learning (Isomap, Locally Linear Embedding, local PCA)
- Low complexity models in neural networks (linear bottlenecks, pooling, pruning, dropout, generative adversarial networks, Gaussian mean width)
Objectives
The course aims to provide participants with a good understanding of basic concepts and applications of both classical mathematical tools like the Fourier or wavelet transform and more cutting edge methods like dropout in neural networks. A variety of applications and algorithms will be presented. Participants should finish the course with a clearer idea of when and how to use various approaches in data analysis and image processing.
Literature
The linear algebra chapter of MIT’s Deep Learning textbook:
http://www.deeplearningbook.org/contents/linear_algebra.html
Lecturer
Emily King is a professor of mathematics at Colorado State University, reigning IK Powerpoint Karaoke champion, an avid distance runner, and a lover of slow food / craft beer / third wave coffee. Her research interests include algebraic and applied harmonic analysis, signal and image processing, data analysis, and frame theory. In layman’s terms, she looks for the best building blocks to represent data, images, and even theoretical mathematical objects to better understand them. She also has a tattoo symbolizing most of her favorite classes of mathematical objects. If you are curious, you should ask her about it over a beer.
Affiliation: Colorado State University
Website: https://www.math.colostate.edu/~king/