An interactive deep learning book with code, math, and discussions, based on the NumPy interface by Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola. With 900 pages, this seems to be one of the most comprehensive one-stop resources that goes from Linear Neural Networks and Multilayer Perceptrons all the way to modern Deep Learning architectures including Attention Mechanisms and Optimization Algorithms – giving you all three: Theory, Math & Code.
Note: We have bi-weekly remote reading sessions goingthrough all chapters of the book. If you'd like to join check out this blog post and join us on Meetup.
Part I: Mathematical Foundations
- Introduction and Motivation
- Linear Algebra
- Analytic Geometry
- Matrix Decompositions
- Vector Calculus
- Probability and Distribution
- Continuous Optimization
Part II: Central Machine Learning Problems
- When Models Meet Data
- Linear Regression
- Dimensionality Reduction with Principal Component Analysis
- Density Estimation with Gaussian Mixture Models
- Classification with Support Vector Machines
by Terence Parr and Jeremy Howard
Abstract
"This paper is an attempt to explain all the matrix calculus you need in order to understand the training of deep neural networks. We assume no math knowledge beyond what you learned in calculus 1, and provide links to help you refresh the necessary math where needed. Note that you do not need to understand this material before you start learning to train and use deep learning in practice; rather, this material is for those who are already familiar with the basics of neural networks, and wish to deepen their understanding of the underlying math. Don't worry if you get stuck at some point along the way---just go back and reread the previous section, and try writing down and working through some examples. And if you're still stuck, we're happy to answer your questions in the Theory category at forums.fast.ai. Note: There is a reference section at the end of the paper summarizing all the key matrix calculus rules and terminology discussed here."
A visual introduction to probability and statistics.
This is a collection of pages demonstrating the use of the interact command in Sage. It should be easy to just scroll through and copy/paste examples into Sage notebooks.
Examples include Algebra, Bioinformatics, Calculus, Cryptography, Differential Equations, Drawing Graphics, Dynamical Systems, Fractals, Games and Diversions, Geometry, Graph Theory, Linear Algebra, Loop Quantum Gravity, Number Theory, Statistics/Probability, Topology, Web Applications.
by Simon Ward-Jones. A visual 👀 tour of probability distributions.
- Bernoulli Distribution
- Binomial Distribution
- Normal Distribution
- Beta Distribution
- LogNormal Distribution
by Simon Ward-Jones. Explaining the basics of bayesian inference with the example of flipping a coin.
3blue1brown, by Grant Sanderson, is some combination of math and entertainment, depending on your disposition. The goal is for explanations to be driven by animations and for difficult problems to be made simple with changes in perspective.
Matrix Methods in Data Analysis, Signal Processing, and Machine Learning
- Spring 2018
- Level: Undergraduate / Graduate
- Course description: "Linear algebra concepts are key for understanding and creating machine learning algorithms, especially as applied to deep learning and neural networks. This course reviews linear algebra with applications to probability and statistics and optimization–and above all a full explanation of deep learning."
- Format: Video lectures
Imperial College London. "This course offers an introduction to the linear algebra required for common machine learning techniques. We start at the very beginning with thinking about vectors and what vectors are, and the basic mathematical operations we can do with vectors, like how to add vectors. We then move on to think about how to find the product of vectors and what the modulus or size of a vector is. In physical spaces that then lets us think about linear algebra geometrically, and therefore when vectors are perpendicular to eachother or have an angle between then. We can think about the basis – the fundamental vectors that make up a vector space – and how to change basis and transform between vector frames. That then lets us think about how to combine matrix transformations and how to do inverse transformations. That then takes us on to think about the eigenvectors and eigenvalues of a transformation and what these “eigen-things” mean. We then finish up the course by applying all this to a machine learning problem – the google pagerank algorithm."
-
Equations, Functions, and Graphs
-
Differentiation and Optimization
-
Vectors and Matrices
-
Statistics and Probability