This is the main repository of course materials for MATH 6600 at RPI, Fall 2024. The syllabus is posted in the README below. Lecture notes, homework, exams, and supplementary materials will be posted here or linked through Piazza and Gradescope (RCS access only).
Course description (from RPI Catalog)
Linear vector spaces; eigenvalues and eigenvectors in discrete systems; eigenvalues and eigenvectors in continuous systems including Sturm-Liouville theory, orthogonal expansions and Fourier series, Green’s functions; elementary theory of nonlinear ODEs including phase plane, stability and bifurcation; calculus of variations. Applications will be drawn from equilibrium and dynamic phenomena in science and engineering.
Prerequisites: differential equations (MATH 2400) and advanced calculus (MATH 4600).
Lectures: Monday/Thursday 10–11:50am in Carnegie 206.
Instructor: Andrew Horning
Office Hours: Monday and Thursday, 12:30-1:30pm in Amos Eaton 329.
(Due to holiday, office hours will take place Tuesday 09/03 and not Monday 09/02.)
Contact: [email protected]
Course Tools: Communicate (announcements, questions, and discussion) through Piazza. View and submit homework assignments on Gradescope. The mid-term will be in-class and the final project will be submitted on Gradescope.
Grading: 50% homework (due weekly on Tuesday at 5pm), 25% mid-term (October 17), 25% final project (December 10). Problem sets must be submitted to Gradescope before the deadline on the due date. Regrade requests can be made on Gradescope within one week of the due date.
Collaboration and Academic Integrity: To maximize your learning objectives, reserve time to work on each problem independently before discussing it with your classmates. Always write up the solution on your own and acknowledge your collaborators. Copying solutions directly from peers, books, internet sources, or AI tools is strictly prohibited.
Accomodations and Disability Services: If you have approved accommodations through the Office of Disability Services for Students (DSS), please reach out to meet with me early in the semester. We are committed to equal access for all students and will be happy to facilitate the use of approved accommodations.
Instead of a final exam, write an 8-10 page report that reviews either a "method" (e.g., principle component analysis) or a "problem" (e.g., parabolic partial differential equations) in applied mathematics. Your report should both include:
Review: Why is this method or problem important, what is its history, and what are the important publications and references? (A comprehensive bibliography is expected: not just the sources you happened to consult, but a complete set of sources you would recommend that a reader consult to learn a fuller picture.)
Description: A concise mathematical description of the problem or method you are studying. If you are focusing on a method, you should clearly describe its scope: what problems can it be applied to? What are its strengths and limitations? If you are studying a problem, you should discuss key properties: what is the key structure/behavior in the problem and its solution? What are the primary methods used to make progress solving or analyzing this problem?
Application: Illustrate the key strengths, limitations, and characteristics of your method by working through a concrete instance of a problem OR study your problem by applying one or more methods to analyze and (if possible) construct solutions or approximate solutions.
The report should be written in clear and concise language at a level that is accessible to your fellow classmates. Graphical illustrations of key concepts are highly encouraged when possible. Numerical experiments are encouraged but not required.
Below are some possible candidates for project topics, with references to get you started.
- Randomized NLA: Fast methods for solving problems in linear algebra using randomized algorithms.
- Operator learning: Learn the Green's function of a partial differential equation from input-output data using, e.g., SVD of Green's function.
- Principle Component Analysis (PCA): extensions of PCA and Kernel PCA that are robust to noise, promote sparsity, etc.
- Dynamic Mode Decomposition (DMD): A PCA-like algorithm for data with a time component (many variants like piDMD, eDMD, mpDMD).
- Applied Koopman Theory: Operator framework for analyzing dynamical systems from data. Often applied in tandem with DMD algorithms for data-driven tasks.
- Kernel machines and reproducing kernel Hilbert spaces in machine learning.
- Homework 1 is due by 5pm on Friday, September 6. (Solutions)
- Homework 2 is due by 5pm on Tuesday, September 17. (Solutions)
- Homework 3 is due by 5pm on Tuesday, September 24. (Solutions)
- Homework 4 is due by 5pm on Tuesday, October 1. (Solutions)
- Homework 5 is due by 5pm on Tuesday, October 15. (Solutions)
- Homework 6 is due by 5pm on Tuesday, October 29. (Solutions)
- Homework 7 is due by 5pm on Tuesday, November 5. (Project proposal, no solutions)
- Homework 8 is due by 5pm on Tuesday,
November 12November 19. (Solutions) - Homework 9 is due by 5pm on Tuesday, November 19. (Solutions)
- Models, maps, and methods in applied mathematics
- Vector spaces of functions
- Norms, inner products, and orthogonality
- Orthogonal bases and Gram-Schmidt orthogonalization
- From monomials to Legendre polynomials
- Coordinate transformations and the Gram matrix
- Matrices and quasimatrices
- Linear transformations
- The matrix of a linear transformation
Notes | Gram matrices in action
- Linear equations
- Existence, uniqueness, and sensitivity
- Fundamental subspaces and adjoint transformations
- The four fundamental subspaces
- The Fredholm alternative (n-dimensions)
- Hilbert spaces and completeness
Notes | Review the four subspaces with Gil Strang
- Separable Hilbert spaces and bases
- Square integrable functions and smooth subspaces
- Differentiation and integration in L^2(Omega)
- Bounded operators on Hilbert spaces
- Operators with bounded inverse
- From differential to integral equations
- Criteria for a bounded inverse
- Well-posed integral equations
- Hilbert-Schmidt operators
- More about Hilbert-Schmidt operators
- The SVD of a matrix
- The SVD of a H-S operator
Notes | Learning Hilbert-Schmidt kernels from data
- "Low-rank approximation" of H-S operators
- Fredholm alternative for H-S operators
- The H-S spectral theorem (self-adjoint)
Notes | Singular value expansions for indefinite kernels
- Self-adjoint (SA) kernels
- Eigenvalue expansion of SA kernel
- Principle component analysis (PCA)
- Covariance and principal components
- Directions of maximal variance
- Min-max characterizations of eigenpairs
- Practical aspects of PCA
- Sample covariance matrix
- The SVD of the data matrix
- Limitations of PCA
- Large nonlinear feature maps
- Kernel PCA and Mercer's theorem
Notes | Demo | Further Reading
- Theory of compact linear operators (summary)
- Unbounded operators and the resolvent
- Differential operators with compact resolvent
- Resolvent set and spectrum
- Properties of the resolvent
- Spectral theorem for operators with compact resolvent
- Regular Sturm-Liouville problems
- Singular Sturm-Liouville problems
- Polynomial and series solutions
- Power series solutions for ODEs
- Convergence of power series
- Legendre's equation
- Blow up at singular points
- Modified power series solutions
- Bessel's equation
- Bessel Functions
- Fuch's Theorem
- Separable PDEs
- More about Bessel functions
- Steady-state temperature in a cylinder
- Vibrations in a circular membrane
- Separable operators and domains
- The structure of separable solutions
- Eigenpairs of separable operators
- Diagonalizing separable operators
- Stationary PDEs
- Time-dependent PDEs
- Functions of operators
- Exponential of a normal operator
- Eigenvalues, eigenfunctions, and well-posedness
- Strongly continuous semigroups
- Spectral analysis of semigroups
- Limitations of spectral analysis