This course provides an in-depth study of diffusion models and their applications in generative AI. Students will gain a solid understanding of the principles and theories behind diffusion-based AI, while also developing programming skills through hands-on project experience. With an emphasis on real-world applications, the course equips students to apply diffusion models in fields like image generation, natural language processing, and beyond.
- Instructor: Siyu Zhu
- Teaching Assistants: Kaihui Cheng, Hui Li
| Lecture | Topic | Content | Slides | Course Materials |
|---|---|---|---|---|
| 1 | Introduction | Course Objectives, Features, and Overall Content | L1-introduction.pdf | N/A |
| 2 | Foudations of Generative AI (1) | VAE, GAN, Flow-Based Models, and Applications | L2-Foudations-of-Generative-AI-(1).pdf | N/A |
| 3 | Foudations of Generative AI (2) | Probability Distributions, Random Variables | L3-Foudations-of-Generative-AI-(2).pdf | N/A |
| 4 | Fundamentals of Diffusion Models (1) | Diffusion Model Principles, DDPMs, DDIMs, SGMs, Score SDEs, VDMs | Fundamentals-of-Diffusion-Models.pdf | N/A |
| 5 | Fundamentals of Diffusion Models (2) | Model Structures for Generation Processes: Unet, DiT, Latent Space | Fundamentals-of-Diffusion-Models.pdf | N/A |
| 6 | Machine Learning Frameworks and Tools | Machine Learning Frameworks and Tools: Training, Inference, Optimization | L6-Machine-Learning-Frameworks-and-Tools | N/A |
| 7 | Images + Diffusion Models | LDM, Stable Diffusion, DALL-E, GigaGAN | L7-Images-Diffusion-Models | N/A |
| 8 | Audio + Diffusion Models | Audio Diffusion Models, VideoDiffusion, Sora | ||
| 9 | 3D + Diffusion Models | NeRF, 3D-VAE, DreamFusion | ||
| 10 | Biology (Structure) + Diffusion Models | AlphaFold3, ESMFold, RFdiffusion, SE(3) Diffusion | ||
| 11 | Physics and Meteorology + Diffusion Models | GraphCast, Pangu-Weather, NowcastNet, Fuxi | ||
| 12 | Advanced Topics in Diffusion Models (1) | External Expert Talk | ||
| 13 | Advanced Topics in Diffusion Models (2) | External Expert Talk | ||
| 14 | Paper and Project Presentations | Student Presentations of Papers and Projects | ||
| 15 | Paper and Project Presentations | Continuation of Student Presentations | ||
| 16 | Paper and Project Presentations | Final Student Presentations |
Stanley Chan,Tutorial on Diffusion Models for Imaging and Vision, arXiv preprint:2403.18103,2024
-
Attendance (5%)
Your presence in class is important. Attendance will be tracked throughout the course and will contribute 5% to your final grade. -
Class Participation (5%)
Active participation in discussions and in-class activities is highly encouraged. This includes contributing meaningfully to classroom conversations, group discussions, and peer feedback. -
Literature Review (45%)
A significant portion of your grade will be based on a thorough literature review. The review should focus on existing work in the field of generative models and their applications. Details on the structure, submission, and grading criteria will be provided in the assignments section. -
Course Project (45%)
The course project is designed to give you hands-on experience with generative AI models. You will be required to develop a project that incorporates diffusion models and aligns with your academic or research interests.
Students are encouraged to create innovative projects that integrate generative models with their respective academic disciplines, aiming for interdisciplinary applications.
If you have any questions or feedback, feel free to reach out:
- GitHub Issues: Submit an Issue
- Email: [email protected] ; [email protected] ; [email protected]