Deep Learning
Course facts
- Payment: Free, no certificate
- Credits: Non Credit
- Learning style: Self-paced, start anytime
- Resources: Lecture Notes; Lecture Videos; Problem Sets; Projects with Examples; Readings
MIT OpenCourseWare details
- As taught in: Fall 2024
- Department: Electrical Engineering and Computer Science
| Course Details | ||||||
|---|---|---|---|---|---|---|
|
||||||
| Outcomes | ||||||
|
||||||
| Price & Terms | ||||||
|
||||||
| Technical | ||||||
|
||||||
| Specification | ||||||
|
||||||
Graduate deep learning: training, generalization theory, CNNs, graph nets, transformers, representation learning, generative models and scaling laws.
Why do deep networks work, and how do you make them work well? This graduate course covers the fundamentals of deep learning in both theory and practice, in recorded lectures with notes and readings: how to train a neural net, approximation and generalization theory, architectures for grids, graphs, memory and transformers, scaling rules for optimization, representation learning, generative models, transfer learning and scaling laws, language models, and practical advice in a hacker’s guide to deep learning.
What You Will Achieve
- Train neural networks with backpropagation and understand the scaling rules behind optimization
- Compare architectures such as CNNs, graph networks, memory models and transformers
- Explain what approximation and generalization theory say about deep networks
- Run and write up an original deep learning experiment that answers a research question
Program Overview
Training and Theory
How networks learn and what theory says about them.
- Introduction and how to train a neural net
- Approximation theory and generalization theory
- Scaling rules for optimization
- A hacker’s guide to deep learning
Architectures and Representations
Networks for different data, and what they learn.
- Architectures for grids, graphs and memory
- Transformers
- Reconstruction-based and similarity-based representation learning
- Representation learning theory
Generative Models and Transfer
Generating data and reusing what models learn.
- Generative models, including conditional models
- Out-of-distribution generalization and transfer learning
- Scaling laws and language models
- Metrized deep learning and inference methods
Course Completion
Job Outlook
Deep learning is used in machine learning research and engineering, computer vision, natural language processing, robotics and applied AI across industries.
Good for: Learners with a machine learning background who want to understand deep learning at the level of current research.
Keep in mind: Only the last of the five homework sets has published solutions, and lecture 22 is missing from the recorded lectures.
Bottom line: A rigorous, up-to-date deep learning course that balances theory with practice and ends in a research project.
Frequently Asked Questions
What is the research project?
A research project on a deep learning question of your choice: you run experiments, analyze them and write the results up as a technical blog post of 2,000 to 3,000 words with four to six figures.
Is there a PyTorch introduction?
Yes. A PyTorch tutorial is included with the lecture videos, alongside the 23 recorded lectures.
What makes a strong project, by the course's own example?
One that asks a real question. The course contrasts a project that only shows transformers beat RNNs at classifying molecules with one that makes a transformer's depth adapt to each molecule's size.
User Reviews
Write your review
You must be logged in to post a review.
Reviews can only be posted by registered learners. Log in or create a free account, then you will return here to post.
There are no reviews yet.