Algorithmic Foundations of Interactive Learning
Spring 2025: 17-740
Tuesday / Thursday 11:00-12:20
Announcements 📣
Week 1 Announcements
We can’t wait to meet you! 👋
Course Overview 📝
Interactive learning is a dynamic approach to machine learning where systems learn and adapt through continuous interaction with their environment or users, receiving feedback and adjusting their behavior in real time. These techniques are currently experiencing a resurgence across various domains of artificial intelligence and machine learning, from robotics to language modeling. In this advanced theory course, students will explore interactive learning from its foundational principles to recent applications, including fine-tuning Large Language Models (LLMs) and robot learning from demonstration. Key topics include:
- Online Learning: Learning under distribution shift.
- Game Solving: Using no-regret algorithms to compute equilibria.
- Reinforcement Learning: Sequential decision making. Model-free, model-based, and hybrid RL.
- Imitation Learning & Applications to Robotics: Learning from demonstrations. Behavioral cloning, DAgger, and inverse RL.
- RL from Human Feedback & Applications to Language Modeling: Learning from preferences. PPO, DPO, SPO.
Schedule (Tentative) 📅
| Unit | Date | Lecturer | Topic | Reading |
|---|---|---|---|---|
| Online Learning | 1/14/2025 | Steven | Course Overview | |
| 1/16/2025 | Drew | Intro to No-Regret Online Learning | ||
| 1/21/2025 | Steven | Online Gradient Descent | ||
| 1/23/2025 | Steven | Hedge / Multiplicative Weights | ||
| 1/28/2025 | Steven | Follow-the-Leader | ||
| 1/30/2025 | Steven | Bandit Feedback: Exp.3 / Exp.4 | ||
| Game Theory | 2/4/2025 | Steven | Regret Minimization | |
| 2/6/2025 | Steven | Minimax Equilibria | ||
| Reinforcement Learning | 2/11/2025 | Steven | Foundations of MDPs | |
| 2/13/2025 | Gokul | API and CPI | ||
| 2/18/2025 | Gokul | Policy Gradients | ||
| 2/20/2025 | Gokul | Natural Policy Gradients | ||
| 2/25/2025 | Gokul | PPO & TRPO | ||
| 2/27/2025 | Guest Lecture: Yuda Song | Hybrid RL | ||
| Spring Break | ||||
| 3/11/2025 | Gokul | Simulation Lemma | ||
| 3/13/2025 | Gokul | Practical Model-based RL | ||
| Imitation Learning | 3/18/2025 | Gokul | DAgger | |
| 3/20/2025 | Gokul | Inverse RL | ||
| 3/25/2025 | Gokul | Fast Inverse RL | ||
| 3/27/2025 | Guest Lecture: Sanjiban Choudhury | Practical IL | ||
| RLHF | 4/1/2025 | Gokul | RLHF I | |
| 4/3/2025 | Gokul | RLHF II | ||
| 4/8/2025 | Guest Lecture: Wen Sun | REBEL / REFUEL | ||
| 4/10/2025 | Gokul | SPO | ||
| Proj. Presentations | 4/15/2025 | Proj. Presentations | ||
| 4/17/2025 | Proj. Presentations | |||
| 4/22/2025 | Proj. Presentations | |||
| 4/24/2025 | Proj. Presentations |
Instructors 👨🏫
Resources 📚
Related Courses
- Intro to Robot Learning at Cornell
- Foundations of Reinforcement Learning at Cornell
- Learning in Games (and Games in Learning) at UPenn