Syllabus

Table of Contents

  1. Overview
  2. Logistics
  3. Prerequisites
  4. Textbooks
  5. Accommodations for Students with Disabilities
  6. Diversity Statement
  7. Communication
  8. Grading
  9. Health & Wellness

Overview

Interactive learning is a dynamic approach to machine learning where systems learn and adapt through continuous interaction with their environment or users, receiving feedback and adjusting their behavior in response. These techniques are currently experiencing a resurgence across various domains of artificial intelligence and machine learning, from robotics to language modeling. In this advanced theory course, students will explore interactive learning from its foundational principles to recent applications, including fine-tuning Large Language Models (LLMs) and robot learning from demonstration.

Key topics include:

  1. Online Learning: Learning under distribution shift.
  2. Game Solving: Using no-regret algorithms to compute equilibria.
  3. Reinforcement Learning: Sequential decision making. Model-free, model-based, and hybrid RL.
  4. Imitation Learning & Applications to Robotics: Learning from demonstrations. Behavioral cloning, DAgger, and inverse RL.
  5. RL from Human Feedback & Applications to Language Modeling: Learning from preferences. PPO, DPO, SPO.

Logistics

  • Title: Algorithmic Foundations of Interactive Learning, Spring 2025
  • Course Number: 17-740
  • Lecture: 11:00AM–12:20PM EST, Tues & Thurs
  • Office Hours: 2:00PM–3:00PM EST, Friday
  • Location: TBD

Prerequisites

This is a theory-oriented course, intended for graduate students and advanced undergraduates and therefore requires mathematical maturity. The (informal) prerequisite is familiarity with machine learning, algorithms, optimization, probability, and standard proof techniques. Prior coursework in these topics will helpful – we recommend students have taken a graduate-level machine learning course (e.g. 36-705, 10-715).

Textbooks

There is no need buy any textbook for this course. We will provide lecture notes in this course. See the resources page for some relevant textbooks.

Accommodations for Students with Disabilities

If you have a disability and are registered with the Office of Disability Resources, we encourage you to use their online system to notify us of your accommodations and discuss your needs with us as early in the semester as possible. We will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, we encourage you to contact them at access@andrew.cmu.edu.

Diversity Statement

It is our goal that students from all diverse backgrounds and perspectives are well served by this course, that students’ learning needs be addressed both in and out of class, and that the diversity that students bring to this class be viewed as a resource, strength, and benefit. Dimensions of diversity include race, age, national origin, ethnicity, gender identity and expression, intellectual and physical ability, sexual orientation, faith and non-faith perspectives, socio-economic class, political ideology, education, primary language, family status, military experience, cognitive style, and communication style. We are intentional in our aim to present materials and activities that are respectful of diversity, based on these dimensions and any other visible and invisible differences not captured in this list. Indeed, in this class you will learn to approach technology design from an empathetic, human-centered perspective that directly examines and challenges bias and inequality. Your suggestions for ensuring that the class lives up to these values are encouraged and welcomed. In addition, if at any time you experience or witness anything in this class that challenges inclusion, is insensitive or othering, or reinforces biases or stereotypes, please report those experiences (responses can be anonymous).

Communication

  • Canvas: We will be using Canvas for all assignments and grades. Please also post all questions on Canvas as discussions instead of sending emails.
  • Email: If you email your instructors, you might want to include the substring “[AFIL Course]” to begin a meaningful subject line and have tried to resolve the issue appropriately otherwise. For example, you should post questions about course material on Canvas first, and then use emails only after an appropriate amount of time has passed without a response. Please use your CMU email account.

Grading

This course will have no exams. Instead, grading will be a product of the following components:

  • Attendance: We want students to attend lectures in person consistently. Students are permitted 2 unexecused absenses, no questions asked, before being docked.
  • Scribing: One goal we have for this course is to release a high-quality set of notes that summarize the content of the lectures. For each lecture, we will assign a scribe who will take and then type up notes for approval by the instructors.
  • Homeworks: This course will have two open-ended implementation-based homeworks on applying techniques from the class that will be solved in small groups. Solutions will then being presented to the class.
  • Project: Students will engage in a semester-long research project related to the themes of the course before presenting them at the end of the semester. Midway through the semester, students will submit project proposals in groups. The grade for the project will be a function of the proposal, final presentation, and final write-up. We expect these to be of the quality of a paper at a top-tier ML conference.
PercentageActivity
10%Attendance
15%HW 1
15%HW 2
20%Scribing
40%Project

Health & Wellness

Take care of yourself. Do your best to maintain a healthy lifestyle this semester by eating well, exercising, avoiding drugs and alcohol, getting enough sleep and taking some time to relax. This will help you achieve your goals and cope with stress.

All of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support sooner rather than later is almost always helpful.

If you or anyone you know experiences any academic stress, difficult life events, or feelings like anxiety or depression, we strongly encourage you to seek support. Counseling and Psychological Services (CaPS) is here to help: call 412-268-2922 and visit their website at http://www.cmu.edu/counseling/. Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.

If you or someone you know is feeling suicidal or in danger of self-harm, call someone immediately, day or night:

CaPS: 412-268-2922

Re:solve Crisis Network: 888-796-8226