CS2680 Modern AI Systems: Agents and System Optimizations
Fall 2026

Announcement

  • Aug 20, 2026Please request your license for Claude Code ASAP. It will be used for your first assignment.
  • Aug 20, 2026Please register an AWS account and add your name, email and account ID to the course sheet. See AWS setup.
  • Aug 17, 2026Welcome to CS2680 Modern AI Systems: Agents and System Optimizations! Class meets Monday and Wednesday; our first meeting is Wed Sep 2, 2026.

Overview

This course follows one agentic system from the agent loop down to the GPU. You first experience the magic of a modern coding agent (Claude Code), then design your own agent served using the same model, measure it, and optimize it. Then you serve an open-weight model yourself and take responsibility for the layers underneath: GPU kernels, batching and scheduling, KV-cache and prefix reuse, routing, quantization, and speculative decoding.

Class time and location
Monday and Wednesday 11:15am – 12:30pm SEC 2.118
Prerequisites
At least one of CS61, CS1610, CS2620. Comfort with Python and PyTorch is assumed; prior exposure to machine learning (e.g. CS1810) is recommended but not required.
Assessment
Five assignments and a final project. A paper presentation and class participation. There is no exam.
Textbook. No textbook is required. Background reading: Computer Systems: Operating Systems: Three Easy Pieces
Large Language Models: Ultra-Scale Playbook, Foundations of Large Language Models
Deep Learning: The Little Book of Deep Learning
Format. The course runs in two parts.
  • Part I — Introduction to LLMs and Agents. You will learn about how to use coding agents (Claude Code) and how to design an agent. You will also explore various agent architectures, use cases and optimizations.
  • Part II — Systems for LLMs and Agents. We will talk about GPU programming, LLM serving (batching, scheduling, routing, caching, quantization, and speculative decoding), and agentic systems.
Throughout the assignments, you will first use an agent (Assignment 1), then design your own agent (Assignment 2), and optimize your agent's performance (Assignment 3), and then choose a model to serve your agent using provided GPUs (Assignment 4), and finally optimize the full stack of the agentic system (Assignment 5).
What this course expects. This course will not teach you every piece of knowledge you need for the assignments. It focuses instead on the skills to learn with, and points you in directions worth going. Expect to spend a significant amount of time learning and experimenting on your own after class — if you want an A, plan on it.
Grading.
  • Assignment 1 agent autopsy10%
  • Assignment 2 build an agent10%
  • Assignment 3 make the agent smarter, faster, cheaper20%
  • Assignment 4 serve your own agent10%
  • Assignment 5 optimize the full stack20%
  • Final Project16%
  • Paper Presentation10%
  • Class Participation4%
  • Bonus course feedback up to 6%; problems current AI cannot solve, 2% eachup to 10%

See policy page for details.

Course website. https://cs2680.com

Staff

Juncheng Yang
Juncheng Yang
Instructor
Yiyu Liu
Yiyu Liu
TA
Yunjia Zheng
Yunjia Zheng
TA
Office hours:
  • Instructor: Appointment only
  • TAs: TBD

Schedule

Tentative and subject to change. Rows with Students as the speaker are student-led: either a paper discussion (see the paper discussion page) or an Assignment sharing session — the Topic cell says which. Readings marked required should be read before class; everything else is optional depth.

Week Date Speaker Topic Course Materials Notes
Part I — Introduction to LLMs and Agents
1 2026-09-02 Juncheng Yang Course overview and logistics Assignment 1 out (use an agent); presentation sign-up opens
2 2026-09-07 No class — Labor Day
2 2026-09-09 Juncheng Yang Modern ML basics: transformers
3 2026-09-14 Juncheng Yang Agents from a user’s perspective
3 2026-09-16 Juncheng Yang Agents from a designer’s perspective I Assignment 1 due Sun Sep 20, 11:59pm
4 2026-09-21 Students Student sharing I What you learned from using an agent (Assignment 1): surprises, failures, lessons, insights, new tools/skills/plugins Assignment 2 out (design an agent)
4 2026-09-23 Juncheng Yang Agents from a designer’s perspective II
Part II — Systems for Agents
5 2026-09-28 Juncheng Yang GPU programming for MLSys: architecture, memory, and kernels
5 2026-09-30 Juncheng Yang Efficient LLM computing: GPU kernels Assignment 2 due Sun Oct 4, 11:59pm (agent loop and problem set)
6 2026-10-05 Juncheng Yang Efficient LLM serving: paging, batching, and scheduling I Assignment 3 out (optimize the agent)
6 2026-10-07 Students Efficient LLM serving: batching and scheduling II
7 2026-10-12 No class — Indigenous Peoples’ Day
7 2026-10-14 Students Efficient LLM serving: routing and load balancing
8 2026-10-19 Juncheng Yang Efficient LLM serving: KV-cache optimization I
8 2026-10-21 Students Efficient LLM serving: KV-cache optimization II Assignment 3 due Sun Oct 25, 11:59pm
9 2026-10-26 Guest Speaker TBD Assignment 4 out (serve your own agent)
9 2026-10-28 Students Student sharing II What you optimized and what it bought (Assignment 3) Project proposal due
10 2026-11-02 Guest Speaker TBD
10 2026-11-04 Juncheng Yang Efficient LLM serving: prefix cache
11 2026-11-09 Juncheng Yang Efficient LLM serving: pruning and quantization Assignment 4 due Tues Nov 10, 11:59pm
11 2026-11-11 Students Efficient LLM serving: speculative decoding Assignment 5 out (optimize the full stack)
12 2026-11-16 Guest Speaker TBD
12 2026-11-18 Juncheng Yang Efficient agent serving systems I
13 2026-11-23 Students Efficient agent serving systems II
13 2026-11-25 No class — Thanksgiving recess
14 2026-11-30 Students Student sharing III What you optimized and what it cost (Assignment 5)
14 2026-12-02 Students Student sharing IV What you optimized and what it cost (Assignment 5) Assignment 5 due Dec 02, 11:59pm
14 (TBD) Students Final project poster and demo Time to show case your fun projects to the world! Final project due Dec 08, 11:59pm; peer reviews due Dec 07, 11:59pm

The semester closes with a final project worth 16%: build a tool that does not already exist — for your research, your studies, your week, or as an extension of one of the assignments — solving a problem no existing system solves. Try the three closest things that exist, name what each cannot do, then measure your own tool against them and say plainly where it does not work. There is no release date, so you can start as soon as you have a problem worth solving. What is fixed is the submissions: a one-page proposal on Oct 28, a poster and demo at the December session, and the report and repository on Dec 8. Half the project's credit is scored by four classmates at that session, and you review four projects yourself. It replaces the cache competition earlier versions of this schedule ran in the same weeks.

Compute Resources

Setup instructions for every environment the course uses are on the computing setup page. Compute for the assignments and the final project comes from:

  • Claude Code — the agent for all five assignments, on an account Harvard FAS provides
  • The course HPC — interactive Slurm jobs, and the gpu-cs2680 GPU partition
  • AWS — a machine that is not your laptop, if you want one

Sponsors

We are grateful for the support of our sponsors.