Brown CSCI 2690: Cloud and Datacenter Operating Systems

Fall 2024

Lecture Schedule: Tuesdays & Thursdays, 9:00 AM - 10:20 AM

Location: CIT Center 101

Instructor: Deepti Raghavan (deeptir at brown.edu)

Office Hours: Tuesdays, 10:30 AM - 11:30 AM, CIT Center 501 & Thursdays, 10:30 AM - 12 PM, CIT Center 501 (please let the instructor know ahead of time you will be attending).

Resources: Canvas, EdStem


Google Datacenter

Course Description

Most of the large-scale web applications and enterprise applications we use daily rely on datacenters for compute, storage, networking and security, commonly via cloud computing interfaces. There are many research questions around how to use the datacenter as a computing platform to architect the “best” applications, all while datacenter hardware is rapidly evolving. This is a graduate-level seminar on cloud and datacenter operating systems. We will study topics such as serverless computing, kernel bypass operating systems, tiered and far memory systems, hardware accelerators, and disaggregation. The goals of the class are to (1) learn about the challenges surrounding cloud and datacenter systems, (2) practice reading, analyzing, and presenting systems research papers on these topics, and (3) perform a semester-long research project and present findings to the class.


Course Structure

The course is structured into the following components:


Class Reading, Reading Responses (30%)

For each class, I will assign 1-2 papers for reading and a paper reading response assignment. Readings and responses must be completed before class:


Class Discussion Lead & Class Participation (25%)

Attendance is mandatory, as a large part of the class is an in-person discussion. However, it is fine to miss class ocassionally, especially if you are sick or have other extenuating circumstances. Frequent absences (including not having read the reading and not participating) will be penalized.

Separately, you will sign up to lead the discussion for at least one class over the entire semester. This specifically involves:


Warmup Assignment (5%)

The goal of the warmup assignment is to familiarize students with Cloudlab, so students may use it as an experimentation platform for their projects:

Semester-Long Project (40%)

More details about the project are here. An ideal project could later become a strong submission to a systems conference such as OSDI, SOSP, ATC, Eurosys, or NSDI. Some administrative notes about the project:

Project Dates:

Date Project Milestone Link
Tues 9/10 Project Assignment Posted Project Info
Mon 9/30 Project Proposal Due Submission Info Coming
Mon 9/30 - Oct 4 Mandatory Initial Check-In Signup Link Coming
Mon 10/28 Mid-semester progress report due Submission Info Coming
Oct 28 - Nov 1 Mandatory Mid-Semester Check-In Signup Link Coming
Tues Dec 10 (and Dec 12 depending on enrollment) In-class project presentations Presentation Info Coming
Fri Dec 13 Final Paper Due (in lieu of a final exam) Final Paper Info Coming

Time Breakdown

Reading papers and writing reviews for them takes time. Students should expect to spend 4-5 hours per week preparing for class meetings, in addition to the 3 hours per week of class meetings themselves. This amounts to approximately 80 hours of class work over the semester. An additional 10 will go towards the warmup assignment and extra prep towards being the discussion leader. The remainder of class time, approximately 90 hours, is dedicated to the class research project.


Collaboration Policy

You are allowed (and in fact encouraged) to discuss readings, reading questions, and the warmup assignments with others (including people who might not be in this class). However:

A word about citing: as you can see we require you to cite any external sources you use for the class. External sources here include but are not limited to previously published articles, blog posts, Stackoverflow or similar sites, conversations with other people, etc. This policy is not meant to discourage the use of external sources, instead it just codifies a standard academic practice. You should be generous with your citations.

Finally, by taking this class, you agree to never post any solutions for the warmup assignment publicly.


Course Readings Calendar

The readings and questionnaires corresponding to a particular lecture date are to be completed before that date.

Date Topics Papers Questionnaire (Due at 5 PM the previous day). Slides/Notes
Thurs 9/6 Introduction to the Course No Assigned Reading No Assigned Questionnaire Intro Slides
Tues 9/10 Multicore Barrelfish
Scalability at what Cost
Questionnaire 1
(Due 9/9 at 5 PM)
Multicore Discussion
Thurs 9/12 What abstractions should the OS provide? Exokernel (up to Section 4)
Dune
Questionnaire 2
(Due 9/11 at 5 PM)
Topic 2 Slides
Tues 9/17 High-Performance I/O IX
Arrakis (up to section 3.2, and 3.6)
Questionnaire 3
(Due 9/16 at 5 PM)
Topic 3 Slides
Thurs 9/19 High Performance I/O cont. XDP
XRP (up to section 3)
Questionnaire 4
(Due 9/18 at 5 PM)
Topic 4 Slides
Tues 9/24 RDMA PRISM Questionnaire 5
(Due 9/23 at 5 PM)
Topic 5 Slides
Thurs 9/26 Kernel Bypass & OS Abstractions Demikernel
Questionnaire 6
(Due 9/25 at 5 PM)
Topic 6 Slides
Tues Oct 1 NIC Interfaces Enso Questionnaire 7
(Due 9/30 at 5 PM)
Topic 7 Slides
Thurs Oct 3 CPU Scheduling Shenango
Killer Microseconds
Questionnaire 8
(Due 10/2 at 5 PM)
Topic 8 Slides
Tues Oct 8 CPU Scheduling Cont. Syrup (optional)
ghOSt
Questionnaire 9
(Due 10/7 at 5 PM)
Topic 9 Slides
Thurs Oct 10 CPU Scheduling & Interference Caladan Questionnaire 10
(Due 10/9 at 5 PM)
Topic 10 Slides
Tues Oct 15 Virtualization Xen and the Art of Virtualization
Are VMMs Microkernels Done Right?
Questionnaire 11
(Due 10/14 at 5 PM)
Topic 11 Slides
Thurs Oct 17 Firecracker Firecracker Questionnaire 12
(Due 10/16 at 5 PM)
Topic 12 Slides
Tues Oct 23 Guest Lecture by Josh Fried Junction
Questionnaire 13
(due 10/22 at 5 pm)
Topic 13 Slides
Thurs Oct 25 NO CLASS (Project Hacking Day)
Use the extra time to make progress on your projects.
Tues Oct 29 Memory Management & Large Pages HugePage Aware Allocator Superpages (optional)
Questionnaire 14
(Due 10/28 at 5 PM)
Topic 14 Slides
Thurs Oct 31 Warehouse-Scale Memory Management Software-Defined Far Memory Questionnaire 15
(Due 10/30 at 5 PM)
Topic 15 Slides
Tues Nov 5 NO CLASS! Election
Thurs Nov 7 Far Memory Fastswap (up to S5; skim S5 and eval)
aIFM
Questionnaire 16
(Due 11/6 at 5 PM)
Topic 16 Slides
Tues Nov 12 Cloud Databases Delta Lake
Questionnaire 17
(Due 11/11 at 5 PM)
Topic 17 Slides
Thurs Nov 14 Data Processing Systems Distributed Ownership
Questionnaire 18
(Due 11/13 at 5 PM)
Topic 18 Slides
Tues Nov 19 Serverless Programming Models Apiary
Questionnaire 19
(Due 11/18 at 5 PM)
Topic 19 Slides
Thurs Nov 21 Serverless Programming Models Cont. gg Questionnaire 20
(Due 11/20 at 5 PM)
Topic 20 Slides
Tues Nov 25 Sustainability Carbon Aware Datacenters
Reducing Embedded Carbon
Questionnaire 21
(due 11/24 at 5 pm)
Topic 21 Slides
Thurs Oct 25 NO CLASS (Thanksgiving)
Tues Dec 3 Microservices: Guest Lecture by Akshay Narayan Service Meshes at Meta Questionnaire 22
(Due 12/2 at 5 PM)
Topic 22 Slides
Thurs Dec 5 Sky Computing Sky Computing HotOS paper Sky Computing Whitepaper Questionnaire 23
(Due 12/4 at 5 PM)
Topic 23 Slides
Tues Dec 10 Class Project Presentations
Thurs Dec 12 Class Project Presentations