Distributed Systems Courses: Comparing Approaches

A topic list says little about a course’s depth. Discussing Raft, implementing it, and testing a service built on etcd are three different learning outcomes. We therefore compare the practical work and how it is checked, rather than counting mentions of an algorithm.

The filter shows our selection of the most relevant examples for each goal, rather than complete topic coverage. We do not combine hours, credits, and assignment counts into a ranking: prerequisites and formats vary too much.

The Fly.io workshop uses Maelstrom for six groups of challenges: echo, unique IDs, broadcast, a counter, a Kafka-style log, and transactions. This is a standalone exercise set, rather than a university semester. Examples use Go; the input/output protocol supports other languages.

Nodes run as local processes exchanging JSON through standard streams. The network is simulated: delays and partitions can be introduced, histories recorded, and their properties checked. Our lab 3 and lab 4 investigate real etcd and NATS instances. These methods complement each other: the first helps uncover protocol bugs, while the second reveals integration behavior.

Fault-tolerant broadcast would make a useful short extension: first achieve delivery after connectivity returns, then measure the message count. A history checker is not an exhaustive correctness proof, and the counter challenge does not, by itself, require students to implement a CRDT.

In our syllabus, 10 lectures and 4 labs share a single task service. An API accepts requests, etcd stores state, and NATS delivers work to a worker. Students complete two key operations: creating and completing a task. The starter provides the API, message delivery, and experiment runner.

• —An uncertain request outcome. In lab 1, the response is lost after a successful write. Students must retry the request while preserving a single logical result.
• —Leader changes and network partitions. Lab 2 demonstrates the behavior of three real etcd nodes. Students investigate an existing consensus implementation and explain its limits.
• —Storage, delivery, and effects. In lab 3, the task and a record for later publication are stored atomically. After redelivery, the worker must not update the local counter a second time; message acknowledgement follows the result commit.

What is checked, and what is outside the scope. The environment contains 12 specified failure scenarios, rather than a general linearizability checker. All processes run on one physical host; there is one broker, and its disk is retained. The guarantee of a single effect applies to a counter in etcd while deduplication records are retained, not to an external payment. Implementing Raft, actual migration between shards, and loss of the broker’s disk are outside the core labs.

This gives the course an applied focus: the central task is to state a guarantee for the whole operation and test where it breaks. MIT and UW go deeper into independent algorithm implementation; our running example emphasizes the boundaries between HTTP, storage, and a queue.

The following are the author’s proposals for a future revision. These assignments have not yet been added to the current labs.

1. Describe the protocol before coding. Following UW’s example: states, messages, timers, an invariant, and the boundary of the atomic operation. Follow the experiment with a short explanation of what caused a failure. This could apply to all four labs without changing the project.
2. Broadcast and merging changes. Add a progression from message ordering to replication in lectures 2, 4, and 8, followed by a small CRDT exercise. Cambridge and Yandex SDA offer theoretical references; Gossip Glomers provides broadcast practice.
3. Check histories and search for counterexamples. Extend the specified failures with randomized event sequences and operation history checking. Separately explore a bounded model of a small protocol, as in DSLabs. Retain the integration experiments with real components.
4. Offer optional advanced practice. Start with consistent hashing or primary/backup replication. Put partition migration and a full Raft implementation in a separate, substantial block with additional workload, drawing on MIT and ITMO-MPP.

Papers connect the curricula

MIT’s reading list includes MapReduce, GFS, Paxos, Raft, Linearizability, and Spanner; UW’s includes Lamport, Paxos, Bigtable, MapReduce, and Dynamo. In our course, these works connect to lectures and standalone Research Insights Made Simple decks. For example, the reading for lecture 2 explores time and causality, lecture 5 leads into consensus, and lecture 7 into transactions.

A useful addition, following MIT’s example, would be one short response before discussion: “Which assumption does the system rely on, and what breaks if it no longer holds?” A deck helps explain a paper; the response and experiment show whether the student understands its limits.