Raft Explained: Leader Election, Log Replication, and Safety

etcd, Consul, CockroachDB, TiKV, NATS JetStream, and Kafka’s KRaft controller share one lineage: they replicate their most important data through Raft. Kubernetes relies on it transitively; every change to cluster state is a Raft commit in etcd. An algorithm reaching that position in roughly a decade is unusual, and the reason is unusual too: Raft was not designed to be faster or more elegant than what came before. It was designed so that a complete implementation could be understood by a single maintainer, and the industry that had to maintain consensus chose it.

The problem Raft solves was defined in distributed consensus, and its place among the alternatives (Paxos, ZAB, and the byzantine family) was staked in consensus algorithms. This deep dive covers the Raft consensus algorithm itself: the model, the two protocols that move all its data, the safety argument underneath, and the machinery the papers leave as homework. It assumes the substrate (partial failure, unreliable networks, and no shared clock) without re-deriving it.

Raft is a consensus algorithm for managing a replicated log, published by Diego Ongaro and John Ousterhout in “In Search of an Understandable Consensus Algorithm” (USENIX ATC 2014). It organizes the cluster around a single strong leader whose authority comes from winning a majority election in a numbered term, and it decomposes consensus into three parts (leader election, log replication, and safety) each specified to the end.

The Raft model: roles, terms, and one log

Three roles, one at a time per node. The leader handles all client interaction and every decision about the log; followers are passive, answering the leader’s requests and election requests; a candidate is a follower campaigning for leadership between the two. Every state change is a change between these three, and the normal case is deliberately boring: one leader, everyone else following.

Terms order the world. A term is a monotonically increasing integer that functions as the algorithm’s logical clock: every election starts a new term, each term elects at most one leader, and every message carries the sender’s current term. A node that sees a higher term adopts it immediately and becomes a follower; a candidate or leader that sees a higher term steps down. This one rule is where Raft’s authority lives; a decision made in term 5 cannot be contradicted by anything stamped term 4.

The log is the shared history. Each entry holds a command and the term in which the leader received it; entries are numbered by index, and committed means stored on a majority of the cluster. Once committed, an entry is never lost, and every node’s state machine applies committed entries in index order. Two RPCs move everything: RequestVote for elections and AppendEntries for the log; heartbeats are simply AppendEntries with no entries.

Raft leader election

Leadership needs no external authority; it is won, term by term, from the followers themselves. A leader maintains its position by sending periodic heartbeats. A follower that receives a heartbeat resets its election timer; one whose timer expires has heard from no leader, and starts an election:

1. The follower becomes a candidate, increments the term, votes for itself, and sends RequestVote to every other node.
2. A node grants its vote in a term only once, and only if the candidate’s log is at least as up-to-date as its own: comparing the term of the last entry first, then the log length.
3. A majority of votes makes the candidate the leader, and it immediately broadcasts heartbeats to claim the position and reset everyone else’s timers.

Two rules do the quiet work. One vote per term per node yields Election Safety: at most one leader per term, because two majorities in the same term would have to share a voter who cannot vote twice. And randomization breaks split votes, if several followers time out simultaneously, each can collect a partial vote count and nobody wins; randomized election timeouts stagger the retries, so a new election usually settles on the first retry. The paper’s worked example uses timeouts randomized in a 150-300 millisecond range, sized against its assumption that a heartbeat round trip takes a few milliseconds at most. Production deployments tune the range to their own networks; the requirement it encodes (election timeout comfortably above the worst-case heartbeat round trip) is the part that transfers.

The up-to-date-log rule ties elections to the data. A candidate trailing the cluster on log contents cannot win, because voters refuse it; any candidate that wins necessarily carries every entry the voters have seen committed. This single rule is what connects Raft’s elections to its safety story below. Election as a general topic (methods beyond Raft, leases, and split-brain risk) has its own treatment in leader election; what follows is Raft’s built-in mechanism only.

Raft log replication

All writes take one path. The leader appends the client’s command to its log as a new entry, then replicates it with AppendEntries, which carries the new entries plus two fields describing where they attach: the index and term of the entry immediately before them. The follower runs a consistency check; it must have an entry at that index with that term, or it rejects. Rejection moves the leader’s idea of the follower’s log back by one entry and it retries; the pair walk backwards until the logs agree, and everything the follower holds after that point is deleted and replaced with the leader’s entries. The numbered flow of a committed write:

1. A client sends a command to the leader. Followers that receive it redirect it, or return the candidate they last voted for, during elections.
2. The leader appends the command to its log and sends AppendEntries to each follower with the entry, plus the index and term of the preceding entry.
3. Followers pass the consistency check, append the entry, and acknowledge.
4. Once a majority has stored the entry, the leader commits it, applies it to its own state machine, and replies to the client.
5. The next heartbeat carries the commit index; followers apply the entry to their state machines too. Every node has now applied the same entry at the same index.

The consistency check gives Raft the Log Matching property: if two logs hold an entry with the same index and term, the logs are identical up through that index. Induction does it; each append succeeds only on top of a verified predecessor, so agreeing on one entry means agreeing on its whole prefix. Conflict repair falls out of the same machinery: a deposed leader may have appended entries that never replicated to a majority, and when the legitimate leader’s AppendEntries reaches those followers, the divergent suffix is truncated. Follower logs converge on the leader’s by deletion, never by leader-side rewriting; the leader’s own log is append-only.

The commit rule carries one subtlety worth keeping: a leader commits entries from its own current term by counting replications, but entries from earlier terms are never committed by counting alone; they commit when a current-term entry above them commits. The paper added this rule to close a subtle hole where a leader could falsely mark an old entry committed on the strength of a majority that no longer guaranteed it; keeping the counting argument inside the current term keeps it airtight.

Raft safety

The paper’s safety argument is five named properties, each with a mechanism:

• Election Safety. At most one leader per term: one vote per term per node, plus majority overlap.
• Leader Append-Only. A leader never overwrites or deletes entries in its own log; it only appends. Followers repair by truncation; leaders never rewrite.
• Log Matching. The same index and term in two logs implies identical prefixes: the per-append consistency check, inductively.
• Leader Completeness. If an entry is committed in a term, it is present in every leader of every later term. The proof is the election rule again: a committed entry sits on a majority, a winning candidate needed a majority’s votes, the two majorities intersect, and the up-to-date rule forces the winner to carry the entry.
• State Machine Safety. If a node has applied an entry at some index, no node ever applies a different entry there. Committed history is one history, everywhere.

Read the list bottom-up and the design appears: quorum intersection plus the term mechanism plus the up-to-date election rule give Leader Completeness; Leader Completeness plus Log Matching give State Machine Safety. The properties cost no extra protocol messages; they fall out of rules the election and replication sections already described, which is the understandability argument in miniature: safety is not a module bolted on, it is the same three rules looked at from below.

Two clarifications prevent over-reading the guarantees. An isolated leader (one partitioned away from the majority) keeps accepting writes that will never commit, because no majority exists to store them; it is deposed the moment it sees a higher term, and its uncommitted entries are truncated by the legitimate leader. And safety is about committed data: a read served from a node’s state is only as current as the last commit that node heard about, which is why reads need their own handling, next.

Terms also fence. A deposed leader’s stale decisions carry its old term, and every node that has seen a higher term rejects them on sight; the majority’s newer stamp wins over the older one’s claims. This is fencing in miniature; the general problem of fencing tokens in distributed locks has its own article, distributed locks.

Reads, performance, and tuning

Reads are where teams most often break a Raft deployment’s guarantees. The state machine’s data is current at the leader (the leader applies every commit first) but “the leader” can be a stale memory: a just-deposed leader that has not heard the news, serving reads from a state the cluster has already moved past. Linearizable reads (every read reflecting all committed writes, the strongest practical guarantee in the CAP theorem and consistency models) take one of three forms:

• Reads through the log. Send each read through as a log entry; it commits like a write and is linearizable by construction. Correct, and it pays a full commit round trip per read.
• ReadIndex. The leader first confirms it still leads (a heartbeat round to the majority) then serves the read from its state at the confirmed commit index. Still linearizable; pays a heartbeat round trip, which is cheaper than a commit.
• Lease reads. The leader reads locally with no round trip, justified by a lease derived from its election timeout. The fastest option, and the only one that leans on clock assumptions; a lease is only as sound as the bound on clock drift.

Follower reads trade guarantees for scale: shift read traffic to followers and accept bounded staleness; each follower is behind by at least its last heard commit index. A useful pattern for heavy read traffic, and no longer linearizable.

Tuning is mostly one inequality: the election timeout must sit comfortably above the worst-case heartbeat round trip, which includes network delay and every pause the leader’s process can take: garbage collection, page faults, a busy host. Too small, and a slow-but-alive leader is repeatedly deposed by its own followers; too large, and failover takes longer than users tolerate. Wide-area clusters inherit the same inequality at inter-region latency, where every heartbeat round trip and every commit pays the distance to the nearest majority member.

Snapshots and cluster membership

A log that grows forever eventually outruns disk and startup time, so Raft deployments compact: the system writes a snapshot of the state machine, records the last log index and term it includes, and discards the log up to that point. A follower too far behind to be caught up by log entries gets the snapshot wholesale (the InstallSnapshot RPC) and resumes from there.

Membership change is the other piece of homework, and Raft specifies it. Changing from one configuration to another in a single step can split the cluster: an in-flight change can leave two disjoint groups, each able to form a majority of the configuration it remembers, and both deciding. Raft’s answer for arbitrary changes is joint consensus; the cluster commits an entry describing both old and new membership, decisions under it need majorities of both, then a second entry retires the old. The common case in production is simpler and safer: change one node at a time, where any majority of the old configuration overlaps any majority of the new, and no split configuration can ever form. New nodes typically join as non-voting members until their logs catch up.

Raft in production

The production lineage is broad enough to be its own recommendation. etcd runs the Kubernetes control plane’s state; Consul coordinates service discovery and health; CockroachDB replicates each of its ranges through a Raft group per range; TiKV does the same per region; NATS JetStream replicates its streams and metadata; Kafka’s KRaft mode replaced its ZooKeeper-based controller with a Raft quorum for metadata. The lesson from a decade of these systems: what a team adopts is never the bare algorithm but the service around it (snapshot management, membership tooling, metrics) and mature libraries such as etcd’s raft library and hashicorp/raft carry those years of hardening, along with the extensions that make them robust against disruptions the 2014 paper left open: pre-vote, which stops an isolated node from churning term numbers, and check-quorum, which makes a leader step down when a majority stops answering.

Operating a Raft cluster reduces to a few durable facts. Quorum sizing is the availability budget: three nodes tolerate one failure, five tolerate two, seven tolerate three, and every failure tolerated is paid for in commit latency, since each commit needs the nearest majority to acknowledge. Leadership flapping is almost always timeouts set against an optimistic picture of the network or the garbage collector. And majority placement decides both latency and survivability: a five-node cluster with three members in one zone has both its quorum and its blast radius in that zone.

Common mistakes

• Running even-sized clusters. Four nodes tolerate one failure, same as three, with extra cost. Valid sizes are 2f+1.
• Calling leader reads linearizable by default. A deposed-leader window exists between leadership loss and discovery. Without ReadIndex or a lease, a leader read is merely probably current.
• Tuning timeouts for the median network. Elections trigger on the tail, not the median. A garbage-collection pause that outruns the election timeout will depose a healthy leader under load.
• Changing membership by more than one node. Two simultaneous additions or removals can produce two disjoint majorities, each electing its own leader. One at a time is the safety rule, not a convention.
• Placing the quorum with the hardware, not the write path. Every commit pays the round trip to the nearest majority member. Quorum placement is a latency decision disguised as a topology one.

FAQ

What is the Raft consensus algorithm?

A consensus algorithm for managing a replicated log, built around one strong leader whose authority is won by a majority election in a numbered term. It decomposes consensus into leader election, log replication, and safety, and specifies all three end to end. etcd, Consul, CockroachDB, TiKV, and Kafka’s KRaft controller all replicate through it.

What happens when a Raft leader fails?

Followers stop receiving heartbeats, an election timer expires, and a candidate wins a majority vote in a new term. Committed data survives by construction, the up-to-date-log rule prevents any candidate missing a committed entry from winning. Writes that were in flight and not yet committed fail and are retried against the new leader.

Why does Raft randomize election timeouts?

To break split votes. If every follower times out at the same instant, each becomes a candidate, each collects a minority, and the term deadlocks; the next round repeats. Randomized timers stagger the candidates, so one usually starts early enough to win outright.

How many nodes should a Raft cluster have?

Three for most systems: it tolerates one failure and commits in one round trip to a second node. Five where a second failure must be survivable: during the rolling restart that follows one node’s failure, for instance. Each additional failure tolerance adds a node to every quorum and to commit latency.

Are Raft reads linearizable?

Only with explicit read handling. Reads through the log and ReadIndex reads are linearizable; lease reads are linearizable under a clock-drift bound; follower reads are stale by design. A default leader read (served from state with no confirmation) has a deposed-leader window and is not a guarantee.

Related articles

• Next read: the Paxos algorithm; the classic Raft was designed against, covered in depth.
• consensus algorithms, the comparison that puts Raft in context: Paxos, ZAB, and PBFT.
• distributed consensus: the problem Raft solves, and the quorum arithmetic underneath it.
• leader election, elections beyond Raft: methods, leases, and split-brain risk.
• distributed locks; leases and fencing tokens, where Raft’s term machinery is applied.