Cache Failure Modes: Stampede, Avalanche, and Penetration

A cache stampede is the failure mode in which a popular key expires and many concurrent requests, all missing on it at once, each fetch the value from the source simultaneously, so a layer built to shield the database instead concentrates N identical fetches into the exact moment it was supposed to prevent. The herd is the point: it is not one wasted fetch but a synchronized one.

Stampede, dogpile, thundering herd: one vocabulary

The terms overlap, and search engines treat them as separate questions, so the mapping is worth stating once:

• Cache stampede and dogpile effect name the same event: a key expires, and concurrent requests pile onto the source to recompute it.
• Thundering herd is the general pattern (many actors waking at the same trigger) of which the stampede is the cache-specific instance.
• Cache avalanche is the aggregate version: not one key expiring but many at once, or the node holding them all dying at the same time.
• Cache penetration differs in kind: requests for keys the cache can never usefully answer, because the data behind them does not exist.

Stampede: the expired key and the herd

The mechanics are worth tracing, because the counters fall straight out of them. A popular product page sits in cache with a 60-second TTL. At the moment of expiry the next request misses, and so do the next hundred, arriving in the milliseconds the first one spends querying the database. All hundred fetch the identical value, the database runs one expensive query a hundred times over, and the page is slow exactly when it is most popular. Two properties make a stampede worse than it first looks: it is synchronized by the TTL, and its size scales with the key’s popularity. The keys you most want cached are the ones that hurt most on expiry.

Four counters, from cheapest to strongest:

• Request coalescing. Make one request the designated fetcher and let the rest wait for its result: a per-key lock in the cache client, or a single-flight wrapper. The herd still arrives; only one member of it reaches the database.
• Probabilistic early refresh. Instead of letting the TTL expire like a cliff edge, let readers near the end of the TTL recompute with a small probability that rises as expiry approaches; the approach Facebook described for its memcache fleet, refreshing values just before the herd moment arrives, spread across the traffic rather than concentrated at it.
• Stale-while-revalidate. Serve the expired value while a background fetch updates it. Readers get instant data that is slightly old; a freshness-for-load trade that only suits data where slightly old is acceptable.
• Pre-warming. For the small set of keys whose expiry you control; nightly rebuilds, scheduled refreshes; recompute before the TTL ends, so the herd never finds the key missing.

Choosing between them reduces to one question: does the value tolerate staleness? If yes, stale-while-revalidate and probabilistic refresh cost almost nothing. If no (a stock level, a permission bit) request coalescing is the counter, because every alternative serves an expired value to at least some readers.

Avalanche: when the cold arrives all at once

A stampede is one key expiring. Cache avalanche is the aggregate: many keys going cold in the same moment, or the whole fleet doing so at once. It arrives through two doors, worth separating because the counters differ:

• Mass simultaneous expiry. A nightly job loads ten thousand keys with identical TTLs; hours later they all expire together, and the first traffic hits a cache holding nothing. The counter is jitter: add a random margin to every TTL (a 60-minute TTL becomes 60 to 70 minutes) so expiries smear across time instead of synchronizing. The same logic applies to bulk loads, warm a large keyspace gradually rather than in one write burst.
• Node or fleet loss. A cache node crashes and its keys are cold; a failover in a replicated setup can leave a new primary with empty memory. Every request that would have hit becomes a miss, and the database experiences the cache’s entire traffic as though the cache never existed. Warm-up is the counter (admit traffic gradually, pre-fill the hottest keys before opening the gates) plus enough headroom in the source to survive a cold start.

Avalanche is the failure mode that turns a caching layer into an overload question: when a whole fleet goes cold, the choices are degraded responses, queuing, and dropping; the same decision space as load shedding, forced on you by a component that was sold as an optimization.

One subtlety makes avalanche more likely on healthy systems: the same engineers who fight stale data shorten TTLs aggressively. Short TTLs are correct for freshness, and they raise the average rate of expiry. A fleet of keys with short, identical TTLs is a fleet of synchronized avalanche triggers, which is why jittered TTLs appear in nearly every production cache configuration, and why the fix is cheap: randomness in a TTL is a one-line change that removes a synchronized single point of failure.

Penetration: keys with no answer

The third mode does not involve expiry at all. Cache penetration is a stream of requests for keys that can never produce a hit, because the data behind them does not exist. Requests for a nonexistent user, a product ID that was never issued, an email that is not registered: every one misses, every one reaches the database, and every one returns nothing worth caching. The cache never fills, no matter how long the stream lasts. Two sources produce these streams (broken clients iterating over wrong IDs, and scanners probing for accounts) and both are dangerous for the same reason: the cache provides no protection against them at all.

The counters, in escalating order:

• Cache the negative answer. A null result is a result: store “does not exist” with a short TTL, so the second request for the same nonexistent key hits. The one counter that makes the cache do work, and the right one when the same absent keys recur.
• Validate keys before the lookup. IDs with a known shape (UUIDs, numeric ranges, checksummed codes) can be rejected before they touch the cache. The cheapest counter when it applies.
• Guard with a bloom filter. A compact probabilistic set of the keys that plausibly exist, checked before the database: a filter that says “definitely not present” can be trusted, at the cost of occasional false positives and a filter that must be maintained.

Penetration is also the mode with a security flavor, and one boundary is worth naming: admission control is the durable answer to hostile scanning. Negative caching defends the database; rate limiting defends the system.

The dependency question: what does the system do without the cache?

All three modes converge on one question, which is the real lesson of cache failure. A cache is a performance optimization: correctness and availability should, by design, not depend on it. The design review question for any cache layer is blunt; name the day the cache returns nothing. What does the database see, and does the system survive it?

• Fail-open. Fall back to direct reads, absorb the burst, run slow; the latency the cache was built to hide, in the latency vs throughput sense, is now paid on every request. Requires database headroom for the cache’s full traffic: the difference between a degradation and an outage.
• Fail-closed. Refuse the work rather than serve it unacceptably slowly. Protects the source, at the price of the cache becoming load-bearing: its failure is now the user’s failure too.

Most systems should fail open. The cost of that decision is capacity, and it is payable in advance.

Summary table

Common mistakes

• Diagnosing load spikes as database problems. The database is the symptom; the cache is the cause. Check hit ratio and miss patterns before adding database capacity.
• Identical TTLs everywhere. Every identical TTL is a synchronized avalanche trigger, and bulk loads make it worst.
• No single-flight on the hottest keys. The cheapest first counter: coalesce concurrent misses for the same key.
• Treating the cache as required infrastructure. The cache is an optimization. Design its death (fail-open with headroom) before it happens on its own schedule.
• Fixing stampedes with longer TTLs only. A longer TTL delays the stampede and enlarges it, adding accumulated traffic to the same expiry moment.

FAQ

What is a cache stampede?

The failure mode in which a popular cached key expires and many concurrent requests miss on it simultaneously, each fetching the value from the source, so the database absorbs N identical queries in the exact moment the cache was supposed to prevent them. The standard counters are request coalescing (one fetcher, the rest wait) and probabilistic early refresh, which spreads recomputation before expiry instead of concentrating it at expiry.

What is the difference between a stampede, a dogpile, and a thundering herd?

They describe one shape at different levels of abstraction. The dogpile effect is the cache-specific pileup on an expired key, synonymous with the cache stampede. The thundering herd is the general pattern of many actors waking on the same trigger, of which the stampede is the cache instance; the term also covers retry storms and mass reconnects beyond caching entirely.

What is cache avalanche?

Many keys going cold at once. It has two triggers: mass simultaneous expiry (identical TTLs expiring together) and node or fleet loss, where a crash or failover leaves a share of the keyspace with no cache behind it. The counters are cheap and preventive: jitter every TTL, warm up gradually after failures, and give the source headroom to survive a cold fleet.

What is cache penetration?

A stream of requests for keys that cannot produce hits, because the data behind them does not exist: invalid user IDs, unregistered emails, never-issued product codes. Every request misses and reaches the database, and caching never helps. The counters: cache the negative answer with a short TTL, validate key shape before the lookup, and guard the database with a bloom filter of plausible keys.

Should a system fail open or closed when the cache is down?

Fail open if the source can absorb the cache’s full traffic; slower service beats no service, and the cache stays the optimization it was designed to be. Fail closed only when serving from the source is genuinely unacceptable, with full awareness of the price: the cache is now load-bearing, and its failure is the system’s failure.

Related articles

• Next read: redis caching, part 4: implementing these patterns and counters with the standard tool.
• caching in system design, part 1: the read and write patterns, eviction, and hit-ratio math underneath.
• distributed caching, part 2: the fleet architecture whose failure modes this article dissects.
• redis architecture, part 5: the replication and failover machinery behind a cold-start scenario.
• cache invalidation strategies, part 6: the deliberate techniques for keeping copies honest, as distinct from the failure modes covered here.
• latency vs throughput; the budget every fail-open fallback spends.