Cold Starts in Serverless and Containers: Causes, Measurement, and Mitigation

Cold Starts in Serverless and Containers matter because the first request pays work that later requests skip. When scale-to-zero meets a user, that user waits for pull, boot, and init. You then miss the latency goal on a call that looks fine in a warm test. The median hides it if most calls hit a warm copy.

A cold start is the delay from no ready instance to a useful response. It includes image pull, runtime boot, dependency init, and pool warm-up. It is not the same as a slow handler.

If the handler is slow when warm, fix the handler. If only the first call is slow, you have a start problem.

In my experience, teams notice this on the first deploy of the day. Still, a traffic spike after a quiet hour does the same thing. If your minimum instance count is zero, every gap creates a cold path. You should measure that path before you promise a tight tail.

This guide shows what actually burns the time. It also shows how to measure it without fooling yourself. Then it covers trade-offs, warm-up traps, and what a cold path costs at scale.

What a cold start is and why it fails in production

Serverless platforms freeze or delete idle copies to save money. The next call creates a new environment. The Lambda runtime environment docs describe that init phase. Containers do a similar thing when a new pod starts or a node pulls an image.

Production fails when the cold path sits on a user call. First, a quiet endpoint scales to zero. A user arrives and waits seconds.

Health checks on warm copies still pass, because they never see the cold one. After the new copy is warm, the graph looks normal again.

A second failure is a deploy. Every new instance is cold. If you roll the whole fleet at once, every call pays init.

As a result, you get a latency spike that looks like an outage. A slow roll keeps some warm copies in front.

A third failure is a spike on top of a small warm pool. The pool covers the baseline. The extra calls create new copies.

Those calls wait. That wait is tail latency, and it returns on every spike if the pool is too small.

Cold start time also interacts with hard caps. A short timeout can fire before init finishes. Read serverless limits before you set a tiny timeout on a heavy init. The call fails, then the next retry starts cold again.

Where the time goes

Image, runtime, and your code

Split the timeline before you optimize. Image pull and layer extract happen before your process starts. Runtime boot covers the language process and the sandbox.

Your init covers clients, caches, and JIT or import work. When you only time the handler, you miss the first two.

Large images dominate container starts. A base image full of tools pulls slowly on a fresh node. Small images start faster and move less data.

If many pods start together, the registry becomes the bottleneck. Then every new node waits on the same pull.

Language choice changes the shape. A small native binary often boots fast. A JVM or a heavy framework spends time on class load and JIT.

Therefore, put only required work in the init path. Lazy-load a rare client after the first response if the user can wait later, not now.

How to measure it

Measure init as its own span. Platforms expose an init duration for serverless. For containers, time from schedule to ready, then from ready to first successful call. If you blend those, you will tune the wrong stage.

Compare cold and warm on the same build. A warm call is the baseline handler cost. A cold call minus the warm call is the start tax.

When the gap is small, the handler is the problem. When the gap is large, the start path is the problem.

Do not trust a warmer that calls itself on a timer as your only metric. The warmer keeps some copies hot and hides the real cold rate. Also log whether the call was cold. You want the share of user calls that paid the tax, not only the lab number.

More memory often means more CPU on Lambda. The Lambda memory page explains that link. A larger setting can shorten init and raise the bill.

Test a few sizes on a canary. Keep the one that meets the tail without a wasteful floor.

Trade-offs of each mitigation

You can keep a warm floor, speed up init, or accept the cold hit on rare paths. A warm floor costs money while idle. A faster init costs engineering time and may drop a safety check you wanted at boot. Accepting the hit is fine for an internal job and a poor fit for a checkout API.

Provisioned concurrency and minimum instances buy a warm pool. They do not make a deploy free, because new versions still start cold. Still, they remove the quiet-hour cliff.

Use them on the user-facing path. Leave rare batch paths at zero if the caller can retry.

Do not pile every mitigation on every function. A warm floor on a weekly job wastes money. A tiny image will not save a handler that loads a huge model at boot. Match the fix to the stage you measured.

Pitfalls and failure modes

Most cold-start projects fail in the same ways. You can warm the wrong version. You can also cut init so far that the first real call times out. Read this list before you set a floor of zero on a user path.

• Rolling every instance at once so no warm copy remains.
• Setting a timeout shorter than init plus one handler run.
• Trusting a ping warmer and never logging cold user calls.
• Putting a large download or a schema migrate in init.
• Assuming a new region or a new node has a hot image cache.
• Judging success from median latency while the tail is cold.

A common mistake I have seen is a warmer that hits the previous version. The alias moved, and the ping stayed on the old one. Users hit the new version cold.

Point the warmer at the same alias users use. Then check the cold flag on user calls, not only on the ping.

1. Split pull, runtime boot, and your init in the timeline.
2. Log a cold flag on user calls.
3. Compare cold and warm latency on one build.
4. Apply one mitigation on one path.
5. Watch the cold share through a quiet hour and a deploy.

A boot path you can trim

The snippet below shows a small init that fails fast if a required client is missing. It does not download a model and it does not migrate a schema. Then the handler assumes those clients exist, because a half-ready process should not take traffic.

func init() {
    // Fail the instance before it is marked ready.
    db = mustOpen(os.Getenv("DB_HOST"))
    cache = mustOpen(os.Getenv("CACHE_HOST"))
}

func handle(w http.ResponseWriter, r *http.Request) {
    // No client setup here. That work already happened.
    row, err := db.Lookup(r.Context(), r.URL.Path)
    if err != nil {
        http.Error(w, "lookup failed", http.StatusBadGateway)
        return
    }
    writeRow(w, row)
}

Keep init idempotent and bounded. If init retries a down dependency forever, the platform will kill the start and try again. After you move a rare client out of init, time a cold call that actually uses it. Otherwise you moved the delay into the tail and called it a win.

The Cloud Run tips page covers startup and concurrency habits that also apply to similar platforms. Read it next to your own timeline. A tip that helps a tiny service can hurt one that needs a strict init order.

Performance, scale, and cost

A warm floor is a standing bill. Each idle instance costs money every hour it waits. In an illustrative production range, a small floor on a hot API is cheap compared with lost checkouts.

A floor on every rare function is waste. You should set the floor from the cold share, not from fear.

Users feel cold starts as P99 latency. If one call in a hundred is cold, the median stays pretty and the tail does not. When you scale from zero on every gap, that share grows. Therefore, put a floor on paths where the tail is a promise.

Autoscaling can make cold starts worse if the scale step is huge. A big jump pulls many images at once and floods init. Scale in smaller steps when pull time is the cliff. A slower scale-up can beat a stampede that times out.

More CPU shortens init and raises the price per millisecond of busy time. Test two or three sizes. Keep the smallest size that meets the cold-start budget at peak. A maxed memory setting that shaves a tiny slice of boot is often a bad trade.

At large scale, cache images on the nodes you actually use. A fresh node is a cold node. If the autoscaler adds many nodes at once, pre-pull the image or slow the surge. Otherwise the registry, not your code, sets the tail.

Alert on the share of user calls marked cold, and on init duration. Also alert when a deploy makes every call cold for more than a few minutes. That means the roll replaced the warm pool too fast. Split that page from a plain error-rate page.

Key Takeaways

• A cold start is pull plus boot plus your init, and only the first call pays it.
• Measure cold and warm separately. Median latency will hide a scale-to-zero path.
• Keep a warm floor on user paths. Leave rare jobs at zero if retries are safe.
• Do not roll every instance at once if a warm copy must stay in front.
• Trim init, but do not hide required setup in the first user call.
• Page on cold share and init duration, then match the fix to the slow stage.

FAQ

Does provisioned concurrency remove cold starts?

It removes them for the pool you paid to keep warm. A deploy of a new version, or a spike above that pool, still starts cold copies. Size the pool for the baseline you must protect. Accept cold starts above that line or raise the floor.

Should every function stay warm?

No. Idle warm copies cost money all night. Keep a floor where a user waits.

Let internal and rare work scale to zero. If a path becomes user-facing later, add the floor then.

Will a smaller image fix a slow JVM?

It will not, if class load and JIT dominate. Shrink the image when pull time is the large slice. Speed the runtime when boot is the large slice. The timeline tells you which bill to pay.

Can a health check hide a cold start?

Yes, if the check only hits warm copies or marks the instance ready before init finishes. Ready must mean the process can serve. Log cold user calls so a green check cannot hide a slow first request.

Time one cold call and one warm call on your user path today. If the gap is large, split pull, boot, and init, then apply one fix and watch the cold share through a quiet hour. Put a warm floor on that path if the tail is a promise. Leave the rest at zero so you do not pay for idle copies you do not need.

Last updated on 21 September 2026.