Garbage Collection Tuning for Backend Engineers: Latency, Throughput, and Pauses

Garbage Collection Tuning matters because a collector pause is user-visible time. When the pause lands on a hot request, that call sits in your tail. You then miss the latency goal even if the median looks fine. CPU spent in the collector is capacity you cannot sell.

Tuning means you pick a collector and a small set of knobs. The goal is a pause budget and a CPU budget you can live with. It is not a hunt for magic flags. If the live set does not fit, no flag will save you.

In my experience, teams copy a flag set from a blog and ship it. Still, that heap size was for a different live set. If your process holds more, the collector thrashes. You should measure alloc rate and live size before you change a flag.

This guide shows when tuning helps and when it hides a leak. It also shows how Go, Java, and dotnet differ. Then it covers trade-offs, traps, and what the pause costs at scale.

What tuning is and why it fails in production

A collector frees objects that no root can reach. Young objects die fast in most services. Old objects are sessions, caches, and static state. Tuning decides how hard the runtime works to keep pauses short.

Production fails in three ways. First, a stop-the-world pause hits a request and blows the tail. Health checks still pass, because they do not sit in that pause.

After the pause, the process looks healthy again. The user already saw the slow call.

A second failure is throughput collapse. If the heap is too small for the live set, the collector runs all the time. Request CPU drops.

Queues grow. As a result, you add pods and still miss the goal.

A third failure is a kill during collection. The container limit sits on top of the heap. The collector needs extra RAM to copy or mark.

If that spike crosses the limit, the kernel kills the pod. You then blame the collector for an out-of-memory death.

Do not tune a leak. If the live set grows for hours with flat traffic, the heap is retained on purpose by your code. Read about memory leaks first. A bigger heap only delays the kill.

How the collectors are built

What you can actually change

You can change three things that matter. Heap size sets how much live data and trash you can hold. The collector choice sets the pause shape. The pace knobs set how often the runtime spends CPU to keep the heap in bounds.

Allocation rate is the input you often ignore. A service that allocates a lot of short-lived trash makes the young collector run often. When that rate is the bug, flags are a weak fix. Cut the allocs in the hot path first.

Live size is the other input. It is what remains after a full collection. If live size is most of the heap, you have no room for trash. Therefore, raise the heap or shrink the cache before you chase pause flags.

Go, Java, and dotnet

Go uses a concurrent mark sweep collector. The Go GC guide explains the CPU goal and the memory limit. GOGC sets how much new heap you allow before the next cycle. GOMEMLIMIT sets a soft cap so the runtime stays under the container limit.

Java gives you several collectors. G1 is the usual default for large heaps. ZGC and Shenandoah aim for very short pauses and spend more CPU.

Parallel GC favors throughput and allows longer pauses. The Java GC tuning guide is the source for those choices.

Dotnet uses a generational collector too. Server GC spreads work across cores and fits a service. Workstation GC fits a small tool.

The dotnet GC fundamentals page shows how generations and segments work. Match the mode to a server, not to a desktop app.

Across all three, the heap must sit below the container limit. Leave room for stacks, native buffers, and the collector itself. If you set the max heap equal to the limit, a spike will kill you. Headroom is part of the tune, not an extra.

Trade-offs of each choice

You trade pause time, CPU, and RAM. A short pause often costs more CPU or a larger heap. A small heap saves money and costs more frequent cycles. A throughput collector is fine for batch work and a poor fit for a user-facing API.

Low latency collectors are not free. They keep pauses small by doing more work beside your threads. If your box is already near the CPU cap, the pause win can become a queue. Still, they are the right pick when the tail goal is tight and you have spare cores.

Do not switch collectors because a graph looked bad for five minutes. Confirm the live set, the alloc rate, and the pause log first. Then change one knob. If the tail does not move, revert and look at the code.

Pitfalls and failure modes

Most bad tunes share the same traps. You can spend a week on flags while the hot path allocates a giant buffer per call. You can also shrink pauses and raise P99 latency because CPU steal grows. Read this list before you touch production flags.

• Setting the max heap equal to the container memory limit.
• Turning GC logs off, so you cannot see pause time.
• Tuning a leak as if it were a collector bug.
• Copying flags from a service with a different live set.
• Holding a lock while the collector waits on your thread.
• Judging a tune from median latency alone.

A common mistake I have seen is to set a pause goal the heap cannot meet. The collector tries, spends more CPU, and still misses. If the live set is huge, the goal is a wish. Grow the heap or cut retained data first.

1. Log pauses and alloc rate for one normal day.
2. Record live size after a full collection.
3. Set the heap with headroom under the container limit.
4. Change one knob on one canary.
5. Watch tail latency and CPU for a full traffic cycle before you roll out.

A config you can start from

The snippet below is a starting point, not a universal tune. It keeps a Go process under a soft cap and gives Java G1 a pause aim. Then you compare the canary tail with the old build, because the flag is only a guess.

# Go: soft cap under a 2 GB container limit.
GOMEMLIMIT=1536MiB
GOGC=100

# Java: fixed heap and a pause aim for G1.
java -Xms2g -Xmx2g -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -Xlog:gc*

Read the pause log, not only the process RSS. If pauses meet the aim and CPU is flat, stop. If the collector runs nonstop, the heap is too small or the alloc rate is too high. After that, fix the hot alloc or raise the heap, then retest.

Keep GC logs on one canary even after the tune. A later cache change can double the live set. When that happens, the old flags lie. You want the log so the next incident starts with pause time, not a guess.

Performance, scale, and cost

RAM and CPU trade money. A larger heap costs more per pod and can cut GC CPU. In an illustrative production range, GC CPU of a few percent is calm.

A collector that eats a large share of a core is a capacity bug. You should page on pause time and on GC CPU, not only on RSS.

The user sees this as tail latency. A pause of tens of milliseconds is noise for some APIs and a miss for others. When many threads stop together, in-flight calls pile up. Therefore, a pause budget belongs next to your latency goal.

More pods do not fix a bad heap. If every pod spends the same share of CPU in the collector, scale-out copies the waste. Fix the alloc rate or the heap first. Then scale to the real request load.

GC threads and your threads share cores. If you pin a lock and a collector safepoint waits, the pause grows. That is a thread safety issue as much as a GC issue.

Do not hold a lock across a slow call. Keep critical sections short so a safepoint can land.

At large scale, sample GC logs from a slice of pods. One canary per zone is enough to see a bad flag. If you log every pause on every pod at full detail, the log pipeline becomes the cost. Keep a short pause histogram on every pod, and a full log on the canary.

Set an alert when pause time or GC CPU stays high for several minutes. Also alert when the live set after a full collection climbs for hours. The second alert is a leak, not a tune. Split those pages so the right person wakes up.

Key Takeaways

• Tune from live size, alloc rate, and pause logs. Do not copy flags from another service.
• A leak is not a collector bug. If a full collection does not drop the heap, fix the retain path.
• Keep the heap below the container limit so a GC spike does not kill the pod.
• Short pauses can cost CPU. Check the tail and the core use before you roll a new collector.
• Change one knob on one canary, then watch a full traffic cycle.
• Page on pause time, GC CPU, and a rising live set as separate signals.

FAQ

Should every Java service move to ZGC?

No. ZGC helps when pause time is the tail and you have spare CPU. If you are already CPU bound, the extra work can grow queues.

Start with G1, a real pause log, and a heap that fits the live set. Switch only when the log says pauses are the miss.

Is it safe to disable Go GC to win CPU?

It is rarely safe in a long-lived service. The heap will grow until the process hits the limit and dies. A memory limit plus a normal GOGC is the usual pair. Disable collection only for a short job that exits before RAM matters.

Does a bigger heap always cut latency?

It cuts collection frequency until the heap no longer fits the machine. A huge heap can make a full collection slower on some collectors. Raise the heap when the live set needs room. Stop when pause time and CPU are inside the goal.

How do you know a tune worked?

Compare pause time, GC CPU, and tail latency on a canary against the old build. One quiet hour is not enough. Watch a peak. If the tail did not move, revert the flag and look at allocs or locks instead.

Pull pause time, alloc rate, and live size from one pod today. If a full collection drops the heap, set a heap with headroom and change one knob on a canary. If it does not drop, hunt the leak before you touch flags. Then alert on pause time so the next regression is obvious.

Last updated on 06 September 2026.