SLOs for Backend Engineers: Defining, Measuring, and Enforcing Service Objectives

SLOs for Backend Engineers turn a vague hope of reliability into a number you can defend. The objective says how good the service must be over a window. When you miss it, you spend trust.

When you beat it by a huge margin, you may be moving too slowly. The point is a clear line, not a slogan.

What It Is and Why It Fails

A service level objective is a target on a service level indicator over a time window. Availability, latency, and freshness are typical inputs. The Google SRE chapter on service level objectives separates this target from a contract with a customer.

The contract is an SLA. The objective is how you run the system. If you mix them, legal text starts to drive paging, and that is a poor design.

SLOs fail when the indicator does not match user pain. You can hit a server-side success target while the client times out. You can also miss an internal target that users never feel.

SLIs explained with a user journey in mind should come first. An objective on a bad indicator is a precise way to manage the wrong thing.

They also fail when the target is a fantasy. One hundred percent leaves no room for deploys, dependency blips, or planned maintenance. A common mistake I have seen is copying three nines from a slide because it sounded serious.

The team then missed it every month and ignored the number. A target you can hold, and that users accept, is the only useful one.

The window matters as much as the target. A one day window is jumpy. A quarter is so slow that a bad week hides inside a good month.

A rolling 28 day window is a common choice because it covers weekday and weekend shape. Pick one window and stop changing it every incident.

From User Journey to Objective

Start with one journey, not with every route. Checkout, login, or the read path that pays the bills is enough. Write the user sentence.

The user can complete checkout in a few seconds, and the result sticks. Then map that sentence to events you can count. Good events over valid events is the usual form.

Set the target from recent reality plus a product choice. If last quarter was worse than users will tolerate, the objective is a goal and you need a project. If last quarter was better, do not lock in the best week.

Leave room to ship. In my experience, an objective with no remaining budget teaches the team to fear every change.

Use more than one objective only when the harms differ. A latency objective and a correctness objective can both be real. Five objectives on one service split attention.

Start with one or two. Add another when an incident shows a gap the first number cannot see.

What the Number Does Not Cover

An SLO is not a capacity plan and not a test suite. It will not notice a slow leak until the window feels it. It will not grade a single host.

Use saturation charts for the leak. Use the objective for user harm over time. If you try to make one number do every job, it becomes noise.

It also does not replace judgment during an incident. You can be inside the objective and still have a severe partial outage for one tenant. Keep a break-glass path.

The objective guides priorities across weeks. It should not block a human who sees data loss right now.

Architecture and Measurement

Measure as close to the user as you can afford. A client or edge count sees timeouts the server never logs. A server count is easier and misses those calls.

If you only have server logs, state that limit next to the objective. Do not pretend the number includes failures you cannot see.

Define valid events with care. A user cancel is not a service failure. A 400 on a bad input is often not one either. A 500, a timeout, and a wrong total are failures.

Write the exclude list down. If you exclude too much, the objective goes green while users are stuck. Review excludes after each incident.

Store the ratio, not only the average latency. For latency, a good event is a request under the threshold. Pick a threshold users feel, such as a few hundred milliseconds for an interactive call.

Then the objective is the share of requests inside that line. Percentiles on a chart are still useful. The objective itself should be a ratio so a budget is easy to compute.

Implementing SLOs walks through windows, burn, and how teams put the number into the release process. Your pipeline can be simpler. A daily job can compute the rolling ratio.

A live rule can estimate burn. Both should use the same good and valid definitions. If they drift, people will argue the chart instead of the incident.

Enforcement Without a Freeze Culture

1. Publish the objective, the window, and the indicator definition in one page.
2. Show remaining budget next to the service in the deploy tool.
3. Page on fast burn and ticket on slow burn, as in alerting that works.
4. When the budget is gone, require a short review before risky launches.
5. Allow an explicit break-glass with a name and a follow-up.
6. Revisit the target each quarter, not during the outage.

Trade-offs You Should Name

A tighter objective protects users and slows change. A looser one speeds shipping and will anger users if you guessed high. A client-side measure is honest and harder to build.

A server-side measure is cheap and blind to some failures. Write down which side you picked so the next incident does not relitigate it.

Multi-service journeys need a rule for shared blame. If checkout fails because payments fail, both teams can claim the other owns the burn. Give the journey an objective, and give each dependency its own.

The journey number is what the user feels. The dependency number tells you where to work. Without both, enforcement turns into a meeting.

Pitfalls and Failure Modes

The classic failure is an objective nobody can compute the same way twice. One dashboard drops 404s. Another includes health checks. The budget jumps when you fix the query, not when reliability changes.

Freeze the definition. Version it. When you change it, mark a break on the chart so history stays honest.

• Setting the target to the best week you ever had.
• Counting health checks and synthetic traffic as user demand.
• Changing the window mid-incident to make the chart look better.
• Enforcing a freeze with no break-glass, so people route around the process.
• Having five objectives and paging on all of them at once.
• Ignoring a dependency SLO and then acting surprised when the journey burns.

Enforcement fails when it is theater. A banner that says the budget is gone, while deploys continue unchanged, teaches people to skip the banner. A hard freeze with no exception teaches people to ship from a side door.

The middle path is a review with a named approver and a written reason. Keep that review short so it does not become punishment.

Error budgets are the part you spend. If the policy is unclear, the objective is only a chart. Say who may spend budget on a launch, who must stop, and what happens after a full burn.

Also say how planned maintenance counts. If maintenance is free, you will hide risk inside it.

We once hit a bottleneck when the SLO included a low-traffic admin route. Two failures moved the ratio more than a real checkout dip. The team chased the admin bug during a sale.

Split high-value routes from rare ones. Or require a minimum volume before the rare route can dominate the burn.

A Definition You Can Start From

The record below is an illustrative objective for checkout. It uses a 28 day window and treats server 500s and timeouts as bad. It excludes client errors.

Change the threshold to your own latency line. Do not copy the target if your history cannot support it.

service: checkout
window_days: 28
indicator:
  good: status is 2xx and latency_ms is under 400
  valid: all user requests except status 400 and 404
objective:
  good_per_1000: 999
alerts:
  fast_burn: page when 2 hours would exhaust the month
  slow_burn: ticket when 3 days would exhaust the month
policy:
  at_zero_budget: review before risky launches
  break_glass: named approver plus a follow-up note

Put this next to the code that emits the events. If the handler changes status codes, update the definition in the same change. Otherwise the objective drifts.

Test the query on a known bad hour and confirm the ratio moves the way you expect. A definition you have not replayed is a guess.

Performance, Scale, and Cost

Computing an SLO should be cheap. Count good and valid in the metrics pipeline. Do not scan raw logs for every graph load.

High cardinality, such as a label per user, will price you out and slow the page. Keep labels to service, route class, and status class.

The expensive part is the reliability you promise. A tighter objective can mean more capacity, slower deploys, and more on-call. Price that before you announce the number to customers.

An SLA should be looser than the SLO, so you have room to react before a contract breach. Do not publish the internal target as the contract.

Scale the process, not only the math. Ten services can share one template. A hundred services with hand-built queries will lie in different ways.

Use one library for good and valid. Review outliers, not every service, each week. Spend human time where the burn is real.

Chaos engineering can test whether a dependency failure moves the SLO the way you think. If a killed dependency does not burn the budget, your indicator is blind. If it burns the whole month in one drill, your target or your fallback is wrong.

Use small drills. Do not discover this in a real region loss.

Finally, watch the cost of false precision. Extra nines sound good and often demand a new architecture. If users cannot tell, keep the simpler system. The objective should match the product, not a leaderboard.

Key Takeaways

• Tie the objective to one user journey and a real indicator.
• Leave room under perfection so deploys and small faults can exist.
• Use one rolling window and do not change it during an incident.
• Measure with a versioned definition of good and valid events.
• Enforce with review and break-glass, not with a secret side door.
• Keep the SLO stricter than any customer contract.

FAQ

How many SLOs should one service have?

Start with one, or with one for availability and one for latency. Add another only when an incident shows a harm the first number misses. Too many objectives create too many pages. A single honest number beats a folder of unread ones.

Should the SLO include planned maintenance?

Yes, if users feel the downtime. You may exclude a window only when users were warned and a fallback worked. If you exclude every deploy, the chart will stay green while the product is down every afternoon. Count user-visible minutes.

What if we miss the SLO every month?

Then the target is not an objective. It is a wish. Either raise the reliability with a funded project or loosen the target to a level you can hold.

Pretending otherwise trains the team to ignore the budget. Change the number in a review, not in the middle of a page.

Do internal services need SLOs?

Yes, when other teams depend on them. Their objective should be tighter than the journey that calls them, or the caller cannot meet its promise. A batch tool with one user can use a looser target. The test is who hurts when you miss.

Pick one user journey. Write the indicator, the target, and the window on a single page. Replay last month and see whether the bad hours you remember actually move the number.

If they do, add a fast burn page and a written rule for what happens at zero budget. If they do not, fix the indicator before you enforce anything.

Last updated on 16 September 2026.