Reading Google's Site Reliability Engineering as an FDE: SLOs, error budgets and postmortems

Betsy Beyer, one of the book’s four editors, is blunt about error budget policies: they must be clear, written down and agreed in advance. In other words, the rule for what happens when things break too often has to exist before the system breaks, not be argued over afterwards.

A Forward Deployed Engineer runs into exactly that question at every customer. You put a system into production, the customer asks whether it is stable, and “fairly stable” is not an answer anyone can measure.

The book is long, but an FDE does not need to read all of it. What follows picks out the chapters most useful for customer-facing work, the order to read them in, and three ideas to carry into the meeting room.

A book Google wrote about how it runs its own systems

The book is copyright Google, was published by O’Reilly Media in 2017 and can be read online at sre.google. It was edited by Betsy Beyer, Chris Jones, Jennifer Petoff and Niall Richard Murphy.

One thing to know up front. A review by a practitioner notes that it is not an easy read, yet still recommends it to most engineers, whether they work in SRE or not.

So do not read it cover to cover. For an FDE, a sensible order is chapter 3 (Embracing Risk), chapter 4 (Service Level Objectives), then chapter 15 (Postmortem Culture: Learning from Failure). After that, turn to the SRE Workbook for its example error budget policy.

Why is 100% the wrong target?

Chapter 3 argues that 100% is almost never the right reliability target. In its place the book offers the error budget. According to the authors, it puts the SRE team and the product development team on the same side: both share an interest and both manage a single budget for failure.

At a customer, those two sides are usually you and their operations team. You want to ship new features every week; they want nobody to touch the system. An error budget turns a gut-feeling argument into a subtraction.

The SRE Workbook spells out the arithmetic: an availability SLO of 99.9% means a budget of 0.1%. If the service receives 1,000,000 requests over four weeks, you are allowed 1,000 failed requests in that period.

While there is budget left, keep releasing. Once it is spent, the example policy requires a halt to all changes and releases, except for P0 incidents and security fixes.

That is also why Beyer stresses agreeing in advance. The first job at a customer is to write this release-freeze rule down and get both sides to sign up to it before there is an incident.

SLIs, SLOs and the trap of overly tight targets

Chapter 4 separates three terms that are often used interchangeably. An SLI is a number that measures a specific aspect of a service, and it must be defined carefully. An SLO is the level you want that number to reach, either a threshold or a range; an SLA is a commitment to users, with consequences for meeting or missing the SLOs it contains.

Without this distinction, customer meetings easily drift into arguments about impressions. “The system is fast” is not an SLI. “The proportion of requests returned under a latency threshold, measured at the load balancer” is an SLI; the percentage you set for it is the SLO.

The chapter’s most practical advice comes down to one sentence: start with a loose target and tighten it over time. For an FDE this is worth remembering, because customers often demand an impressive number in the first week, before you have any baseline data.

Picture the first week of deploying an API at a customer. You do not yet know how the system really behaves, so you set an SLO of 99% for the first four weeks: with 1,000,000 requests, the budget is 10,000 failures, wide enough for the team to learn how to run the system without constantly freezing releases.

After four weeks, suppose the data shows the real success rate sitting consistently around 99.95%. Now you have grounds to tighten the internal SLO to 99.9%, a budget of 1,000 failures, while the figure promised to the customer stays lower.

That gap is deliberate. The book advises keeping internal SLOs tighter than the SLOs published to users, leaving room to deal with persistent problems before they touch the promise made to the customer.

Postmortems: fix the system, not the person

Chapter 15 treats the blameless postmortem as the foundation of SRE culture: writing one is not a punishment but a learning opportunity for the whole company. The chapter also advises defining the criteria for when a postmortem is required before any incident happens.

Picture an incident right after you push an update to a customer’s system: the API returns errors for 40 minutes because of a faulty configuration file.

A blame-driven postmortem would read: “Cause: the deploying engineer pushed a bad configuration to production without checking it.” After reading it, the only action is “be more careful next time”, and next time the same failure happens again.

A blameless version tells the same story but asks why the system allowed it to happen. “Cause: the pipeline allows configuration to be pushed to production without an automated validation step; there is no staged rollout, so the fault reached every user.” The actions that follow are now concrete: add a mandatory validation step, roll out to small groups at a time.

The second version puts nobody on the defensive, including the customer’s team, so people are willing to say what actually happened. Beyer also argues that blame only leads to the truth being hidden. When writing, go through the cause section: any sentence whose subject is a person, rewrite so that the subject is a process or a tool.

Turning three chapters into a line on your CV

When preparing to apply for FDE roles, do not stop at saying you have read the SRE book. Show how you put systems into production and kept them running reliably.

This skill does appear in real job postings. A listing for a Deployed Engineer role at Paladin Drones asks candidates to take part in incident response, submit RCAs within SLA deadlines, maintain SLOs and track remediation actions through to closure.

When reading a job description, wherever you see incidents, RCAs or SLOs, you have an opening to bring up these three chapters.

On your CV, write a specific line such as “set SLOs and an error budget policy for a system deployed at a customer; led blameless postmortems that changed the release process”. In interviews, be ready to walk through a budget calculation like the 99.9% example above, and to recount an incident the blameless way.

Doing that subtraction on a whiteboard carries far more weight than a line saying “read the SRE book”.