The outage is over. The site is back. Now comes the meeting — and it can go one of two ways.
In the first, everyone wants to know who ran the command that took prod down. The engineer who did it stares at the table. Next time something breaks, that person — and everyone watching — will be very, very quiet about it.
In the second, the room asks how a single command was even able to take prod down with no safeguard, no warning, no quick way back. The same engineer talks freely, because they're not on trial. The team leaves with three concrete fixes.
That second meeting is a blameless postmortem, and it's one of the highest-leverage habits a team can build.
What a postmortem is
A postmortem (or incident retrospective) is a written review after a significant incident. It captures what happened, why, and — most importantly — what you'll change so it doesn't happen the same way again.
"Blameless" is the crucial qualifier. It means the analysis focuses on systems and processes, not individuals. Not "Sam deployed the bad config" but "our deploy process let an invalid config reach production with no validation."
Why blame backfires
Pointing fingers feels satisfying and accomplishes nothing. Worse, it actively makes you less reliable:
| Blame culture | What it produces |
|---|---|
| People fear being the scapegoat | They hide mistakes and near-misses |
| Postmortems get sanitised | You lose the honest detail you need to fix things |
| The "human error" box gets ticked | The real systemic cause goes unfixed — and recurs |
| Good engineers burn out or leave | Your most experienced people stop taking risks |
The core insight: people almost never want to cause an outage. If a competent person made a mistake, the real question isn't "why were they careless?" — it's "why did the system let a normal human action cause this much damage?" Fix that, and the next competent human is safe too.
The template
Here's a structure you can copy for any incident. Keep it factual, keep it kind, keep it focused on the future.
| Section | What goes in it |
|---|---|
| Summary | 2–3 sentences a non-engineer could understand: what broke, how long, who was affected. |
| Impact | The concrete damage — duration, users/requests affected, revenue or SLA impact. |
| Timeline | Key moments with timestamps: when it started, when it was detected, key actions, when it resolved. |
| Root cause | The underlying systemic cause — not "who", but "what in the system allowed this". |
| What went well | Yes, really — what helped you detect or recover faster? Reinforce it. |
| What went wrong | Where detection, response or tooling fell short. |
| Action items | Specific, owned, dated fixes. This is the part that actually prevents recurrence. |
The action items are the whole point. A postmortem with no owned, dated follow-ups is just a sad story. Each item needs a name and a date, or it won't happen.
A mini example
To make it concrete, here's the shape of a good entry:
- Summary: Checkout was unavailable for 38 minutes after a config change disabled a required service.
- Impact: ~1,200 failed checkouts; ~$9k estimated lost revenue; SLA error budget for the month down by 40%.
- Root cause: The deploy pipeline accepted a config with a missing field and shipped it to all servers at once, with no validation and no staged rollout.
- What went well: Alerting caught it in 3 minutes; rollback worked cleanly.
- Action items: (1) Add config validation to the pipeline — Priya, by Fri. (2) Roll config out to one server first — Marco, next sprint. (3) Add a checkout SLO alert on burn rate — Dana, this week.
Notice there's not a single name attached to blame — only to fixes.
How to run the meeting
A few habits keep it blameless in practice, not just in name:
- Write the doc first, meet second. Circulate the facts so the meeting is about analysis, not discovery.
- Use neutral language. "The config was deployed", not "Sam deployed the config."
- Ask "how", not "who". Every "who" question has a "how did the system allow this" version — ask that instead.
- Welcome the person closest to it. They have the most to teach, and treating them well teaches everyone else it's safe to be honest.
- Track the action items to done. Review them in the next one. Unfinished fixes are how the same incident comes back.
Common mistakes
| Mistake | Do instead |
|---|---|
| Only doing them for huge outages | Do them for near-misses too — cheap lessons |
| Vague action items ("be more careful") | Specific, owned, dated, systemic fixes |
| Naming and shaming | Name systems and processes, not people |
| Writing it and filing it away | Share it widely; the lessons are the value |
| Skipping "what went well" | Reinforce what saved you, so it keeps working |
The bottom line
| Principle | In one line |
|---|---|
| Blameless | Fix the system, not the person. |
| Honesty | Safety produces the truth you need to improve. |
| Action items | Owned and dated, or it didn't happen. |
| Near-misses count | The cheapest lessons you'll ever get. |
A great postmortem turns your worst day into your most valuable one. The outage already happened — the only question left is whether you spend the aftermath finding a culprit or finding the fix. Choose the fix, every time.
Good postmortems start with good incident records. Our incidents guide and root cause analysis show how every WatchFor incident captures the timeline and evidence you'll build your postmortem on.