Why your Postgres index is not being used
Four reasons the planner walks straight past the index you added, and how to check which one it is.
Notes from a backend engineer
Write-ups of the bugs that took far too long to find
Databases, retries, deploys and the occasional flame graph. Each post is something I learned at work and wish I had read first.
Writing
Recent posts
Long enough to be useful, short enough to read over coffee.
Retries with jitter: a ten-line fix for a self-inflicted outage
Every client retried after exactly one second. So the service fell over again, exactly one second later.
Reading a flame graph for the first time
Width is time. Height is the call stack. Colour usually means nothing.
Feature flags are debt. Pay it back.
We had 140 flags. Nobody knew what a third of them did.
What I check before merging a migration
A short checklist that has saved me from at least two outages.
Topics
What I write about
Mostly the unglamorous parts of keeping a service up.
Databases
Indexes, migrations and the queries that are fine until they are not.
Reliability
Retries, timeouts and failing gracefully when a dependency does not.
Performance
Profiling before guessing, and reading what the profiler says.
Shipping
Small changes, safe deploys and cleaning up after yourself.
Debugging
How a bug was found, step by step, including the wrong turns.
Working in teams
Reviews, on-call and writing things down.
A new post every second Tuesday, give or take a production incident.