The retry storm
Postgres slows from 40 ms to 900 ms. Requests time out, and every service immediately retries three times. Traffic to the database is now four times the original load.
900 req/s · DB p99 900 ms · 3,600 attempts/s
Your decision
Prod simulator
Fifteen production incidents. Read the signals, make one decision, and see what happens to the system.
No trick questions—just the failure modes worth remembering.
Postgres slows from 40 ms to 900 ms. Requests time out, and every service immediately retries three times. Traffic to the database is now four times the original load.
900 req/s · DB p99 900 ms · 3,600 attempts/s
Your decision