learn / field notes
These notes explain the mechanisms behind Torollo incidents. Each one starts with a failure you can observe, then separates the fixes by the trade they make.
field note 01
One expired key, one correlated burst
12 min · by Torollo
A cache can remove repeated database reads while leaving simultaneous misses untouched. When a hot key expires, that distinction becomes a production incident.
Read the field notefield note 02
The acknowledgement gap
13 min · by Torollo
A broker can preserve a job across worker crashes, but it cannot know whether the job's external side effect already happened. The application must make repetition safe.
Read the field notefrom explanation to incident
Redis + PostgreSQL
One expired key turns a healthy cache into correlated database load.
Inspect the cache incidentRabbitMQ + Redis
A recovered message repeats a customer-facing side effect.
Inspect the payment incident