-
What SKIP LOCKED Solves, and What It Leaves to You
SKIP LOCKED lets many workers take jobs from one table without blocking each other. It solves the claim very well. Crashes, slow workers, double results and database load are still on you. — read more
-
Behind Cloudflare and nginx, All My Users Had the Same IP
None of my rate limits worked. When I fixed it, every user behind Cloudflare and nginx turned out to share one IP. A plain-words story, a quick check for your own app, and 3 interview questions. — read more
-
Split-Brain Needs a Cluster. Mine Happened on One Server.
Split-brain explained simply: what it is, how partitions and paused processes cause it, the classic cases, and how quorum, terms and fencing tokens defend. Plus my surprise: it can happen on one server with one Postgres. — read more
-
My Uptime Monitor Was Set to Every 5 Minutes. In July It Ran 15 Times a Day.
My GitHub Actions health check was configured for 288 runs a day. In July it ran 15. It had been degrading every month since January and nothing told me, because a scheduled run that never happens looks exactly like a scheduled run that found nothing wrong. — read more
-
Nobody Alerts on Silence: Wiring Sentry Into an LLM Pipeline
My LLM router silently fell back to a CPU container and burned an hour at 390% CPU with zero errors. Here is how I made silence audible — a route-reason span attribute, a throttle that turns 106 events into 1, and why I skipped Sentry’s OpenTelemetry bridge. Then, on day one in production, it caught… — read more
-
My Backup Leaked 156 GB, Filled the Disk, and Broke Itself. The Fix Was One Letter.
The nightly backup verify started flapping with “postgres did not become ready”. df -h said 61% used. The real disk — the one Docker lives on — was at 100%: 57 orphaned pgdata volumes, one leaked per run, for 34 days. The fix: docker rm -f → docker rm -fv. — read more
-
Stuck at “Preparing… 0/0” Forever. My State Machine Had No Exit.
Some books stayed at “Preparing… 0/0” forever. The status Indexing had no way out: not Failed, so no retry; not Ready, so no answers. The fix: 202 early, an atomic claim, and a sweep that gives every state an exit. — read more
-
Nothing Crashed. Nothing Errored. My CPU Sat at 390% for an Hour.
A user PDF stuck at “Preparing… 0/0”, a CPU-only container at 390%, and zero traffic to OpenAI. The bug: a silent fallback after a reliability fix. The fix: one config block. — read more
-
Your RAG Eval Isn’t Flaky. Your Retrieval Is Non-Deterministic.
Same query.Same documents.Same model.And the RAG eval can still hand back a different Recall@8. Not because the model is flaky. Because of an ORDER BY clause. I didn’t find this by watching a metric wobble. I found it reading the retrieval code, and realized the score would drift run to run even if the model… — read more
-
AI Wrote a Thread-Safe Counter. The CPU Made It 5x Slower.
Four threads, no locks, no shared state, no bugs — and still 5x slower than it should be. The code was correct. The memory layout was not. A 30-line C# demo of false sharing, and why AI-generated code makes this knowledge more valuable, not less. — read more