Webhooks Arrive Out of Order: How My Scheduler Killed Live Builds

Wed, 07 Oct 2026 21:10:00 -0700

The scheduler in ephemerd is event-driven. GitHub sends a workflow_job webhook when a job is queued, when it starts and when it completes. ephemerd creates and destroys runners to match. For a while that worked well enough that I stopped thinking about it. Then a docker-build job failed with “The self-hosted runner lost communication with the server”. I pulled the node’s journal. ephemerd had destroyed that runner itself, on purpose, while it was running the build.

Gossip-Based Clustering for PHP Apps

Tue, 04 Aug 2026 20:52:00 -0700

Most PHP developers never touch clustering primitives. That’s by design. PHP is share-nothing. Each request starts from zero, and anything shared lives somewhere else. Sessions go in Redis. Cache goes in Redis or Memcached. Locks go in Redis. When you run two app servers, the “cluster” is really two stateless boxes pointed at the same Redis. That works until Redis is the thing you’re trying not to run.

ePHPm has its own KV store in the same process as PHP. On one node that’s straightforward. On three nodes behind a load balancer, a session written on node 1 has to be readable on node 3. That means the nodes have to know about each other and agree on where data lives. This post is how that works as of v0.5.2, including the parts I’m not happy with yet.

Distributed-Systems on Luther Monson

Webhooks Arrive Out of Order: How My Scheduler Killed Live Builds

Gossip-Based Clustering for PHP Apps