Two short ones from the Windows node of ephemerd from the last few weeks. Neither is complicated once you know what’s going on. Both took longer than they should have to find, because the failure showed up a long way from the cause.

GitHub Actions has a container: key that runs a job’s steps inside an image of your choosing. It’s popular because it gives you a reproducible toolchain without building a custom runner image. On a self-hosted runner it works by having the runner drive Docker. On ephemerd, “Docker” is a shim that speaks the Docker API and creates containers in containerd. Getting bind mounts right through that shim took four PRs in a week. A couple of months later the version that worked…

The standard way to run a WordPress site is four services:

After a week of Windows firewall rules that listed fine and enforced nothing, I went back to the Linux side of ephemerd feeling pretty good about it. iptables is the boring, well-understood one. Then I looked at the packet counters on a live node. I found a rule that had never matched a packet in its life. That node had pushed 9.8 GB through the same chain.