Github-Actions on Luther Monson

Webhooks Arrive Out of Order: How My Scheduler Killed Live Builds

Wed, 07 Oct 2026 21:10:00 -0700

The scheduler in ephemerd is event-driven. GitHub sends a workflow_job webhook when a job is queued, when it starts and when it completes. ephemerd creates and destroys runners to match. For a while that worked well enough that I stopped thinking about it. Then a docker-build job failed with “The self-hosted runner lost communication with the server”. I pulled the node’s journal. ephemerd had destroyed that runner itself, on purpose, while it was running the build.

Bind Mounts Across Mount Namespaces: Four PRs and a Container Escape

Thu, 24 Sep 2026 21:02:00 -0700

GitHub Actions has a container: key that runs a job’s steps inside an image of your choosing. It’s popular because it gives you a reproducible toolchain without building a custom runner image. On a self-hosted runner it works by having the runner drive Docker. On ephemerd, “Docker” is a shim that speaks the Docker API and creates containers in containerd. Getting bind mounts right through that shim took four PRs in a week. A couple of months later the version that worked turned out to be an escape.

shipmates Short Ones: 8 Minutes Extracting Go, and Three Voice-Loop Bugs

Sat, 12 Sep 2026 14:35:00 -0700

A few shipmates bugs don’t need a whole post each. They were too good to leave in commit messages, though.

setup-go spends 8 minutes extracting Go on Windows CI

shipmates CI runs on my own ephemerd runners. On Windows those jobs run inside a Hyper-V utility VM. The windows-x64 leg kept timing out in actions/setup-go. Here’s the log from PR #48:

20:58:46  Setup go version spec 1.26.6
20:58:46  Attempting to download 1.26.6...     <- tool-cache MISS
20:58:48  Acquiring 1.26.6 from ...zip
20:58:53  Extracting Go...
21:06:46  ERROR: The action 'Run actions/setup-go@v5' has timed out after 8 minutes

Seven seconds from the start of the step to Extracting Go.... Then almost eight minutes extracting until the step timeout killed it.

ephemerd: One Daemon, a Fresh Runner Per Job, Every OS

Tue, 18 Aug 2026 21:05:00 -0700

I have CI that needs Linux amd64, Linux arm64, Windows and macOS. GitHub-hosted runners for all four get expensive fast, especially the arm64 and Mac minutes. Self-hosted runners fix the bill and create a different problem. A stock actions/runner on a box you own runs whatever the workflow says. It keeps whatever the last job left behind. And it’s one fork PR away from somebody running code on your LAN.

Escaping WSL: Booting My Own Linux VM Through Hyper-V's HCS API

Sun, 14 Jun 2026 20:12:00 -0700

I’ve been building ephemerd, a daemon that runs self-hosted GitHub Actions runners with a fresh environment per job. On a Windows host, Windows jobs run in Hyper-V isolated containers. Linux jobs still need a Linux kernel from somewhere. The obvious answer on Windows in 2026 is WSL2, and that’s what I used first. It worked great from my terminal and fell over the moment ephemerd became a real Windows service.

WSL2 doesn’t do LocalSystem

A Windows service that wants to create containers and Hyper-V VMs runs as LocalSystem. WSL2 doesn’t support that context at all. You get WSL_E_LOCAL_SYSTEM_NOT_SUPPORTED and that’s the end of the conversation. There’s no flag for it. WSL is a per-user feature and a service account isn’t a user.