Setting up a self-hosted GitHub Actions runner
Background
I’ve led a few teams over the years in my career where I’ve been in charge of our CI/CD systems. Most recently, we’ve been using both CircleCI, and GitHub Actions. Both of these platforms support the concept of a “self-hosted runner”. A self-hosted runner is a virtual or physical machine managed by you (and not the CI platform) where your CI workflow is actually run. You still utilize the SaaS platform for overseeing the lifecycle of your workflow (status, retries, etc.), but otherwise you have complete ownership of the platform.
Why Would You Want to Do This?
Cost Savings
If you have spare machines that are sitting around but are otherwise sitting idle much of the day, you may be able to take advantage of their downtime and run your CI workflows on them.
And if you’re using a cloud provider like AWS for your infrastructure, you may be able to utilize EC2 spot instances and their cheaper rates over regular on-demand instances to achieve even greater savings.
Be sure to do the math to compare what you might be paying GitHub / CircleCI hosted runners on a per minute rate, vs. what you might pay to self-host. Don’t forget to factor in any free minutes that the platforms might provide to you.
Security
If you don’t feel comfortable with the idea of your workflows executing on servers you don’t directly control, self-hosted runners can be a good option. Assuming you have expert-level DevSecOps skills, you may well be able to create a private environment that would be more secure than one hosted in a public cloud.
As a word of caution here: ensure that you don’t associate a self-hosted runner with a public GitHub repository.
According to GitHub:
> any forks of your repository would use your self-hosted runners… this could potentially allow a bad actor to execute potentially malicious code on your machine.
That’s probably something you’d want to avoid.
You Want to Define Your Own Resource Classes
These platforms typically have the concept of a “resource class”, whereby you get to choose how much CPU, RAM, etc. get devoted to jobs within your workflow. Typically the more resources, the faster your workflows will run, albeit at a higher cost per minute. Depending on what you need, the platform might not provide it to you, or it may not be cost effective. Self-hosting can be a way to work around these constraints, and give you more options.
Setting Up the Runner
Just as a quick experiment, I decided to use my 6-Core Ryzen 5 5600H SER5 Mini PC running Windows 11 Pro that I bought from BeeLink via Amazon over the holidays. It’s a snappy little machine that takes up little space on my desk, and has been able to handle pretty much whatever I’ve thrown at it so far.
Create an Ubuntu Linux VM Using Multipass / Hyper-V
So as to not mess with this machine and its host OS directly, I spun up a quick Ubuntu 22.04 VM w/ two CPU cores, 2GB of RAM, and 10GB of SSD space. Those specs should be sufficient to run most workflows, and seem to be more or less in line with the default “ubuntu latest” runners that GitHub Actions provides to you.
To easily create the VM from the command line, I used the Multipass utility provided by Canonical (the company behind Ubuntu). In the end, it created a pre-configured VM using Windows Hyper-V.
<todo: multipass CLI and Hyper-V screenshots>
This could have just been done by clicking a few buttons in the Hyper-V manager, but using multipass does nicely abstract away some of the choices that you might need to make if doing things manually. Multipass would also likely come in handy if you wanted to create multiple VMs, or set up some automation to do it for you.
Install the GitHub Actions Runner Daemon
For the VM to be able to accept new jobs from GitHub Actions, it first needs to have the runner daemon installed, and have it registered with your GitHub repository/organization/enterprise.
Setup only took a few minutes, and can be done by following the directions here: <todo: link to github article>
Once completed, you should see your self-hosted runner listed in GitHub as ready and waiting to accept new jobs.
<todo: insert image of self-hosted runner list in GitHub>
Install Docker (optional)
If you’re unfamiliar with Docker, there’s no time like the present to learn! Here’s a good resource for learning more: https://www.docker.com/101-tutorial/.
If you’ve grown accustomed to containerizing your jobs in your workflows, then good news… they’re able to run on your self-hosted runner just as they would anywhere else. Which of course, is one of the main reasons to use Docker in the first place. Here's how to install: https://docs.docker.com/engine/install/ubuntu/.
After installing, you can verify that the daemon is running by executing:
```
sudo docker ps
```
From your command line. And you should see output like the following:
So, that’s great. Docker’s all setup and running. But before you can actually run containers via the GitHub Actions runner, you’ll need to add the user account under which the GitHub actions daemon is running to the `docker` users group. You can do so by running:
```
sudo usermod -aG docker $USER
```
Having done this, you can verify it like so:
```
su ubuntu -
docker ps
```
You should now see the same output you saw above when you executed the command via `sudo`.
Putting It All Together and Executing a Test Workflow
Now that the installation and configuration is complete, let’s create a quick workflow to verify that everything is working as we expect.
I created this simple example workflow that allows us to test jobs that run both directly on the Linux VM itself, and within the VM as a docker container.
<todo: insert the sample workflow code>
Upon push up to GitHub, we should then be able to navigate over to the “Actions” tab in GitHub, and see that the workflow is executing.
<todo: insert screenshot of actions workflow>
Then, by clicking into each of the jobs, we should be able to see the simple output that we are expecting to see.
<todo: insert screenshots of the jobs>
Success!
Conclusion
While this was a fairly simple example, it should give you all of the steps you need to create a basic self-hosted runner setup suitable for use on smaller projects.
To set up something more complex that would be suitable for meeting the needs of an enterprise software development organization, there would be many more steps involved in the setup of a production-grade cluster of self-hosted runners. But regardless, the basic steps would be the same.
Hopefully you’ve found this helpful. I invite you to contact me via my contact page with any questions or feedback.