Self-hosted CI/CD: the concept, the architecture and runners

Self-hosted CI/CD: the concept, the architecture and runners

In the first part of the series we took apart what CI/CD is, what a pipeline consists of and how Continuous Delivery differs from Continuous Deployment. One question was left there without an answer: on which machine, exactly, are all these builds and tests executed?

By default the answer is simple: on someone else's. You write runs-on: ubuntu-latest, and GitHub brings up a virtual machine for you, runs your code on it, and then destroys it. This is convenient exactly up to the moment when the limits, the cost or the requirements about where exactly your code has to be executed start getting in the way. Then the self-hosted runner comes onto the scene: your own machine that does the same work, but under your control.

This article is not about installation (that will be in parts 3, 4 and 5 for GitHub Actions, GitLab CI and Gitea Actions respectively), but about the architecture: how the runner is arranged, why it does not need open ports, what kinds of executors there are, how many resources are really needed and which mistakes in the setup turn your runner into an open door into the infrastructure.

1.1 Cloud runners: how this works by default

When the pipeline is executed on the platform's infrastructure, every job gets a clean virtual machine with a pre-installed set of tools: compilers, Docker, typical SDKs, a cache of popular images. After the job finishes, the machine is destroyed together with all the data.

The advantages are obvious: nothing has to be administered, the environment is always clean, the isolation between jobs is complete, the platform updates the images itself. For most projects this is enough, and it is exactly cloud runners that you should start with. But the model has four limitations that a growing team runs into sooner or later.

• Minute limits. The free quota is finite, and it grows not as fast as the number of pipelines.
• The cost at a large volume. Minutes above the quota are paid for, and the bill grows linearly with the number of builds.
• A closed perimeter. The code and the secrets are executed on someone else's infrastructure. For a part of projects this is unacceptable by contract or by regulation.
• Fixed resources. A standard runner is 2 vCPU and a few gigabytes of RAM. If the build requires a GPU, 32 GB of memory or an ARM processor, the standard configuration does not fit.

1.2 Self-hosted: your own server executes the jobs

A self-hosted runner is an agent process that you start on your own server, on a VPS or even on a machine in the office. It registers in your project on GitHub or GitLab and starts receiving jobs from there. All the rest does not change: the pipeline is described by the same YAML, the logs and the statuses are visible in the same interface, the buttons and the notifications work as before.

Only one thing changes: the line with the choice of the runner in the pipeline config.

# GitHub Actions: instead of ubuntu-latest
jobs:
  build:
    runs-on: [self-hosted, linux, x64]

# GitLab CI: the job will go to a runner with the matching tag
build:
  tags:
    - self-hosted

It is important to understand the boundary of responsibility. The platform keeps at itself: the repository, the interface, the job queue, the storage of secrets, the artifacts. You take on only the execution: the machine, its resources, OS updates, isolation and security. This is not a replacement of GitHub or GitLab with your own server, but the moving of one specific part of the system closer to yourself.

2. When self-hosted is justified

Your own runner is not "more correct" and not "the grown-up level". It is a trade-off: you get control and a predictable cost, and in exchange you take on the administration. Below are four situations in which this exchange is profitable.

2.1 Exhausted free minutes

This is the most frequent reason. The quotas current as of the middle of 2026 look like this:

Two details that regularly come as a surprise. The first: in GitHub the quota is spent with a multiplier depending on the OS, that is, a build on Windows eats twice as much quota for the same time, and on macOS ten times as much. The second: in GitLab the free minutes are counted for the whole top-level group, and not per user or project, therefore a team of five people and a team of fifty start from the same figure.

2.2 A closed perimeter

The second reason has nothing in common with money. There are projects in which the code must not physically end up on someone else's infrastructure: customer requirements, the banking or medical sector, state systems, work with personal data.

Here it is important to understand a nuance. A self-hosted runner by itself does not make the perimeter closed: if the repository lives on GitHub.com, the code is already there anyway. Your own runner solves a different task — so that the execution takes place inside your perimeter: a build with access to internal package repositories, tests against a database in a private network, a deploy to servers that do not have a public address. A fully closed perimeter is already a self-hosted git platform (GitLab CE, Gitea, Forgejo) plus your own runners.

A practical bonus of the same property: a runner inside your network has access to the production servers by private addresses, and you do not need to open SSH to the outside for the sake of the deploy.

2.3 Specific hardware and environment

The standard runners of the platforms are universal Linux machines of medium power. If your task goes beyond these limits, your own runner is often the only option:

• GPU for training and testing models or for a build with CUDA.
• ARM for a native build for ARM servers, Raspberry Pi or mobile platforms. Emulation through QEMU works, but is several times slower.
• A specific OS version: building a package for CentOS 7 or testing on a kernel that is not in the standard images.
• A lot of RAM: building a large monorepository or integration tests that bring up a dozen services at once.
• A fast local cache: an NVMe with a warmed-up cache of dependencies and Docker layers gives an acceleration that will not be there on a clean machine every time.

2.4 The cost: a calculation instead of feelings

The comparison makes sense only on specific figures. Let us take a team that does 60 builds a day of 10 minutes each, that is, approximately 13,000 minutes a month:

For a small project with a few builds a day the picture is the opposite: the free quota is enough with a reserve, and there is no sense in paying for a VPS. The break-even point in most teams comes between 3,000 and 10,000 minutes a month, depending on the platform and on how heavy the builds are.

And an honest warning right away: "free" in the column of your own runner concerns only the minutes. The real cost is the VPS plus the engineer's time for the initial setup and the further maintenance (OS updates, cleaning the disk, updating the agent itself). For one machine this is a few hours at the start and approximately an hour a month afterwards. If your saving on minutes is smaller than this hour, stay on cloud runners.

3. How a runner is arranged

A runner looks more complex than it is. In reality it is an ordinary background process (a systemd service or a container) that does one simple thing in a loop: it asks the platform whether there is work for it.

3.1 The polling model: the runner goes for the jobs itself

The key architectural property: the platform never connects to the runner. It is all the other way round. After registration the agent opens an outgoing HTTPS connection to the platform and holds it, periodically asking for the next job. GitLab Runner does this with a request to the API with an interval of a few seconds, the GitHub agent holds a long connection and waits for a job to be assigned.

Three practical consequences follow from this:

• No public IP and no open ports are needed. The runner works behind NAT, in a home network, behind a corporate firewall.
• No incoming traffic is needed at all. For the firewall this means: incoming connections are forbidden completely, except for your SSH for administration.
• The runner survives a loss of connection. After the network is restored it will simply continue polling, and the platform will put the job in the queue and wait for a free agent.

Registration is done once with a token issued by the platform. Afterwards the agent has its own credentials and no longer uses the registration token. A token with the rights to register a runner is a sensitive secret: having it, an outsider can connect their own machine to your project and start receiving your jobs together with the secrets.

3.2 What happens inside a job

When the platform assigns a job, the agent goes through one and the same cycle:

1. Receiving the description of the job. The repository, the commit, the environment variables, the list of steps and a temporary access token that is valid only while the job lasts.
2. Preparing the working directory. Creating or cleaning the workspace, bringing up a container (if the executor is a container one), connecting service containers of the database-for-tests kind.
3. Checkout. Getting the code with the temporary token, usually a shallow clone for speed.
4. Executing the steps. The commands are executed sequentially, each in its own process. A non-zero return code stops the job, unless the step is explicitly marked as optional.
5. Streaming the logs. The output is transmitted to the platform in parts during the execution, therefore the log is visible in real time and not after the finish.
6. Artifacts and cache. The specified files are packed and uploaded to the platform's storage, the cache is saved for the following runs.
7. Finishing and cleaning up. The agent reports the status and frees itself for the next job.

3.3 Executors: where exactly the commands are executed

The executor is the answer to the question "in what environment does the agent run your commands". Both the convenience and the level of isolation depend on the choice.

Shell. The commands are executed directly on the host, on behalf of the user under which the agent works. The simplest option: nothing has to be installed, everything that is on the server is available in the build. The downsides are direct too: no isolation between jobs at all, a shared environment that gradually gets polluted, and a full dependence of the build on what somebody installed on the server last month.

Docker. Every job is executed in a fresh container from the specified image. This is the standard choice for most teams: the environment is described in the pipeline config, every job starts from a clean state, the versions of the tools are fixed by the image.

# config.toml for GitLab Runner with the Docker executor
concurrent = 4
check_interval = 3

[[runners]]
  name = "build-01"
  url = "https://gitlab.com/"
  token = "glrt-..."
  executor = "docker"
  [runners.docker]
    image = "alpine:3.20"
    privileged = false
    volumes = ["/cache"]

Kubernetes. Every job is a separate pod in the cluster. It gives automatic scaling under load and natural isolation, but makes sense only there where the cluster already exists and there is somebody to maintain it.

In GitHub Actions there is no terminology of executors: by default the agent executes the steps on the host (an analogue of shell), and containerisation is switched on at the level of the job with the container: key. For clusters there is a separate component that creates one-time runners in Kubernetes.

4. Infrastructure requirements

4.1 How many resources are really needed

The agent itself consumes almost nothing: it is a few tens of megabytes of memory in the waiting state. It is not it that eats the resources, but your builds, therefore the requirements are determined by the heaviest job in the pipeline.

The disk is exactly what is underestimated most often. Docker images, the layer cache, downloaded dependencies and artifacts grow unnoticeably, and the first outage on your own runner in most teams is not a lack of memory, but no space left on device in the middle of a build. Plan for automatic cleaning on a schedule and monitoring of free space from the first day.

The second most frequent one is a shortage of RAM in frontend builds: building a large JavaScript project easily runs into a gigabyte or two, and the process finishes without an understandable error, it simply disappears. If the build fails unpredictably and without logs, the first thing to look at is whether the memory-freeing mechanism killed it.

4.2 The network: only outgoing connections

For its work the runner needs only outgoing HTTPS to the platform and to those resources from which it pulls the dependencies: package registries, image registries, distribution mirrors. No incoming ports are needed at all.

A minimal reasonable firewall configuration on the runner's server:

# incoming: only SSH for administration, better from known addresses
ufw default deny incoming
ufw default allow outgoing
ufw allow from 203.0.113.0/24 to any port 22 proto tcp
ufw enable

A separate story is the runner's access to production. If the runner performs the deploy, it has to reach the target servers, and the best option is a private network between them: the deploy goes by an internal address, the public SSH on the production server remains closed. Putting the runner into the same network as production is worth it exactly for this, and not for convenience.

4.3 One runner or several

One agent by default executes one job at a time. If there are three parallel checks in a stage, they will be executed one after another, and there will be no gain in time.

Parallelism is achieved in two ways. In GitLab Runner one process can serve several jobs at once: the concurrent parameter in config.toml sets the total number. In GitHub Actions one agent is one job, therefore for parallel execution several agents are started, including on one machine.

A practical rule: one parallel job for every 2 vCPU. There is no sense in setting concurrent = 8 on a two-core VPS, because the jobs will start competing for the processor and each will be executed longer than if they went one after another.

When there is more than one runner, it is useful to divide them by purpose right away through tags or labels: a separate runner for builds and tests, a separate one for the deploy to production. This is not only about resources, but also about security: the production secrets then get only onto the machine that really does the deploy.

5. Security: the main principles

A self-hosted runner is a machine that by definition executes code from the repository. If it has access to production and to the secrets, it automatically becomes one of the most valuable targets in your infrastructure. The four rules below close most of the typical mistakes.

5.1 Do not connect your own runner to public repositories

This is the main rule, and it has no exceptions for beginners. Both platforms directly do not recommend using self-hosted runners in public projects, and the reason is fundamental: any outside person can create a fork of your repository, change the pipeline file and open a pull request. If your runner takes this job, it will execute someone else's code on your server.

Further the scenario develops fast: reading the files on the disk, an attempt to reach the secrets in the memory of the process, scanning the private network into which the runner is placed, gaining persistence through cron or a systemd unit. For public repositories the correct answer is cloud runners: there every job is executed on a one-time machine that is not a pity to lose.

If your own runner for a public project is nevertheless needed (for example, a build for ARM), the obligatory minimum is one-time (ephemeral) runners that are destroyed after every job, an isolated network without access to anything valuable and an obligatory manual confirmation of the pipeline run for pull requests from external forks.

5.2 The Docker executor as the minimal level of isolation

A container is not a full-fledged sandbox, but it is much better than executing commands directly on the host. The minimal set of requirements:

• Without privileged. A privileged container is in fact root rights on the host.
• Without mounting the Docker socket. Access to /var/run/docker.sock allows the job to start a container with the host's root partition mounted.
• Resource limits. CPU and memory limits per container, so that one job does not take down the whole server.
• A fresh state. Every job starts from a new container, without reusing the previous one.

5.3 A separate system user with minimal rights

The agent must never work as root. Create a separate system user on behalf of which the service works:

sudo useradd --system --create-home --shell /bin/bash ci-runner
# without sudo, without access to other people's home directories
sudo chmod 750 /home/ci-runner

Further down the list: this user has no sudo rights, the working directory belongs only to them, the service is started through systemd on their behalf, and the SSH keys for the deploy (if they are needed here) are issued to a separate restricted user on the target server, and not to root.

5.4 Secrets only through the platform's mechanism

There must not be a single password, token or key in the repository. The secrets are kept in the platform's mechanism (Secrets in GitHub, CI/CD Variables in GitLab) and get into the job as environment variables for the time of its execution.

A few rules that are worth applying right away:

• Masking in the logs. Both platforms can hide the values of secrets in the output. This is not a protection against deliberate extraction, but it saves you from an accidental echo.
• Binding to protected branches. Production secrets are available only to jobs from a protected branch or a defined environment, and not to any branch of any developer.
• Minimal rights. A token for the deploy can only deploy. An SSH key is bound to one user and, if needed, to one allowed command.
• Rotation. Secrets are changed after a person with access leaves and after any suspicious incident with the runner.

6. Conclusion

A self-hosted runner is a component simple in architecture: the agent polls the platform, receives a job, executes the steps in the chosen environment, gives back the logs and the artifacts. All the complexity is not in the agent, but in what is around it.

• When to move. When the quota of minutes is exhausted every month, when the execution has to take place inside your perimeter, when hardware is needed that the standard runners do not have. For a small project with a dozen builds a day cloud runners are cheaper and simpler.
• The network. The runner works on outgoing connections, therefore it does not need a public IP and open ports. This is convenient and safe at the same time.
• Resources. You can start with 1 vCPU and 1 GB of RAM, a realistic working minimum for builds with Docker is 2 vCPU, 4 GB and 50 GB of disk. The disk and cleaning up after jobs are more important than it seems at the start.
• Security. No public repositories without isolation, a Docker executor without privileged, a separate user without sudo, secrets only through the platform and only for protected branches.

One key difference from a cloud runner remains: there every build starts from a clean sheet, and at your place the cleanliness has to be maintained on your own. This is exactly the price you pay for control, a predictable cost and access to your own network.

What comes next in the series

The architecture is common to both platforms, but the implementation is noticeably different: in GitHub Actions it is an agent with labels and steps from the marketplace, in GitLab CI it is gitlab-runner with tags, executors and its own config. Therefore the next two parts are practical and parallel in structure.

In the third part we put our own runner for GitHub Actions on a VPS and assemble the first working pipeline: registering the agent, the systemd service, the structure of the workflow, the cache. In the fourth the same for GitLab CI: gitlab-runner, executors, .gitlab-ci.yml, stages and rules. The fifth part is about Gitea Actions, when the git platform and the pipeline live on one own server. The sixth sums up the series with a comparison of the three platforms and an answer to the question of what to choose for specific conditions.

Frequently asked questions