Runner Security

Runners are the most sensitive component of a GitLab CI/CD setup. They execute arbitrary code from your repositories, they have access to the secrets you've configured, and they often have privileged access to your infrastructure — cloud credentials, deployment keys, and network access to internal systems. A compromised runner is a serious security incident.

What makes runner security especially challenging is that runners execute code that changes constantly. Every push to a repository might introduce new code that runs on a runner. If a contributor's account is compromised, or if a malicious contributor is added to a project, they could push code that attacks the runner or exfiltrates its secrets. A secure runner architecture anticipates this — it assumes that job code is untrusted and designs accordingly.

The good news is that GitLab and Kubernetes provide excellent tools for securing runners. The bad news is that securing them requires deliberate effort. This guide covers the controls you need: protected runners, tags, isolation, ephemeral runners, secrets management, and network policies. Together these form a defense-in-depth strategy that keeps your CI/CD infrastructure safe even when job code is malicious.

Before diving into controls, it's worth being explicit about what we're protecting against. A runner faces several classes of threats:

A protected runner is a runner that only accepts jobs from protected branches and tags. This is one of the most important security controls you can put in place, because it establishes a clear boundary: only code that has been reviewed and merged can run on trusted infrastructure.

Why does this matter? Feature branches are where developers experiment. They may include work-in-progress code, unreviewed contributions from external collaborators, or code from a compromised developer account. If a feature branch can access a production deploy runner, an attacker with push access to that branch can run arbitrary code on the runner — potentially with production credentials. By marking the runner as protected, you ensure that only code on main (or other protected branches) can run there.

Protected runners are configured during registration with the --access-level ref_protected flag, or can be modified later in the runner's configuration. When a non-protected branch pushes a job that needs the runner's tags, GitLab will not assign that job to the protected runner — the job stays pending (or is picked up by another non-protected runner with matching tags, if one exists).

Tags are not just for routing jobs — they're also a security mechanism. By tagging runners according to their trust level (e.g., trusted, production, untrusted), you can ensure that jobs only run on runners with the right trust profile. A job that builds untrusted code should be tagged untrusted so it can't accidentally end up on a production runner.

The most common pattern is to have separate runners for different environments. A "staging" runner might accept jobs from feature branches, while a "production" runner only accepts jobs from main. This is enforced by combining tags with ref_protected: the production runner has the production tag and ref_protected access level, so only protected branches with the production tag can use it.

A related control is to disable run_untagged on production runners. If a runner has run_untagged = true, it will accept jobs that have no tags at all — which means a job in an untrusted project could accidentally end up on your production runner. Setting run_untagged = false forces every job that wants to use the runner to explicitly declare the runner's tags, which makes routing intentions explicit.

Isolation is the single most important security property of a runner. When a job is isolated, any damage it does is confined to its own environment — it can't see other jobs' data, can't affect the host, and can't attack other infrastructure. Without isolation, one compromised job can bring down everything.

The shell executor provides no isolation at all. Jobs run directly on the host with the runner user's permissions. They can read each other's files, interfere with running processes, and potentially compromise the host. This is why the shell executor should only be used for trusted code.

The Docker executor provides strong isolation. Each job runs in its own container with its own filesystem, process namespace, and network namespace. A job can't see other jobs' files, can't kill other processes, and generally can't break out of its container (barring a container escape vulnerability, which are rare but not impossible).

The Kubernetes executor provides even stronger isolation when configured correctly. Each job runs in its own pod, which can have its own security context, network policy, and runtime class. With gVisor or Kata Containers, jobs run in a sandboxed kernel, providing isolation even against kernel exploits.

Secrets are the most attractive target for an attacker who compromises a job. If a job can read your cloud credentials, it can use them to exfiltrate data, spin up expensive resources, or pivot to other systems. Managing secrets safely is therefore a critical part of runner security.

The first principle is to limit which jobs can access which secrets. GitLab provides this through protected variables, which are only available on protected branches. By marking production credentials as protected, you ensure that only code on main can access them — feature branches and untrusted contributions never see the production secrets.

The second principle is to limit the damage a leaked secret can cause. Use secrets with the minimum necessary permissions. Use short-lived credentials where possible (e.g., temporary AWS STS tokens instead of long-lived access keys). Rotate secrets regularly. Use masking to prevent accidental disclosure in logs.

The third principle is to use file variables for secrets that are naturally files (SSH keys, TLS certificates, service account JSON). File variables don't leak into logs the way environment variables can, and they work naturally with tools that expect file paths.

An ephemeral runner is a runner that runs exactly one job and then shuts down. This is the strongest isolation model: since the runner doesn't persist between jobs, a compromised job can't affect subsequent jobs, and there's no state for an attacker to persist.

There are several ways to achieve ephemeral runners. GitLab's Kubernetes executor can be configured to run in "ephemeral mode" where each job gets a fresh runner pod. The GitLab Runner Operator can create ephemeral runner pods on demand. And the Docker Machine executor creates a fresh VM per job — one of the original ephemeral patterns.

Ephemeral runners are especially valuable for untrusted code and for CI/CD environments where job isolation is critical. The trade-off is startup cost — spinning up a fresh runner per job is slower than reusing a long-lived one. But for security-sensitive workloads, this cost is worth paying.

Runners often need network access to GitLab, to container registries, and to the infrastructure they deploy to. But they don't need access to everything. Network segmentation limits what a compromised runner can reach, and network policies enforce that segmentation.

The key controls are: use NetworkPolicy to restrict egress from runner pods; don't run runners in the same network as production; use VPC peering or private links instead of exposing services publicly; firewall rules to restrict what runners can reach; and egress proxies to mediate and log outbound traffic.

For cloud environments, the runner's network access is often controlled by security groups or firewall rules at the network layer. This is even stronger than Kubernetes NetworkPolicy because it can't be bypassed by pod configuration changes.

Runner tokens are the credentials that allow a runner to authenticate to GitLab and pick up jobs. They're sensitive: anyone with a runner token can register a runner, and anyone with a runner's authentication token can impersonate that runner. Treating tokens as secrets is essential.

There are two types of tokens: registration tokens (used once to register a runner) and authentication tokens (used by the runner to authenticate with GitLab). The newer GitLab versions favor "runner authentication tokens" (created via UI/API) which don't require a registration token to be present at all. This is more secure because there's no long-lived registration token to leak.

Best practices: rotate tokens periodically; store tokens as Kubernetes Secrets (not ConfigMaps); never commit tokens to a repository; use protected runners for production; and audit registered runners regularly to detect unauthorized registrations.