Secrets Management
CI/CD Variables
Masked & Protected
Vault Integration
GitLab Secrets Manager
Learn how to store, protect, and retrieve secrets in GitLab CI/CD. From masked variables to external secret managers, build a secure credential workflow for your pipelines.
CI/CD Variables
External Secrets
GitLab
Why Secrets Management Matters
Every pipeline needs credentials: API tokens to push images, database passwords to run migrations, SSH keys to deploy to servers, and cloud provider keys to provision infrastructure. These secrets are the keys to your kingdom. If a secret leaks — through a log, a misconfigured variable, or a compromised dependency — an attacker gains the same access your pipeline has. The Axios npm package compromise in March demonstrated exactly this risk: organizations running a poisoned version had to assume every credential their pipelines touched was in an attacker's hands [citation:8].
GitLab offers a spectrum of options for handling secrets, from simple CI/CD variables to dedicated secrets managers. The right choice depends on sensitivity. Non-sensitive configuration — a log level, a feature flag name — can live in plain CI/CD variables. Sensitive credentials should be stored in a secrets manager, where access is scoped, audited, and revocable [citation:1]. The challenge is that many teams default to CI/CD variables for everything because they're convenient, not because they're secure.
Understanding the layers of protection available for CI/CD variables — masking, hiding, protection, and environment scoping — lets you use them safely for moderately sensitive data. For the most sensitive credentials, GitLab's native integrations with external providers (HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager) and the new GitLab Secrets Manager provide the strict access controls and audit trails that production security requires [citation:2].
Core Principle: Use CI/CD variables for non-sensitive configuration. Use masked and protected variables for moderately sensitive data. Use a dedicated secrets manager for credentials that would cause harm if exposed.
CI/CD Variables
Convenient but less secure. Stored in project/group settings. Suitable for non-sensitive data or when combined with masking and protection.
Masked Variables
Hidden in job logs. Prevents accidental exposure. Does not prevent access by malicious scripts.
Protected Variables
Only available on protected branches and tags. Prevents feature branches from accessing production secrets.
External Secrets
HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager. Secrets stored outside GitLab with strict access controls.
GitLab Secrets Manager
Native GitLab feature (Beta). Secrets scoped to specific jobs, environments, and branches. Built on OpenBao.
ID Tokens
JWT tokens for authenticating with external secrets providers. Enable keyless authentication to cloud providers.
CI/CD Variables: The Foundation
CI/CD variables are the simplest way to pass configuration and credentials to jobs. They are key-value pairs defined at the project, group, or instance level, and they're injected into every job that runs in that scope. A variable defined at the group level is available to all projects in the group; a project-level variable is available only to that project. Variables can also be defined directly in .gitlab-ci.yml, but this is only appropriate for non-sensitive values since the file is version-controlled [citation:1].
The convenience of variables is also their risk. A variable defined at the project level is available to every job, regardless of whether that job needs it. A test job that runs on a feature branch has the same access to a production database password as the deploy job — unless you use protection. This violates the principle of least privilege, and it's why GitLab provides masking, protection, and environment scoping as layers of defense.
Variables can also be overridden at the pipeline level (via the UI or API when triggering a pipeline), which makes it difficult to determine after the fact which value was actually used. And a misconfigured pipeline — for example, a script that accidentally echoes a variable — can expose the value in job logs. These risks don't mean variables are unusable, but they do mean variables alone are insufficient for highly sensitive credentials [citation:1].
# Define variables in .gitlab-ci.yml (non-sensitive only)
variables:
NODE_VERSION: "18"
BUILD_ENV: "production"
LOG_LEVEL: "info"
# Variables are available in all jobs
build:
script:
- echo "Building with Node $NODE_VERSION"
- npm run build
# Job-level variables override global ones
test:
variables:
NODE_ENV: "test"
script:
- npm test
# Variable precedence (highest to lowest):
# 1. Pipeline variables (UI/API trigger)
# 2. Project variables (UI)
# 3. Group variables (UI)
# 4. Instance variables (UI)
# 5. .gitlab-ci.yml job-level variables
# 6. .gitlab-ci.yml global variables
# 7. Predefined variables
# Never commit secrets to .gitlab-ci.yml
# BAD:
# variables:
# DATABASE_PASSWORD: "supersecret123"
# GOOD: Define in GitLab UI, not in code
# Settings > CI/CD > Variables > Add Variable
Variable Risks:
- Scope creep: Variables are injected into every job unless scoped
- Overridability: Pipeline variables can override UI variables, making auditing difficult
- Log exposure: A misconfigured script can print a variable's value
- Settings access: Anyone with Maintainer access to project settings can view non-masked variables
Masked Variables
Masking is the first line of defense for sensitive variables. When a variable is masked, GitLab replaces its value with [MASKED] in job logs. If a script accidentally echoes the variable — or if a tool prints its environment — the actual value never appears in the log. This prevents the most common cause of secret leaks: accidental logging [citation:1].
Masking has requirements and limitations. The value must be at least 8 characters and cannot contain newlines. These constraints exist because masking works by pattern-matching the value in log output; short values would produce too many false positives, and multi-line values cannot be reliably masked. Masking is also not a complete defense: a malicious script can encode the value (base64, hex) and print the encoded form, bypassing the mask. Masking protects against accidents, not against a determined attacker who already has code execution in your pipeline [citation:17].
For variables that are both masked and hidden, GitLab adds another layer: the value is not displayed in the UI at all, even to users with Maintainer access. The masked_and_hidden attribute (available via the API) creates a variable whose value cannot be retrieved through the UI or API — only the job that uses it sees the value [citation:3]. This is the most secure form of variable storage within GitLab's native system, short of using a dedicated secrets manager.
# Masked variable configuration (UI)
# Settings > CI/CD > Variables
#
# Key: DATABASE_PASSWORD
# Value: supersecretpassword123
# Masked: ✓
# Protected: ✓ (recommended)
# Environment scope: production (recommended)
# Using masked variable in a job
deploy_production:
stage: deploy
script:
- echo "Connecting with password $DATABASE_PASSWORD"
# Log shows: Connecting with password [MASKED]
- ./deploy.sh
environment:
name: production
rules:
- if: $CI_COMMIT_BRANCH == "main"
# Masking requirements:
# - Minimum 8 characters
# - No newlines
# - Must be a single line
# - Cannot contain certain special characters
# Masking limitations:
# - Does not prevent script access to the value
# - Can be bypassed by encoding the value
# - Does not protect against malicious pipeline code
# Masked + Hidden (API only, GitLab 17.4+)
# Variable value is not displayed in UI at all
# Only the job that uses it can see the value
curl --request POST \
--header "PRIVATE-TOKEN: " \
--url "https://gitlab.example.com/api/v4/projects/1/variables" \
--form "key=DATABASE_PASSWORD" \
--form "value=supersecret123" \
--form "masked=true" \
--form "hidden=true" \
--form "protected=true"
Masking Best Practices:
- Mask every variable that contains a credential, even moderately sensitive ones
- Combine masking with protection for production secrets
- Use masked + hidden variables for the most sensitive credentials
- Never rely on masking alone — treat it as accident prevention, not attack prevention
- Rotate any secret that was accidentally logged, even if masked
Protected Variables
Protection controls where a variable is available, not whether its value is visible. A protected variable is only injected into pipelines running on protected branches or protected tags. This is a critical security boundary: a feature branch pipeline — which might run code from an untrusted contributor or a compromised dependency — cannot access production credentials. Only pipelines running on branches that maintainers have explicitly protected can use the variable [citation:17].
The combination of masking and protection is the recommended baseline for production secrets stored as variables. Masking prevents accidental log exposure; protection prevents feature branches from accessing credentials. Together, they enforce a simple rule: only trusted code on trusted branches can access sensitive credentials. A merge request from a fork or a feature branch runs without the protected variable, so even if the MR contains malicious code, it cannot exfiltrate the secret [citation:1].
Protection does not solve the scope problem entirely. A protected variable is available to every job on the protected branch, including test jobs that don't need it. For finer-grained control, environment scoping limits a variable to jobs deploying to a specific environment. A production database password scoped to the production environment is only available to jobs with environment: production — not to a test job that happens to run on the same branch.
# Protected variable configuration (UI)
# Settings > CI/CD > Variables
#
# Key: AWS_SECRET_ACCESS_KEY
# Value: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
# Masked: ✓
# Protected: ✓
# Environment scope: production
# Only jobs on protected branches can access this variable
deploy_production:
stage: deploy
script:
- aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
- aws s3 sync dist/ s3://my-bucket/
environment:
name: production
rules:
- if: $CI_COMMIT_BRANCH == "main" # main is protected
# Feature branch job cannot access protected variable
test_feature:
stage: test
script:
- echo $AWS_SECRET_ACCESS_KEY # Empty! Protected variable not available
rules:
- if: $CI_COMMIT_BRANCH != "main"
# Environment-scoped variable (UI)
# Key: DATABASE_URL
# Value: postgres://prod-db:5432/app
# Environment scope: production
# Masked: ✓
# Protected: ✓
# Only jobs deploying to production environment can access
deploy_prod:
environment:
name: production
script:
- echo $DATABASE_URL # Available
rules:
- if: $CI_COMMIT_BRANCH == "main"
test_staging:
environment:
name: staging
script:
- echo $DATABASE_URL # Empty! Scope mismatch
rules:
- if: $CI_COMMIT_BRANCH == "main"
# Protected branches configuration:
# Settings > Repository > Protected branches
# - main: Maintainers can push, No one can force push
# - production: Maintainers can push
# - v*: Maintainers can push (tags)
# Protected tags configuration:
# Settings > Repository > Protected tags
# - v*: Maintainers can create
Protection + Masking + Scoping = Layered Defense:
- Masking: Prevents accidental log exposure
- Protection: Prevents feature branches from accessing credentials
- Environment scoping: Prevents jobs from accessing credentials for other environments
External Secrets: HashiCorp Vault Integration
For organizations with strict security requirements, storing secrets in GitLab variables — even masked and protected ones — is not enough. Dedicated secrets managers like HashiCorp Vault provide centralized storage, fine-grained access policies, audit logging, and secret rotation. GitLab integrates with Vault natively: CI/CD jobs can authenticate to Vault using a JWT (ID token) and retrieve secrets at runtime, without ever storing the secret value in GitLab [citation:2].
The authentication flow uses OpenID Connect. GitLab generates an ID token (a signed JWT) for the job, containing claims about the project, branch, environment, and user. The job presents this token to Vault, which verifies it against GitLab's OIDC discovery URL and checks it against a configured role. If the claims match the role's bound claims — for example, "only jobs from project 42 on protected branches deploying to production" — Vault returns a short-lived token with access to specific secrets [citation:10].
This approach has significant advantages. Secrets never exist in GitLab's database; they live in Vault, governed by Vault's policies. Access is audited in Vault's logs, showing exactly which job accessed which secret and when. Rotation is handled in Vault, and jobs always retrieve the current value. The trade-off is operational complexity: you must run and maintain Vault, configure authentication and policies, and manage the integration. For organizations already using Vault, this is a natural fit. For others, GitLab Secrets Manager (covered next) offers a simpler native alternative.
# Configure Vault integration
# 1. Set CI/CD variables for Vault connection
# VAULT_SERVER_URL: https://vault.example.com:8200
# VAULT_AUTH_ROLE: myproject-production (optional)
# VAULT_AUTH_PATH: jwt (default)
# VAULT_NAMESPACE: admin (for HCP Vault)
# 2. Define a job that retrieves secrets from Vault
deploy_production:
stage: deploy
image: alpine:latest
id_tokens:
VAULT_ID_TOKEN:
aud: https://vault.example.com
secrets:
DATABASE_PASSWORD:
vault: production/db/password@ops
token: $VAULT_ID_TOKEN
script:
- echo "Connecting to database..."
- ./deploy.sh --password $DATABASE_PASSWORD
environment:
name: production
rules:
- if: $CI_COMMIT_BRANCH == "main"
# Vault secrets syntax:
# vault:
@
# Example: production/db/password@ops
# - path: production/db/password
# - engine: ops (KV v2 engine mount point)
# Multiple secrets in one job
deploy:
secrets:
DATABASE_PASSWORD:
vault: production/db/password@ops
token: $VAULT_ID_TOKEN
API_KEY:
vault: production/api/key@ops
token: $VAULT_ID_TOKEN
SSH_KEY:
vault: production/ssh/key@ops
token: $VAULT_ID_TOKEN
file: true # Write to file instead of variable
script:
- ./deploy.sh
# Vault server configuration (one-time):
# vault auth enable jwt
# vault write auth/jwt/config \
# oidc_discovery_url="https://gitlab.example.com" \
# bound_issuer="gitlab.example.com"
#
# vault policy write myproject-production - <
Vault Integration Benefits:
- Secrets never stored in GitLab
- Fine-grained access policies in Vault
- Complete audit trail of secret access
- Secret rotation without pipeline changes
- Dynamic secrets (database credentials generated on demand)
GitLab Secrets Manager
GitLab Secrets Manager is a native secrets management solution built on OpenBao, introduced in public beta with GitLab 19.0. It addresses the gap between the convenience of CI/CD variables and the security of external vaults: it keeps secrets within GitLab's existing project and group structure, governed by the same access controls you already use for code, but with the scoping and audit capabilities of a dedicated secrets manager [citation:8].
The key difference from CI/CD variables is scoping. A secret in Secrets Manager is not injected into every job by default. Instead, each job must explicitly request the secrets it needs using the secrets: keyword. The secrets backend checks the job's identity — its branch, environment, and protection status — against the secret's scope rules before returning the value. A production database password can be scoped to only be available to jobs on protected branches targeting the production environment. A test job on a feature branch cannot access it, even if it's part of the same pipeline [citation:8].
GitLab Secrets Manager also improves security at the point of use. By default, secrets are written to a temporary file, and the job receives the file path as an environment variable, rather than the secret value itself. This reduces exposure in subprocesses, crash dumps, and telemetry. When the job ends, the secret is discarded; nothing persists on the runner. This is a significant improvement over CI/CD variables, which remain readable in project settings indefinitely [citation:4].
# GitLab Secrets Manager usage (Beta, GitLab 19.0+)
# Requires enabling Secrets Manager for the project/group
# Define a secret (via UI or API)
# Key: DATABASE_PASSWORD
# Value: supersecret123
# Scope: environment: production, branch: protected
# Use the secret in a job
deploy_production:
stage: deploy
secrets:
DATABASE_PASSWORD:
gitlab_secrets_manager:
name: db-password
script:
- echo "Connecting to database..."
- ./deploy.sh --password $DATABASE_PASSWORD
environment:
name: production
rules:
- if: $CI_COMMIT_BRANCH == "main"
# By default, GitLab writes the secret to a temp file
# and passes the file path as an environment variable
# The script reads the file:
deploy:
secrets:
DB_PASSWORD:
gitlab_secrets_manager:
name: db-password
script:
- cat $DB_PASSWORD # Reads the file containing the secret
# Or:
- ./deploy.sh --password "$(cat $DB_PASSWORD)"
# To pass the value directly (less secure):
deploy:
secrets:
DB_PASSWORD:
gitlab_secrets_manager:
name: db-password
file: false # Pass value, not file path
script:
- echo $DB_PASSWORD
# Secret scoping:
# - By environment: production, staging, review/*
# - By branch: main, protected, feature/*
# - By protection status: only protected branches
# Wildcards are supported
# Benefits over CI/CD variables:
# - Secrets are not injected into every job
# - Access is checked at runtime
# - Secret values are discarded after job
# - No persistent storage in project settings
# - Audit trail of secret access
GitLab Secrets Manager Status:
- Beta: Introduced in GitLab 19.0, subject to change
- Paid add-on: Limited availability, billed via GitLab Credits (as of Aug 2026 update)
- Native integration: Built on OpenBao, part of GitLab platform
- Requires setup: Must be enabled for project/group before use
Choosing the Right Approach
The right secrets management approach depends on the sensitivity of the credential and your organization's security requirements. There is no single answer that fits every team, but there is a clear progression from least to most secure. Most organizations use a combination: plain variables for configuration, masked and protected variables for moderately sensitive data, and a secrets manager for production credentials.
For a small team just starting with GitLab CI/CD, masked and protected variables are a reasonable starting point. They provide meaningful protection against accidental exposure and feature-branch leaks, with minimal operational overhead. As the team grows and the number of secrets increases, the limitations become apparent: variables are hard to audit, hard to rotate, and always available to every job on the protected branch.
For organizations with compliance requirements, multiple teams, or a need for audit trails, an external secrets manager (Vault, AWS Secrets Manager, etc.) or GitLab Secrets Manager is the right choice. The operational cost is higher, but so is the security guarantee. The decision is ultimately a trade-off between convenience and control — and for production credentials, control should win.
Plain Variables
Non-sensitive config only. Log level, feature flags, version numbers. No protection needed.
Masked + Protected
Moderately sensitive. API keys for non-critical services, staging credentials. Good default for most teams.
GitLab Secrets Manager
Production credentials. Native GitLab, job-scoped, audit trail. Beta, paid add-on.
External Vault
Strictest requirements. Compliance, audit, rotation, dynamic secrets. Vault, AWS SM, Azure KV.
# Decision guide:
# 1. Is the value sensitive? (Would exposure cause harm?)
# NO -> Plain CI/CD variable
# YES -> Continue to 2
# 2. Is it a production credential?
# NO -> Masked + Protected variable
# YES -> Continue to 3
# 3. Do you have compliance/audit requirements?
# NO -> Masked + Protected + Environment-scoped variable
# YES -> Continue to 4
# 4. Do you already use HashiCorp Vault (or AWS/Azure/GCP)?
# YES -> External secrets integration
# NO -> GitLab Secrets Manager (if available) or external vault
Complete Secrets Management Pipeline
Here's a complete example showing a pipeline that uses all three approaches: plain variables for configuration, masked variables for a staging API key, and Vault for production database credentials.
# Complete secrets management pipeline
stages:
- build
- test
- deploy
# Non-sensitive configuration (plain variables)
variables:
NODE_VERSION: "18"
BUILD_ENV: "production"
# Build job: no secrets needed
build:
stage: build
image: node:$NODE_VERSION
script:
- npm ci
- npm run build
artifacts:
paths:
- dist/
# Test job: uses staging API key (masked + protected)
test_api:
stage: test
image: node:$NODE_VERSION
script:
- npm test -- --api-key $STAGING_API_KEY
rules:
- if: $CI_COMMIT_BRANCH == "main"
# Deploy staging: uses staging credentials
deploy_staging:
stage: deploy
image: alpine:latest
script:
- ./deploy.sh staging
environment:
name: staging
rules:
- if: $CI_COMMIT_BRANCH == "main"
# Deploy production: Vault for database password
deploy_production:
stage: deploy
image: alpine:latest
id_tokens:
VAULT_ID_TOKEN:
aud: https://vault.example.com
secrets:
DATABASE_PASSWORD:
vault: production/db/password@ops
token: $VAULT_ID_TOKEN
API_KEY:
vault: production/api/key@ops
token: $VAULT_ID_TOKEN
script:
- echo "Deploying to production..."
- ./deploy.sh production --password $DATABASE_PASSWORD
environment:
name: production
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: manual
# Variables needed in GitLab UI:
# Settings > CI/CD > Variables
#
# STAGING_API_KEY
# Value: sk_staging_xxx
# Masked: ✓
# Protected: ✓
# Environment scope: staging
#
# VAULT_SERVER_URL
# Value: https://vault.example.com:8200
# Masked: ✗ (URL is not secret)
# Protected: ✗
#
# VAULT_AUTH_ROLE
# Value: myproject-production
# Masked: ✗
# Protected: ✓
Troubleshooting Secrets Management
# Masked variable not masking in logs
# - Check value is at least 8 characters
# - Check value has no newlines
# - Check variable is actually masked (UI shows mask icon)
# - Check job log for [MASKED] placeholder
# Protected variable empty in job
# - Check branch is protected (Settings > Repository)
# - Check job runs on protected branch/tag
# - Check variable is protected (UI shows shield icon)
# - Check environment scope if set
# Vault secret not available
# - Check VAULT_SERVER_URL is set
# - Check VAULT_AUTH_ROLE is correct
# - Check ID token audience matches Vault config
# - Check Vault role bound claims match job claims
# - Check Vault policy allows read on the path
# - Check secret path and engine are correct
# Vault authentication fails
# - Verify Vault has jwt auth enabled
# - Verify oidc_discovery_url matches GitLab instance
# - Verify bound_issuer matches
# - Check token audience (aud) in id_tokens block
# - Check role name matches VAULT_AUTH_ROLE
# GitLab Secrets Manager not working
# - Check Secrets Manager is enabled for project/group
# - Check secret exists with correct name
# - Check job's branch and environment match secret scope
# - Check you have permission to read the secret
# - Check GitLab version (19.0+ for beta)
# Secret exposed in logs despite masking
# - Check if script encodes the value (base64)
# - Check if a tool prints environment (e.g., env, printenv)
# - Check if secret is passed as command-line argument
# - Rotate the secret immediately if exposed
# - Use file: true to pass file path instead of value
# Debug: list variables (names only, not values)
script:
- env | grep -E '^[A-Z_]+=' | cut -d= -f1
# Debug: test Vault authentication
script:
- echo $VAULT_ID_TOKEN | cut -d. -f2 | base64 -d 2>/dev/null
# Decode JWT payload to see claims
# Debug: check secret availability
script:
- |
if [ -z "$DATABASE_PASSWORD" ]; then
echo "Secret not available!"
else
echo "Secret available"
fi
Frequently Asked Questions
What's the difference between masked and protected variables?
Masked hides the value in job logs (shows [MASKED]). Protected restricts the variable to only protected branches and tags. Use both for production secrets.
Are CI/CD variables secure enough for production?
Not on their own. Variables are stored in GitLab and accessible to users with settings access. For production credentials, use a secrets manager (Vault, GitLab Secrets Manager, or cloud provider) with masking and protection as a minimum.
How do I prevent feature branches from accessing production secrets?
Use protected variables. They are only available to jobs running on protected branches (like main). Feature branch pipelines cannot access them.
Can I use environment-scoped variables with a shared key?
No. Environment-scoped variables must be unique per environment. You cannot define the same key twice with different environment scopes in the UI. Use different keys or environment-specific config files.
What is GitLab Secrets Manager and is it available?
It's a native GitLab feature (Beta, GitLab 19.0+) built on OpenBao. It scopes secrets to specific jobs, environments, and branches. As of August 2026, it's a paid add-on with limited availability.
How does Vault authentication work in GitLab?
GitLab generates an ID token (JWT) for the job. The job presents it to Vault. Vault verifies the token against GitLab's OIDC discovery URL and checks the claims against a configured role. If valid, Vault returns a short-lived token for accessing secrets.
Can I use AWS Secrets Manager instead of Vault?
Yes. GitLab supports AWS Secrets Manager, Azure Key Vault, and GCP Secret Manager natively. Use the aws_secrets_manager, azure_key_vault, or gcp_secret_manager keyword in your job definition [citation:6].
What should I do if a secret is exposed in logs?
Rotate the secret immediately. Even if masking was enabled, assume the value is compromised. Then investigate how it was exposed and fix the script or configuration to prevent recurrence.
Secrets are the keys to your infrastructure. Treat them with the care they deserve — mask, protect, scope, and rotate.