Docker in CI/CD Pipelines

What this article covers

• What Docker brings to CI/CD.
• GitHub Actions and GitLab CI for Docker builds.
• Multi-stage builds and caching strategies.
• Testing, scanning, and deployment in pipelines.
• Tips for local and self-hosted runners.

Introduction: Docker in CI/CD Pipelines

Continuous Integration and Continuous Delivery (CI/CD) automate the build, test, and release of software. Docker is particularly well suited for this because builds are reproducible and the same environment can be used for testing, staging, and production. By automating Docker image builds, testing, and pushes to a registry on every commit, you save time and reduce errors.

This article shows how to integrate Docker into GitHub Actions and GitLab CI, and what to watch out for when running self-hosted runners.

Key terms

• CI: Continuous Integration, regular merging and testing.
• CD: Continuous Delivery or Deployment, automated release.
• Pipeline: A sequence of steps.
• Runner: A machine that executes the pipeline.
• Multi-stage build: A Dockerfile with multiple build stages.
• Layer caching: Caching of Docker layers.
• Registry: Storage location for images.
• Artifact: An intermediate product of a pipeline.

Why Docker in CI/CD?

• Same environment for build, test, and production.
• Reproducible builds.
• Easy scaling via runners.
• Fast rollbacks through versioned images.
• Dependency isolation.
• Simple deployment across different environments.

GitHub Actions example

name: Docker Build and Push

on:
  push:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

GitLab CI example

stages:
  - build
  - test
  - deploy

variables:
  IMAGE_NAME: $CI_REGISTRY_IMAGE:latest

build_image:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  script:
    - docker build -t $IMAGE_NAME .
    - docker push $IMAGE_NAME
  only:
    - main

test_image:
  stage: test
  image: $IMAGE_NAME
  script:
    - pytest

deploy_image:
  stage: deploy
  image: docker:24
  script:
    - docker pull $IMAGE_NAME
    - docker run -d -p 8080:8080 $IMAGE_NAME

Multi-stage builds

Multi-stage builds reduce image size and improve security:

FROM python:3.11 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY . .
CMD ["python", "main.py"]

Testing in Docker

- name: Test in Docker
  run: |
    docker build -t myapp:test .
    docker run --rm myapp:test pytest

Image scanning

Find vulnerabilities in your image:

- name: Scan image
  uses: anchore/scan-action@v3
  with:
    image: "ghcr.io/user/repo:latest"

Caching

Caching significantly speeds up builds:

• Use cache-from and cache-to in GitHub Actions.
• Docker layer caching with BuildKit.
• Registry as a cache backend.
• Local cache directories on self-hosted runners.

Self-hosted runners

For larger workloads or private infrastructure, self-hosted runners are a good fit:

# Download and configure GitHub Actions Runner
./config.sh --url https://github.com/USER/REPO --token TOKEN
./run.sh

Docker must be installed on the runner. For GPU builds, you’ll need the appropriate Container Toolkit.

Deployment

Images can be deployed automatically after the build:

• To a Docker host via SSH.
• Into a Kubernetes cluster.
• Via Docker Swarm.
• To your own server with docker compose up -d.

Tips

• Don’t store secrets in images.
• Generate tags from Git hashes or timestamps.
• Fail fast on failing tests.
• Enable layer caching.
• Update images regularly.
• Avoid privileged runners when possible.
• Keep your build context small.

Common pitfalls

• Large images: Not using multi-stage builds.
• Missing registry login: Push fails.
• Slow builds: No caching enabled.
• Secrets in environment variables: Exposed in logs.
• Runner without Docker: Pipeline fails.
• Wrong tags: Overwrite or lost images.
• Oversized build context: Use .dockerignore.

Further reading and resources

• BotServ.de Docker Registry
• BotServ.de Docker Security
• BotServ.de Docker Backup
• BotServ.de Kubernetes

FAQ: Docker in CI/CD

Do I need GitHub Actions or GitLab? Both work. Self-hosted solutions like Jenkins or Woodpecker are also options.

Should I build images on every commit? Yes for main or tagged releases, optional for intermediate commits.

How fast are builds? With caching, usually minutes instead of several minutes each time.

Can I run GPU tests in CI/CD? Only with runners that have a GPU available.

Are self-hosted runners secure? If isolated and maintained properly. Never use them in public repos without proper safeguards.

Sources and further reading

• GitHub Actions: https://docs.github.com/en/actions
• GitLab CI: https://docs.gitlab.com/ee/ci/
• Docker BuildKit Caching: https://docs.docker.com/build/cache/
• Docker Build-Push Action: https://github.com/docker/build-push-action

Summary: Docker in CI/CD Pipelines

Docker fits naturally into CI/CD workflows. GitHub Actions and GitLab CI enable automated builds, tests, and deployments of container images. Multi-stage builds, caching, and regular scanning improve both speed and security. Self-hosted runners offer flexibility for specialized hardware like GPUs. By protecting secrets, leveraging layer caching, and using clean tags, you can build a robust container workflow.