Loading...


What is CI/CD

CI/CD is two acronyms glued together:

  • CI (Continuous Integration): every push automatically runs lint, tests and a build. The goal is to catch broken code before it merges, instead of relying on someone remembering to run the tests.
  • CD (Continuous Delivery / Deployment): once tests pass, the artifact is shipped to production automatically. Delivery usually keeps a human button in the loop; Deployment goes all the way without one.

Chain them together and you get a pipeline: every step between a commit and a live site is written as code, executed by a machine, and produces the same result every time.

The payoff is concrete:

  1. Repeatable: the process lives in YAML, so nobody can forget a step.
  2. Fast feedback: minutes after a push you know whether the build is broken.
  3. Rollback: every release maps to an immutable image; if something breaks, switch back.
  4. A calmer server: compilation happens in CI; the VPS only runs.

The old way: five manual steps

This blog runs on a single VPS with Docker Compose managing three containers: Next.js, nginx and certbot. Shipping looked like this:

  1. Run npm run build locally to make sure it passes.
  2. git push.
  3. SSH into the VPS.
  4. git pull.
  5. docker compose up -d --build.

The pain points were obvious:

  • The VPS compiles its own code: next build eats most of a small VPS's CPU and memory, and the site slows down while it runs.
  • No test gate: forgetting to run tests still lets you ship.
  • No rollback: if the build is broken, it stays broken until you fix it and build again.
  • Humans remember steps: skip one of the five and the site is stuck on the old version, or half-way between.

How larger teams do it

At any scale the skeleton is the same; each box is just filled with heavier tooling:

StageCommon toolsIdea
CIGitHub Actions, GitLab CI, Jenkins, BuildkitePRs trigger lint / test / build
ArtifactContainer registry (GHCR, ECR, Harbor)Images tagged with the commit SHA, immutable
CDKubernetes + ArgoCD / Flux (GitOps)Change the tag in a manifest in Git; the cluster syncs itself
Environmentsdev → staging → productionProduction usually needs a human approval
Release strategyblue-green, canaryAutomatic rollback on failure
ObservabilityPrometheus / Grafana, SentryHealth checks decide whether a release succeeded

Kubernetes is far too heavy for one VPS, but the principles carry over unchanged: immutable images, SHA tags, health checks as the gate, rollback on failure. What follows is a scaled-down version of exactly that architecture, with the CD box swapped from ArgoCD to "SSH into the VPS and run docker compose".


Our pipeline

The whole pipeline lives in one workflow file with four jobs that run in sequence:

push main ──► test ──► build ──► deploy ──► promote
              lint     docker    SSH→VPS    :latest
              vitest   push      pull/up    only on
                       :sha-xxx  healthcheck  success
                                 rollback
  • PR or push to dev: only test and build run (build just proves the Dockerfile works; nothing is pushed).
  • Push to main: all four stages run and the site goes live.
  • Manual trigger with a tag: test and build are skipped and an existing image is deployed directly. That is the rollback button.

1. test: lint and unit tests

Loading...

Nothing fancy. What matters is that it sits in front of build. If tests fail, nothing after it runs.

2. build: build the image in CI, push to GHCR

Loading...

Design points:

  • The tag is the first seven characters of the commit SHA (sha-abc1234). A tag always maps to exactly one revision and is never overwritten.
  • GHCR (GitHub Container Registry) lives under the same account as the repo, so GITHUB_TOKEN can log in without any extra credentials.
  • cache-from / cache-to: type=gha stores Docker layer cache in the GitHub Actions cache. When only source files change and package.json does not, the npm ci layer is a cache hit and the build gets much shorter.
  • build-args pass the NEXT_PUBLIC_* values in. Why that is needed is explained in the next section.

3. deploy: SSH into the VPS, pull the new image, restart

Loading...

What the remote script does, step by step:

  1. git fetch + git merge --ff-only: docker-compose.yml and the nginx templates are bind-mounted from the git checkout on the VPS, so it still has to be updated. The fetch goes over HTTPS with the job's GITHUB_TOKEN, so the deploy user on the VPS needs no GitHub SSH key. --ff-only means that if someone hand-edited a file on the server, the deploy fails loudly instead of creating a merge commit.
  2. Log in to GHCR with the workflow's GITHUB_TOKEN. It is only valid while this job runs, so no long-lived PAT has to live on the server.
  3. docker compose pull fetches only the nextjs image.
  4. docker compose up -d --wait recreates the container and waits for the healthcheck to turn healthy. A timeout counts as a failed deploy.
  5. nginx reload: nginx resolves proxy_pass http://nextjs:3000 to a container IP at startup. After the container is recreated the IP may change, so one graceful reload makes it resolve again, with no dropped connections.
  6. Smoke test: hit the homepage from outside to confirm the whole Cloudflare → nginx → Next.js path works.
  7. Any failure calls rollback, which pulls :latest and restarts.

rollback is a shell function inside the remote script:

Loading...

The healthcheck itself is defined in docker-compose.yml:

Loading...

The Alpine Node image has no curl, so it uses the fetch built into Node 20 to hit the homepage.

4. promote: point :latest at the image only after a successful deploy

Loading...

This is the decision in the pipeline most worth explaining.

The intuitive approach is to tag the image with both :sha and :latest at build time. But then, if the new version fails to deploy, :latest already points at the broken image and there is nothing to roll back to.

So :latest gets a different meaning here: the last image that deployed successfully. Only after the deploy job is green does the promote job use imagetools create to re-point :latest at this run's SHA tag. Nothing is rebuilt; it only rewrites the manifest tag on the registry and takes a few seconds.

That gives image: ghcr.io/your-name/your-blog:${IMAGE_TAG:-latest} in docker-compose.yml a safe default: the deploy passes IMAGE_TAG for the new version, while a rollback, or anyone running docker compose up -d by hand on the VPS, always gets the last known good one.

Rollback

Two paths:

  • Automatic: if any step of the deploy script fails, it immediately pulls :latest and restarts.
  • Manual: on the GitHub Actions page, click Run workflow and enter the tag to go back to (for example sha-abc1234). test and build are skipped, that image is deployed directly, and on success it is promoted to the new :latest.

Design principles: five rules

Before going through the settings one by one, here are the five principles behind the pipeline. Every setting below maps back to one of them.

  1. Immutable artifacts: one commit, one image, tagged with the SHA and never rebuilt. Deploying and rolling back are both just "switch the tag".
  2. Build-time and run-time are separate: anything baked into the bundle (NEXT_PUBLIC_*) is supplied by CI; anything only the server needs is supplied on the VPS. They never mix.
  3. Health checks are the gate: if the container does not turn healthy or the homepage does not respond, the deploy did not succeed. A machine decides, not a human eyeballing the site.
  4. Least privilege, short-lived credentials: every GitHub operation in CI uses GITHUB_TOKEN, which dies with the job; the VPS holds exactly one SSH key that can only log in as the deploy user.
  5. Fail closed: whatever step fails, the outcome is "the site stays on the old version". :latest only moves after success, so even a manual docker compose up -d gets a good version.

The workflow, key by key

Top level

KeyValueMeaning
namePipelineName shown on the Actions page
on.push.branches[main, dev]Only pushes to these two branches trigger a run; other branches do not, saving minutes
on.pull_request.branches[main, dev]PRs targeting these branches trigger a run; PRs only run test + build
on.workflow_dispatch.inputs.image_tagstring, default emptyThe input box for manual runs. Filling it in means rollback mode
permissionscontents: read, packages: writeNarrows GITHUB_TOKEN: read the repo, push to GHCR, nothing else
env.IMAGE_NAMEghcr.io/<owner>/<repo>A constant shared by the whole workflow. GHCR requires lowercase
env.NODE_VERSION20Matches node:20-alpine in the Dockerfile
concurrency.grouppipeline-${{ github.ref }}Only one run per branch at a time
concurrency.cancel-in-progressgithub.ref != 'refs/heads/main'A new push cancels the stale run on non-main branches; main never cancels, it queues

Job level

KeyMeaning
needsDependencies. build needs test, deploy needs [test, build]. When an upstream job fails, downstream jobs are skipped by default
ifWhether the job runs at all. When an upstream job is skipped, needs.x.result is skipped, not success; conditions have to tell them apart
runs-on: ubuntu-latestA throwaway VM from GitHub, destroyed after the job
environment.name: productionMarks this as a production deploy. The Actions page keeps a history, and required reviewers can be added later for manual approval
environment.urlThe link shown once the deploy finishes
outputsStrings for downstream jobs, e.g. the tag deploy resolved, read by promote
job-level envVariables visible to every step in the job. The test job uses it to hand NEXT_PUBLIC_* to next lint
job-level concurrencydeploy also declares production-deploy with cancel-in-progress: false, so a deploy is never killed half-way

The if on deploy

Loading...
  • !cancelled(): the default success() would stop the job whenever any upstream job is skipped, but "dispatch with a tag" skips test/build on purpose, so this is used instead.
  • First path: build succeeded, and the event is a push to main or a manual run. A PR's build also succeeds, but the event is wrong, so no deploy.
  • Second path: manual, a tag was given, and both test and build were skipped by their own if. All three are required; otherwise "test failed, so build was skipped" would be mistaken for rollback mode. That is a real bug hit while writing this, covered below.

Steps and action parameters

Step / actionParameterMeaning
actions/checkout@v4Clones the repo into the runner. build needs the full context
actions/setup-node@v4node-version, cache: npmInstalls Node; cache: npm caches ~/.npm keyed by the package-lock.json hash so npm ci is fast
run: npm i -g [email protected]The runner ships npm 10, which reads the lock file differently from the npm 11 used locally and in the Dockerfile
docker/setup-buildx-action@v3Enables the BuildKit builder, required for cache-from/to and multi-platform builds
docker/login-action@v3registry, username: github.actor, password: GITHUB_TOKENLogs in to GHCR. Only runs when the run will push
docker/build-push-action@v6context: .The build context is the repo root
platforms: linux/amd64The VPS is x86. So is the runner, so no QEMU
pushOnly true on a push to main; PRs only build to validate
tagsIMAGE_NAME:sha-xxxxxxx
build-argsMap to the Dockerfile ARGs; values come from vars.*
cache-from: type=gha / cache-to: type=gha,mode=maxLayer cache stored in the Actions cache. mode=max also saves intermediate stages
appleboy/ssh-action@v1host, port, username, keySSH connection details, all from Secrets
envsAllow-list of runner environment variables to export into the remote shell
scriptThe shell script to run on the VPS
docker buildx imagetools create--tag :latest :sha-xxxRewrites only the manifest tag on the registry; nothing is downloaded or rebuilt

Expressions and contexts used

SyntaxMeaning
${{ github.sha }} / ${GITHUB_SHA::7}The triggering commit; the shell takes the first seven characters as the tag
${{ github.ref }}The full ref, such as refs/heads/main
${{ github.event_name }}push, pull_request or workflow_dispatch
${{ github.actor }}Who triggered the run; used as the GHCR login user
${{ github.repository }}owner/repo, used for the remote fetch
${{ inputs.image_tag }}The dispatch input; an empty string for other events
${{ secrets.X }} / ${{ vars.X }}Secret / plain-text configuration
${{ needs.build.result }}success / failure / skipped / cancelled
${{ steps.tag.outputs.tag }}A value a step in the same job wrote with echo "tag=..." >> "$GITHUB_OUTPUT"
secrets.VPS_PORT || 22The expression ||: an empty string counts as false, so it falls back to 22

What the Docker and Compose settings mean

Dockerfile

Loading...
  • Three stages: the deps layer is rebuilt only when package*.json changes; builder runs every time; runner only copies the output, with no devDependencies or source.
  • ARG, not ENV: an ARG exists only at build time, and is unset when not supplied. ENV X=${X} would turn a missing value into an empty string and override .env.production.
  • USER nextjs: runs as a non-root user.

The nextjs service in docker-compose.yml

KeyMeaning
image: ghcr.io/…:${IMAGE_TAG:-latest}Pulled from the registry. ${VAR:-default} is Compose interpolation; without a value it is latest
build: .Keeps the local docker compose up --build path. With both image and build, pull fetches the image and only --build builds locally
env_file: ${ENV_FILE:-.env.production}Source of run-time variables, never in the image
healthcheck.testCommand run inside the container; exit 0 means healthy
healthcheck.interval: 30sProbe every 30 seconds
healthcheck.timeout: 10sA single probe over 10 seconds counts as a failure
healthcheck.retries: 3Three consecutive failures mark the container unhealthy
healthcheck.start_period: 20sFailures in the first 20 seconds do not count, giving Next.js time to warm up
restart: alwaysRestarted automatically after a daemon restart or a crash

docker compose up -d --wait --wait-timeout 180: -d detaches, --wait blocks until every service is running or healthy, --wait-timeout gives up after 180 seconds with a non-zero exit. This is what turns the healthcheck into a deploy gate.

.dockerignore

COPY . . sends the entire context into the builder. Excluding certbot/ (certificate private keys), .git, .next, tests, docs and .github keeps the context small, the build fast, and the keys out of any layer.


The remote script, line by line

CommandMeaning
set -euExit on the first failing command; exit on any undefined variable. No pipefail, because the remote shell may be dash
trap '… docker logout …' EXITLog out of GHCR however the script ends, so the token never stays on the VPS
git fetch "https://x-access-token:${GH_TOKEN}@github.com/${GH_REPO}.git" "+main:refs/remotes/origin/main"Fetch main over HTTPS with the job token and write it to the origin/main tracking ref. The VPS needs no GitHub SSH key
git checkout -q mainMake sure we are on main
git merge --ff-only origin/mainFast-forward only. If someone edited files on the VPS by hand this fails, which is better than creating a merge commit
printf '%s' "$GH_TOKEN" | docker login … --password-stdinReads the password from stdin so it never appears in the process list
rollback() { … }Defined before any step that can fail. Prints logs, pulls :latest, restarts, reloads nginx, exit 1
IMAGE_TAG=… docker compose pull nextjsPulls only the new nextjs image. A failed pull exits before anything has changed
IMAGE_TAG=… docker compose up -d --wait … || rollbackRecreates the container and waits for healthy; a timeout rolls back
docker compose exec -T nginx nginx -s reloadnginx resolved nextjs to an IP at startup and the recreated container may have a new one; reload resolves again. -T means no TTY
curl --fail --retry 6 --retry-delay 5 --retry-all-errors "$SITE_URL/" || rollbackHits the homepage from outside, up to 6 retries 5 seconds apart, retrying on any error
docker image prune -fRemoves images no container uses, so the disk does not fill up with old SHA tags

Things to watch out for

1. NEXT_PUBLIC_* is baked in at build time

This is the easiest trap when moving a Next.js project to CI/CD. Environment variables prefixed with NEXT_PUBLIC_ are inlined into the client bundle during next build. When the build ran on the VPS, Next.js read the server's .env.production. Once the build moves to CI, that file is not in git, so CI cannot see it.

The fix is to declare ARGs in the Dockerfile's builder stage and pass them from CI via build-args:

Loading...

An ARG without a default is unset inside RUN, not an empty string, so a local docker compose up --build behaves exactly as before: Next.js still reads the .env.production in the build context.

A related rule: the values on the CI side must match the ones in .env.production on the VPS. Server Components read the server's env at runtime while the client bundle uses what CI baked in. If they differ you get hydration mismatches that are very hard to trace.

2. Separate build-time from run-time variables

KindExamplesBelongs in
build-timeNEXT_PUBLIC_*CI build-args
run-time (server side)R2 access keys, admin toggles.env.production on the VPS, injected via compose env_file

Secrets must never go into build-args. They stay in the image's build history, visible to anyone who can pull the image.

3. Keep Secrets and Variables apart

GitHub offers two kinds of repository-level settings:

  • Secrets: stored encrypted and automatically masked in logs. Put the SSH private key, host address and server path here.
  • Variables: stored in plain text, for configuration that was never secret. NEXT_PUBLIC_* values end up in the public client bundle anyway, so they belong here.

Keeping them apart means Variables are visible in workflow logs, which makes debugging easy, while Secrets show up as *** even if you accidentally echo them.

4. Use a dedicated SSH user, not root

Create a user on the VPS that only does deploys, add it to the docker group, and put only the CI key in its authorized_keys. Make that user own the repo directory, or git merge will fail on permissions. If your firewall allows it, restrict SSH to GitHub Actions' IP ranges, or go through Tailscale / Cloudflare Tunnel instead.

5. Deploys queue up and must not be cancelled

Loading...

Two pushes to main in a row means the second one waits for the first deploy to finish. If cancellation were allowed, the first docker compose pull could be killed half-way and leave the server in a half-applied state. The opposite is right for PR CI: cancel-in-progress: true lets a new push discard the stale run and save minutes.

6. .dockerignore is more than a size optimisation

COPY . . in the Dockerfile copies the entire build context into the builder stage. When builds ran on the VPS, the private keys under certbot/ were copied into an image layer this way. The final runner stage does not contain them, but the builder layer still sat in the VPS's Docker cache.

Listing certbot, .git, __tests__, cypress, docs and .github in .dockerignore shrinks the context, speeds up the upload, and closes that hole.

7. Treat manual inputs as untrusted

inputs.image_tag from workflow_dispatch ends up inside a remote shell script. Only repo collaborators can trigger it, but it still gets an allow-list regex first:

Loading...

8. Docker Compose version

docker compose up --wait --wait-timeout needs Compose v2.17 or newer. Run docker compose version on the VPS first.

9. It is not zero-downtime

While docker compose up -d recreates the nextjs container, there are a few seconds of 502 between the old container stopping and the new one turning healthy. Fine for a personal blog. True zero-downtime needs blue-green: run two containers side by side and switch the nginx upstream. That is a topic for another day.


The kinds of parameters you can pass in

"Parameters" in a pipeline come from many layers, each with its own lifetime, visibility and security level. In one table:

KindDefined inRead asGood forNotes
Secretsrepo Settings → Secrets${{ secrets.NAME }}SSH key, host, pathsmasked in logs; unavailable to PRs from forks
Variablesrepo Settings → Variables${{ vars.NAME }}non-secret config such as NEXT_PUBLIC_*plain text, visible in logs
Environment-scoped Secrets / Varsrepo Settings → Environmentssame, but only inside jobs that declare environment:different values for staging vs productionsupports required reviewers and branch restrictions
GITHUB_TOKENgenerated automatically${{ secrets.GITHUB_TOKEN }}logging in to GHCR, calling the GitHub APIone per job, expires when the job ends; scoped by permissions:
workflow_dispatch inputson.workflow_dispatch.inputs in the workflow${{ inputs.NAME }}parameters for manual runs, such as the tag to roll back touser input, validate it
env:workflow / job / step level${{ env.NAME }} or $NAME in shellconstants such as the image name or Node versionconfiguration, never secrets
Contextsprovided by GitHub${{ github.sha }}, ${{ github.ref }}, ${{ github.actor }}computing tags, branch checks, login userread-only
Job outputsa job's outputs: plus $GITHUB_OUTPUT in a step${{ needs.job.outputs.NAME }}passing values between jobs, e.g. the tag deploy resolved for promotestrings only
Docker build-argsbuild-args: of docker/build-push-actionARG in the Dockerfilepublic build-time configpersists in image history, never put secrets here
Docker build secretssecrets: of docker/build-push-actionRUN --mount=type=secret in the Dockerfilebuild-time secrets such as a private registry tokennever written to a layer
Runtime env.env.production on the VPScompose env_file: / environment:server-side secrets and config needed at run timenot in git, not in the image
Compose interpolationthe shell running compose, or a .env file${IMAGE_TAG:-latest}switching image tag, ports, domainsupports default values
ssh-action envs:the step's env: plus an envs: allow-list$NAME in the remote shellforwarding CI variables into the VPS scriptanything not listed in envs: is not forwarded

One sentence to remember it by: secrets go in Secrets or the VPS runtime env; public config goes in Variables; build-time values go through build-args; human parameters go through inputs; jobs talk through outputs.


Pitfalls hit along the way

None of the settings above were right the first time. The first push to main went red three times before it went green, and each failure is worth writing down.

1. npm ci says the lock file is out of sync

npm error `npm ci` can only install packages when your package.json and package-lock.json are in sync.
npm error Missing: @swc/[email protected] from lock file

npm ci worked fine locally. The difference was the npm version: Node 20 on the runner ships npm 10, while the local machine and the Dockerfile use npm 11. The lock file has a nested @swc/core under next-intl that declares an optional peer on @swc/helpers >=0.5.17; npm 11 treats optional peers as satisfied, npm 10 treats it as missing. The lock file was fine and so was the code; the toolchains disagreed.

Fix: the test job runs npm i -g [email protected] before npm ci, matching the Dockerfile. Lesson: align the CI toolchain with the build image, or the same lock file will be read two different ways.

2. Host key verification failed

On the second run the SSH connection succeeded and the very first command died:

==> sync repo
Host key verification failed.
fatal: Could not read from remote repository.

The origin remote on the VPS is an SSH remote ([email protected]:). It used to be operated as root, and root had a GitHub key and a known_hosts entry. The new deploy user had neither.

One option is to generate a key for deploy and register it as a deploy key on the repo, but that is another long-lived credential on the server. Instead the fetch uses the job's GITHUB_TOKEN over HTTPS:

Loading...

The refspec +main:refs/remotes/origin/main writes the fetched main into origin/main, so the following merge --ff-only origin/main did not change at all. The origin remote on the VPS stays there for humans.

3. Tests failed, yet deploy ran

The third run was stranger: lint was red, Deploy to VPS ran anyway, and docker compose pull failed with manifest unknown. Of course it did: the image had never been built.

The cause was the if on deploy. It originally said needs.build.result == 'success' || needs.build.result == 'skipped', meant to cover "build is skipped when a tag is given manually". But when test fails, build is also skipped, and the condition still holds.

The fix spells out rollback mode as three conditions: the event is a dispatch, a tag was given, and both test and build are skipped. Lesson: skipped is not success; any condition using needs.*.result has to account for skips caused by an upstream failure. Fortunately the script exited on the failed pull before touching anything, so the VPS was unaffected.

4. next lint blew up on next.config.js

Invalid next.config.js options detected:
  "images.remotePatterns[0].hostname" is missing, expected string

remotePatterns in next.config.js used process.env.NEXT_PUBLIC_R2_PUBLIC_URL directly. Locally the .env files always supply a value; CI has none, so config validation failed when next lint loaded it. As a side note, this also meant nobody could run lint on a fresh clone without creating a .env file first.

Fix: add a fallback, || 'media.example.com', consistent with how the rest of the code already does it, and also pass the NEXT_PUBLIC_* Variables into the test job. Lesson: CI has no .env files; anything that reads env in config or at build time needs a fallback or an explicit value.

5. A variable missing on the VPS

While comparing against the GitHub Variables, NEXT_PUBLIC_SITE_URL turned out to be missing from .env.production on the VPS. The site was not broken, because the code has a fallback, but this is exactly where "client bundle uses CI's value, server uses the VPS's value" would diverge. Added, both sides now match.

6. Moving the directory and the Compose project name

The repo used to live under /root, which the deploy user cannot enter. When moving it to /home/deploy/<repo>, remember that Compose derives the project name from the directory name: keep the name and it is still the same set of containers. Run docker compose up -d once after the move so the bind mounts point at the new path.

7. Small things

  • A post without a cover breaks the list page: cover is required.
  • docs/ is in .gitignore but some files in it are already tracked, so new files need git add -f.
  • gh auth login is interactive and hangs when run inside a non-interactive session.

The fourth run, after three red ones, was all green. Every push since has gone live within minutes, untouched by hand.


Closing

After the change, day-to-day shipping is one command: git push. A few minutes later the Actions page is green and the site is on the new version. If it is red, the site is still on the old version and nothing needs to be done.

Not done yet, on the list:

  • A staging environment: use GitHub Environments to add staging, deploy main there first, and promote to production after approval.
  • E2E in the pipeline: Cypress currently only runs locally; it could spin up a container after build and run a pass.
  • Zero-downtime: blue-green, or at least have nginx resolve the upstream dynamically with resolver.
  • Notifications: post deploy success / failure to Discord or Slack.

Once the process is code, each of those is just one more job in the YAML.