Dependency Update Checks

1. Overview

Every repository that references a third-party artefact — a container image, a Maven dependency, an npm package — carries a currency problem. The artefact keeps moving; our reference to it does not. This page describes the design of the checks we use to keep those references honest, and the rules for applying them to a repository.

The companion operational guide — how to review what the checks propose, when to approve it, and how to implement it — is Dependency Update Review.

This page is about third-party artefacts. Our own application images (christhonie/event-, christhonie/ems-) are advanced by the release-promotion pipeline and are deliberately excluded from these checks. See Scope Deliberately: Never Race the Pipeline.

2. The Problem: Two Failure Modes

Dependency currency is usually framed as one problem ("keep things up to date"). It is really two, and they fail in opposite directions.

Unpinned drift Frozen pin

	Unpinned drift	Frozen pin
Reference looks like	`image: some/thing:latest`	`image: some/thing@sha256:abc…` set once and forgotten
What goes wrong	The artefact changes underneath you with no commit, no review, and no record. A pod restart months later silently runs different software.	The artefact never changes, including when it should. Known vulnerabilities accumulate behind a reference that looks deliberate.
How it presents	An incident with no corresponding change in git history — the hardest kind to diagnose.	A security finding, or an upgrade that has become too large to perform safely.

Reference looks like

image: some/thing:latest

image: some/thing@sha256:abc… set once and forgotten

What goes wrong

The artefact changes underneath you with no commit, no review, and no record. A pod restart months later silently runs different software.

The artefact never changes, including when it should. Known vulnerabilities accumulate behind a reference that looks deliberate.

How it presents

An incident with no corresponding change in git history — the hardest kind to diagnose.

A security finding, or an upgrade that has become too large to perform safely.

Both are live risks in this estate. The unpinned-drift mode caused a real outage:

The 2026-08-17 bastion lockout. The SSH bastion ran lscr.io/linuxserver/openssh-server:latest with imagePullPolicy: Always. An unattended rebuild moved it onto OpenSSH 10.2, which enables PerSourcePenalties by default (introduced upstream in 9.8). Combined with NodePort source-NAT, an internet brute-force accrued penalties against the shared per-node masquerade address and locked out every legitimate user. Nothing in git changed; the behaviour arrived on its own.

The design therefore has to address both modes at once. Pinning alone converts an unpinned-drift risk into a frozen-pin risk. A pin nobody bumps is worse than no pin — it carries the appearance of deliberate control without the substance.

3. Design

3.1. Pin by Tag and Digest Together

Container images are referenced in the combined form:

image: lscr.io/linuxserver/openssh-server:10.2_p1-r0-ls229@sha256:67d4c3a1402179a6579aa217a38b52ced557eb8a0c17a8e32fe986a4549fdee4

Each half does a different job:

Component Purpose

Component	Purpose
Tag (`:10.2_p1-r0-ls229`)	Forensics and review. Records which version is running, in the manifest, readable without a registry lookup. A digest-only pin tells you nothing during an incident, and gives an update bot no version to compare against.
Digest (`@sha256:…`)	Determinism. This is what is actually pulled. Immutable, so a retag upstream cannot change what runs. When both are present, the digest governs resolution.

Tag (:10.2_p1-r0-ls229)

Forensics and review. Records which version is running, in the manifest, readable without a registry lookup. A digest-only pin tells you nothing during an incident, and gives an update bot no version to compare against.

Digest (@sha256:…)

Determinism. This is what is actually pulled. Immutable, so a retag upstream cannot change what runs. When both are present, the digest governs resolution.

Pinning by digest alone is a common instinct and should be avoided: it is deterministic but opaque, and it defeats the automation described below.

3.2. Automate the Bump, Not the Merge

The check does not update anything. It opens a pull request proposing an update, which a human assesses and merges. This is deliberate:

The GitOps repositories are wired to ArgoCD with automated.prune and automated.selfHeal. A merge to main is a deployment. There is no staging step between merge and production for infrastructure components.
The value of the check is the prompt, not the change. It converts "nobody remembered" into "somebody decided", and leaves an auditable record of the decision either way.

Approval criteria live in Dependency Update Review rather than here, because they are a judgement call that changes with exposure and release state.

3.3. Tooling: Dependabot

We use GitHub Dependabot with package-ecosystem: docker.

The important and widely-missed capability is that this ecosystem reads Kubernetes manifests, not only Dockerfiles. Dependabot includes any YAML file that parses as a mapping carrying both apiVersion and kind keys, which covers our plain manifest repositories without any restructuring.

Consideration Position

Consideration	Position
Why not Renovate?	Renovate is more capable for manifest repositories and worth revisiting if our needs grow. Dependabot is already available on these repositories with no additional service to host, credential to manage, or self-hosted runner to maintain. Start with the tool that costs nothing to adopt.
Recursion	Dependabot does not search subdirectories from a single `directory` entry. Each directory is opted in explicitly. For this estate that is a feature, not a limitation — see Scope Deliberately: Never Race the Pipeline.
Unusual version strings	Some publishers use version formats that Dependabot cannot order (LinuxServer.io’s `10.2_p1-r0-ls229` build-number suffix, for example). If a dependency goes quiet, fall back to `:latest@sha256:…`, where Dependabot reliably proposes a digest bump each time the tag moves. The version is then no longer visible in the manifest, so prefer the explicit tag where it works.

Why not Renovate?

Renovate is more capable for manifest repositories and worth revisiting if our needs grow. Dependabot is already available on these repositories with no additional service to host, credential to manage, or self-hosted runner to maintain. Start with the tool that costs nothing to adopt.

Recursion

Dependabot does not search subdirectories from a single directory entry. Each directory is opted in explicitly. For this estate that is a feature, not a limitation — see Scope Deliberately: Never Race the Pipeline.

Unusual version strings

Some publishers use version formats that Dependabot cannot order (LinuxServer.io’s 10.2_p1-r0-ls229 build-number suffix, for example). If a dependency goes quiet, fall back to :latest@sha256:…, where Dependabot reliably proposes a digest bump each time the tag moves. The version is then no longer visible in the manifest, so prefer the explicit tag where it works.

3.4. Scope Deliberately: Never Race the Pipeline

This is the rule most likely to cause damage if ignored.

The GitOps repositories mix two kinds of image reference:

Our own applications — christhonie/event-admin-service, christhonie/event-admin-ui, christhonie/ems-clone-stage. Their tags are advanced by the release-promotion pipeline, which produces commits such as chore(prod): promote admin-portal to 0.1.3-RELEASE.
Third-party images consumed as-is — the bastion sshd, busybox, postgres, sonarqube, the Grafana observability components, and similar.

A repository-wide check would open pull requests against category 1, competing with the pipeline for the same lines and potentially reopening promotions that were deliberately closed. Two safeguards apply together:

Opt directories in individually. Do not use a repository-wide glob, even where the syntax allows it. This also prevents the first run from opening a dozen pull requests at once.
Add an explicit ignore for our own images, as a second line of defence independent of directory scoping.

4. Reference Configuration

.github/dependabot.yml in the GitOps repository:

version: 2

updates:
  - package-ecosystem: "docker"
    directory: "/ssh-bastion"          (1)
    schedule:
      interval: "weekly"
      day: "monday"
      time: "07:00"
      timezone: "Africa/Johannesburg"  (2)
    open-pull-requests-limit: 5
    commit-message:
      prefix: "chore(deps)"
    labels:
      - "dependencies"
      - "ssh-bastion"
    ignore:
      - dependency-name: "christhonie/*"   (3)

1	One entry per opted-in directory. Copy the block to extend; do not glob the repository.
2	Monday morning local time, so proposals are waiting at the start of the week rather than arriving mid-change.
3	Never propose our own images, in any directory — independent of the scoping in <1>.

5. Applying This to a Repository

Establish the reference form first. Convert third-party images in the target directory to tag@digest. Verify the digest resolves to the tag you claim, and that the running workload already uses that digest, so the first change is a no-op in substance.
Add or extend .github/dependabot.yml with one docker entry for that directory.
Confirm the ignore rule for our own images is present.
Let the first scheduled run complete and confirm the proposals are the ones you expect. A pull request against an application image means the scoping is wrong — fix it before merging anything.
Record the component’s exposure tier in Dependency Update Review so reviewers know how much scrutiny it warrants.

6. Known Gaps

Coverage is partial by design. Only opted-in directories are checked. Directories carrying third-party images that have not been opted in (/sonarqube, /chatwoot, /flowise, /goalert, /wordpress, /mysql, /storage) are still subject to frozen-pin risk. They should be added one at a time.
Logs written to a file inside a container are invisible to the central stack. Alloy tails /var/log/pods, which is stdout and stderr only. A component that writes its real log to a file in its own filesystem — as the LinuxServer images do by default, via their s6 log service — will appear in Loki as a few lines of startup banner and nothing else, and a restart destroys the rest. This is not hypothetical: it is why the 2026-08-17 lockout was hard to diagnose. The bastion now sets LOG_STDOUT=true; verify log delivery when opting any new directory in, rather than assuming the collector sees everything.
Dependabot proposes; it does not verify. Nothing in this design tests the new image before it reaches production. The verification burden sits entirely on the reviewer, which is why the approval criteria are explicit.

7. Related

Dependency Update Review — the operational counterpart: cadence, approval conditions, implementation, and rollback.
ArgoCD Deployment — why a merge to main is a deployment.
GitHub Actions CI/CD — where our own images get their tags.