Provisioning a WordPress dev site

The design — the two postures, the seed overlay, the three promotion lanes — is Dev site and promotion. This runbook is the one-time provisioning that has to happen before any of it works, in the order it has to happen.

Worked here against tourdeworcester.co.za; the shape is the same for any event site.

Preconditions

Why it matters

A --dev image is published

scripts/build-image.sh <tag> --dev builds --target runtime-dev and tags <tag>-dev. A mode: mutable release running an ordinary image has no /opt/wp-seed, so the PVCs mount over the baked plugins and the site comes up looking empty with no error.

The chart version supporting mode is published

0.2.0 or later. Point only the dev Application at it; moving production’s targetRevision takes the live site down for 1-3 minutes.

A namespace that is not production’s

wordpress-dev, and fullnameOverride different from production’s. Two releases sharing a fullname collide on every resource they create.

1. Database

Its own schema and user. Never production’s — see the warning at the end of this page.

CREATE DATABASE wp_tourdeworcester_dev
  CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci;
CREATE USER 'tdw_dev'@'%' IDENTIFIED BY '<set in step 2>';
GRANT ALL PRIVILEGES ON wp_tourdeworcester_dev.* TO 'tdw_dev'@'%';

Leave the password until step 2 — it is generated there, not chosen here.

This is the shared production MySQL cluster. Keep the dev pod small and do not raise its replica count: a max_connections exhaustion on this server has already been traced to dev and stage connection bloat.

2. Secrets

scripts/create-dev-secret.sh generates everything on your machine and writes it straight into the cluster. No secret is printed, so none reaches shell history, a terminal scrollback or an agent transcript.

scripts/create-dev-secret.sh --namespace wordpress-dev --name wp-tourdeworcester-dev

It creates three Secrets and copies a fourth:

Secret Contents

wp-tourdeworcester-dev

DB_PASSWORD, the eight WordPress salts, WP_ADMIN_USER / WP_ADMIN_PASSWORD / WP_ADMIN_EMAIL.

wp-tourdeworcester-dev-basicauth

The auth file the nginx ingress reads. Its only job is to gate the site at the edge, so WordPress never sees an unauthenticated request. Format is an apr1 hash, which is one-way.

wp-tourdeworcester-dev-basicauth-plain

The same username and password in readable form. It exists precisely because the hash above is one-way: without it the password would be unrecoverable the moment the terminal closed.

christhonie-docker

The image pull secret, copied from the production namespace. Not generated — copied, because it is the same registry credential.
The pull secret is the easiest one to forget

A fresh namespace has none, and the symptom is ImagePullBackOff with nothing pointing at a secret as the cause. The script copies it; if the source namespace is not readable it says so and prints the create secret docker-registry command instead.

What the basic-auth secret is protecting against

Three things, and the third is the one that changes over the life of the site:

  1. Discovery. A dev copy of a live store on a public hostname is findable, and a visitor cannot tell it from the real one.

  2. Indexing. blog_public = 0 and the X-Robots-Tag header ask crawlers not to index it; basic auth means a crawler never gets a page to consider in the first place.

  3. Personal data. The moment a production content refresh is pulled in, the site holds real customers, real orders and real addresses. That is when edge auth stops being tidiness and starts being the control that matters — paired with the scrub, which handles what is inside the site.

Basic auth covers the whole host, including machine callers

A server-to-server call has no credentials to offer and gets a 401. That silently breaks the event payment plugin’s inbound order-create endpoint — which is most of the reason for running this site in the cluster rather than on a laptop.

ingress.unauthenticatedPaths renders a second Ingress, for the same host, carrying no auth annotations. nginx merges rules across Ingress objects and the more specific path wins, so listing a path exempts exactly that path:

unauthenticatedPaths:
  - /wp-json/payment-api/

Keep the list as narrow as the caller needs — anything on it is reachable by anyone who knows the URL. Never put /wp-admin or /wp-login.php on it.

Re-running is safe: without --rotate it leaves existing secrets alone, because regenerating the salts logs every session out and regenerating DB_PASSWORD breaks the site until the grant is changed to match.

Then set the database grant to the generated password:

kubectl -n wordpress-dev get secret wp-tourdeworcester-dev \
  -o jsonpath='{.data.DB_PASSWORD}' | base64 -d; echo
ALTER USER 'tdw_dev'@'%' IDENTIFIED BY '<the value above>';
FLUSH PRIVILEGES;

Retrieving the logins later:

# WordPress admin
kubectl -n wordpress-dev get secret wp-tourdeworcester-dev \
  -o jsonpath='{.data.WP_ADMIN_PASSWORD}' | base64 -d; echo

# Ingress basic-auth
kubectl -n wordpress-dev get secret wp-tourdeworcester-dev-basicauth-plain \
  -o jsonpath='{.data.password}' | base64 -d; echo

3. DNS and TLS

Put the dev host in a zone external-dns owns, and the record and certificate both appear without intervention. For Tour de Worcester that is tdw-dev.myriadevents.co.za, not a subdomain of tourdeworcester.co.za.

That choice is not cosmetic. The tourdeworcester.co.za zone is manually managed, so a host there needs a hand-created A record before the first sync — otherwise cert-manager’s HTTP-01 challenge cannot resolve, the certificate never issues, and the site answers on plain HTTP or not at all. Picking an external-dns zone removes the step and the failure mode with it.

4. Sync

Add the Application to the ops repo and sync it. Then watch the bootstrap, which is where provisioning mistakes actually surface:

kubectl -n wordpress-dev logs deploy/wp-tourdeworcester-dev -c bootstrap
What you see What it means

=⇒ Seeding plugins/themes from the image

Correct. The seed overlay found /opt/wp-seed.

WARN: mode=mutable but /opt/wp-seed is missing

The image was not built with --target runtime-dev. The site will come up with no plugins and no theme. Rebuild with --dev and bump image.tag.

db not ready (attempt N) repeatedly

The grant in step 2 does not match the Secret, or the schema does not exist.

=⇒ Installing WordPress core on a later start

The schema was emptied, or the release was pointed at a different database.

=⇒ Recording harvest baseline

First start, or the declared config changed. Both are expected; see below.

5. Mail

Nothing to provision. The dev site sends to GreenMail in the event-dev namespace (greenmail.event-dev.svc.cluster.local:25), which has auth and TLS disabled, so there is no credential and nothing in the Secret.

Read what the site sent in SnappyMail at https://event-mail-dev.idealogic.co.za.

SnappyMail is configured per sender domain, and its list covers greenmail, example.com, test.com, localhost, acme.com and idealogic.co.za — not myriadevents.co.za. That is why the dev sender is an idealogic.co.za address: using the production address would send perfectly well and then be unreadable in the webmail. Adding the domain to the SnappyMail ConfigMap’s domain loop is the alternative.

Confirm the transport works end to end:

kubectl -n wordpress-dev exec deploy/wp-tourdeworcester-dev -c wordpress -- \
  wp --path=/var/www/html/web/wp eval \
  'var_dump(wp_mail("[email protected]", "dev site check", "hello"));'

false means no transport. true with nothing in SnappyMail usually means the sender address is not mapped — see FluentSMTP, which also covers the two other delivery routes (Microsoft 365 Graph and Amazon SES) and when to choose them over plain SMTP.

6. Post-sync, in wp-admin

  1. UpdraftPlus — activate the Migrator licence, and only that. Do not perform the Premium connect on a dev site: that consumes a Premium site slot, whereas the Migrator add-on’s licence explicitly does not. See wordpress-hosting:plugins/updraftplus.adoc#content-lane.

  2. PayGate — test mode. Never live credentials on a site that is not taking real money.

  3. Confirm the payment plugin points at stage, not production:

    kubectl -n wordpress-dev exec deploy/wp-tourdeworcester-dev -c wordpress -- \
  wp --path=/var/www/html/web/wp option list --search='epa_*'

    epa_admin_api_url is a base URL and carries no /api suffix — the plugin appends the path itself, so a suffix produces /api/api/… and a 404.

7. Verify the posture actually took

Three checks, each catching a different silent failure:

# Plugins are writable — this is what makes it a dev site
kubectl -n wordpress-dev exec deploy/wp-tourdeworcester-dev -c wordpress -- \
  sh -c 'touch /var/www/html/web/app/plugins/.writetest && echo writable && rm /var/www/html/web/app/plugins/.writetest'

# Core is NOT writable — this is what keeps composer.lock authoritative
kubectl -n wordpress-dev exec deploy/wp-tourdeworcester-dev -c wordpress -- \
  sh -c 'touch /var/www/html/web/wp/.writetest 2>&1 || echo "core read-only, correct"'

# The site is not crawlable and not publicly readable
curl -sI https://tdw-dev.myriadevents.co.za/ | head -1          # expect 401
curl -sI -u <user>:<pass> https://tdw-dev.myriadevents.co.za/ | grep -i x-robots-tag

Finally, confirm a restart does not wipe an unpromoted change: install anything from wp-admin, kubectl rollout restart, and check it survived. If it did not, the plugins PVC is not mounted and you are looking at an emptyDir — check persistence.enabled.

The failure this whole runbook is shaped around

The archived wordpress-wpca-test Application crashlooped for 45 days — 12,671 restarts — because its PVC was empty while the database it pointed at already held a complete WordPress install. Every restart ran a fresh wp core install against a populated schema, exited 1, and opened another connection to the shared production MySQL on the way.

A dev site gets its own schema. Not production’s, and not another environment’s.

Related