Which Test Suites Actually Run

Two pull requests can each be green in isolation and produce a red develop when both land. Git merges them without conflict because they touch no common file; the semantic collision only exists in the merged result.

The pull_request event builds base merged with branch, so the check does catch this — but only for collisions that already existed when the branch was last built. The required check is deliberately non-strict, meaning a pull request is not forced to rebuild after the base moves, so the second of two concurrently-open pull requests can merge on the strength of a merge commit computed before the first one landed. The reasoning behind that choice is in Repository Rules.

The shape, constructed:

If A merges while B’s check is already green, nothing recomputes B’s merge commit before B merges in turn.

The blast radius is larger than a red suite: push-dev chains test, package, docker, helm and deploy, so a failing test starves the dev environment of builds entirely for as long as develop stays red.

Two habits contain it:

A build can fail with zero test failures:

This means target/jacoco-it.exec is truncated, not that a test failed. It happens when an earlier Maven run on the same working tree was killed mid-write — a session restart, a Ctrl-C, an IDE restart — leaving a partial exec file that the next run appends to.

Diagnose by counting real failures in the log; if the count is zero, nothing failed:

Fix by clearing the exec files and re-running:

Always re-run from clean before reporting a red build caused this way.

Strip ANSI escapes first or greps silently miss lines:

The two summary lines are the Tests run: N, Failures: … lines with no - in <Class> suffix. The failsafe summary is emitted at [WARNING] level whenever anything is skipped, so a grep anchored on ^\[INFO\] finds only the surefire total and silently loses the integration-test total.

A test can execute, pass, and still assert nothing about the behaviour it appears to cover. The recurring shape in admin-service is a controller test that constructs the controller directly and mocks the service it delegates to: the mock’s stubbed return makes the test green while saying nothing about how the service selects among candidates. It looks like coverage and is not, which is how selection regressions reach develop unnoticed.

Behaviour that lives in a service — instance selection, filtering, ordering — belongs in a database-backed integration test annotated @IntegrationTest, not in a controller unit test.

The front-end test check is not proof a suite passed. The shared front-end test workflow in the parent-pom repository exposes a run-tests input that defaults to false, alongside a step that emits a "front-end tests were skipped" workflow warning so the gap is legible in the run summary. Until a consumer explicitly passes run-tests: true, the job installs dependencies and builds without executing a test.

Check whether the caller opted in before citing that check as evidence. Repositories are opted in individually as their suites go green; the default flips once the last one has.

Do not grep .github/workflows/ for ng test and conclude there is no test step. The call is a reusable-workflow reference:

Follow reusable-workflow references before making any claim about what CI runs. Note also that the parent-pom repository commits workflow files straight to main and consumers reference @main, so a change there goes live for every consumer at once — see CI and Release Pipeline Traps.

Many of the generated specs were never repaired after their components were hand-customised, so a substantial fraction of suites fail on develop. A red run therefore proves nothing about your change until you have compared it against the branch baseline:

Compare suite counts and test counts. Both matter, because of the next trap.

A spec file with a compile error is reported as one failed suite and zero tests run. A whole file of tests can therefore be silently dead while the summary looks like a single ordinary failure — for example a spec using a type it never imported, whose tests had never executed since the day they were written.

When the suite-count delta and the test-count delta disagree, a file is dead rather than failing. Fix the compile error before reading anything else in the run.

jest.conf.js in membership-ui declares no transform and no preset. The TypeScript transform is injected by @angular-builders/jest from the angular.json configuration, so the config file is only half a configuration on its own.

Run tests as:

Running npx jest --config jest.conf.js directly fails misleadingly, in two stages: first a Babel version clash, because the Babel preset chain wants a newer @babel/core than the Angular version pins, and then TypeScript syntax errors such as "Missing initializer in const declaration", because babel-jest has been handed TypeScript with no TypeScript handling. Neither is a real dependency problem. Do not add package.json overrides to chase them — the lockfile is correct and the invocation is wrong.

Note also that npm test runs a pretest step of full-repository ESLint, which has pre-existing errors. Use npx ng test directly for suites, and compare ESLint findings against develop with the same stash technique rather than expecting a clean run.

describe, it and expect are ambient globals supplied by @types/jest. A spec that does this:

fails to resolve the module, and the whole suite is reported as failed to run rather than as one bad assertion. Use jest.fn() and jest.Mock; never vi.*.

ng build excludes specs — they are compiled under a separate tsconfig.spec.json. A spec can be badly broken while ng build passes cleanly. Verify specs with the Jest run, separately, every time.

The front-end test job is gated by dorny/paths-filter (see GitHub Actions CI/CD). A pull request that touches none of the matched paths skips the job entirely, so a broken spec — a vitest import, a stale assertion — can merge unrun and surface only on a later pull request that happens to re-trigger the job. The later author then inherits several failing suites they did not write.

Two mitigations: run the suite locally on any pull request that adds or edits specs regardless of what CI decides, and treat a skipped front-end check as unknown, never as a pass.

Where a spec pins UI structure — a hardcoded navigation order, for instance — the same array is often asserted in more than one spec file. Adding or reordering such a row requires updating every spec that pins it, not just the first one the search turns up.

Every plugin class opens with the standard WordPress guard:

Outside WordPress, ABSPATH is undefined, so the first time PHPUnit autoloads any plugin class the guard calls exit(). That terminates PHP before PHPUnit prints its results, and the process ends with status 0 — indistinguishable from a clean pass to anything reading the exit code.

The fix is a tests/bootstrap.php that defines ABSPATH before requiring the autoloader — the order is the whole point — wired in via phpunit.xml. Add a minimal get_option() stub returning the caller’s supplied default; loading real WordPress would require a database.

Do not trust the exit code. Make PHPUnit write a JUnit log and fail the job when the log is missing or reports fewer than one test:

Two packaging details when wiring this up: the common PHPUnit action writes no JUnit log unless explicitly asked, and the companion Composer action validates its dev input as the literal string "yes"/"no" and rejects a YAML boolean.

Finally, check when the plugin’s tests run. A release workflow that fires only when a pull request closes into main and packages without running tests is not a test gate at all; the tests need their own workflow on pull-request open.