Mutation-testing Ghost core's unit suite: strong overall, plus a small test PR offer

Contributing to Ghost
1

I ran a mutation-testing pass over Ghost core’s unit suite (the vitest project on
current main) and wanted to share the results here before opening anything on
GitHub, since that seems to be the preferred flow for non-bug contributions.

The method: plant small realistic defects (inverted guards, shifted boundaries,
dropped negations) one at a time and check whether the suite notices. 400 planted
defects, weighted toward well-covered code: the suite caught 356. Of the 44 misses,
26 are equivalent mutations (they change the text but provably not the behavior),
which puts the meaningful catch rate around 95 percent. That’s a strong result, but it’s worth keeping in mind this is the unit suite only, so anything the integration or e2e projects would
catch is not counted either way.

The most actionable miss is memberCountRounding in
core/frontend/utils/member-count.js. Its existing test uses mid-range values
(55, 580, 5555…), so the range thresholds are unpinned. Three examples that keep
every test green today:

  • change 10000 to 9999 and memberCountRounding(10000) returns undefined
  • change 1000000 to 999999 and memberCountRounding(1000000) returns undefined
  • change 100000 to 100001 and the format flips from 100,000+ to 100k+

I have a ~20-line test-only commit ready that pins each boundary (verified to fail
against each planted defect above and pass on clean source), written to the repo’s
commit conventions. Happy to open the PR if that’s welcome. There are also a dozen
or so smaller misses (mostly untested negations in serializer edge paths and the
members repository) I can share details on if anyone wants them.

Found while mutation-testing this repo (I’m building a tool in this space).