Skip to content

Manual Test Runbook — F11: Module Versioning + Private Registry

Owner: Sagar  |  Time: ~6 min (Parts A + B offline) / +5 min (optional Part C live release)  |  Sandbox: none required (git + metadata only)

Overview

F11 versions the SnowOps module library and publishes it as a private registry: each module is a git tag (<module>/v<version>) recorded in modules/registry.json, with a per-module CHANGELOG.md. Consumers pin a tag in their source. Three pieces:

  • apps/module-registry/ — the TS tool. Pure core (validate / index / release plan / pin audit) behind a Collector seam (FsRegistryCollector reads the manifest + CHANGELOGs + git tag; FixtureCollector for tests).
  • modules/registry.json + per-module CHANGELOG.md — the source of truth.
  • .github/workflows/module-release.yml — on merge to main, validates then tags + releases every module whose manifest version isn't tagged yet.

Convention + pin strategy: docs/conventions/module-versioning.md.

Parts A + B verify validation, index-building, release planning, and the pin audit offline. Part C optionally exercises a real tag + GitHub Release.

Part A — Tool + registry validation (offline, ~4 min)

A1. Build + typecheck + unit tests

cd apps/module-registry
npm ci
npm run typecheck
npm run build
npm test

Expect: 3 suites, 27 tests pass — semver ordering, validation (CHANGELOG sync, duplicate name, bad semver, version regression), release planning, the consumer pin audit (unpinned / ref-mismatch / unknown-version), the renderer, and a guard that loads the real committed modules/registry.json + every CHANGELOG and asserts the registry is internally valid and in sync.

A2. Validate the live registry + build the index

cd ../..
node apps/module-registry/dist/index.js \
  --manifest modules/registry.json \
  --out-dir /tmp/f11 --now 2026-06-01T00:00:00.000Z --fail-on-issues true
echo "exit=$?"
cat /tmp/f11/registry-index.json | node -e "const i=require('/dev/stdin');console.log(i.modules.map(m=>m.name+' -> '+m.tag).join('\n'))" 2>/dev/null \
  || node -e "const i=require('/tmp/f11/registry-index.json');console.log(i.modules.map(m=>m.name+' -> '+m.tag).join('\n'))"

Expect: stderr registry: OK — 10 modules, 0 validation error(s), 0 pin finding(s), 10 to release; exit 0. The index lists 10 modules with tags contracts/v0.1.0, baseline/v1.0.0, … audit-log-archive/v1.0.0. (All 10 are "to release" until the first tags exist — see Part C.)

A3. A broken registry is caught

Temporarily desync a CHANGELOG and re-run:

node -e "const fs=require('fs');const p='modules/registry.json';const m=JSON.parse(fs.readFileSync(p));m.modules.find(x=>x.name==='acr').version='2.0.0';fs.writeFileSync('/tmp/bad-registry.json',JSON.stringify(m,null,2))"
node apps/module-registry/dist/index.js --manifest /tmp/bad-registry.json --repo-root . \
  --out-dir /tmp/f11bad --fail-on-issues true; echo "exit=$?"

Expect: exit 2; stderr shows ✗ [changelog-out-of-sync] acr: CHANGELOG top version '1.0.0' != manifest version '2.0.0'. (No repo files were modified — the edit went to /tmp.)

Part B — Consumer pin audit (offline, ~1 min)

B1. A correctly-pinned consumer passes

node apps/module-registry/dist/index.js \
  --consumer-dir apps/module-registry/examples/consumer-pinned \
  --out-dir /tmp/f11p --fail-on-issues true; echo "exit=$?"

Expect: exit 0; registry-report.json pinAudit.checked == 2, findings == [].

B2. An unpinned / stale consumer fails with remediation

node apps/module-registry/dist/index.js \
  --consumer-dir apps/module-registry/examples/consumer-unpinned \
  --out-dir /tmp/f11u --fail-on-issues true; echo "exit=$?"

Expect: exit 2; two findings — [unpinned] 'modules/azure/baseline' … (pin it to baseline/v1.0.0) and [unknown-version] ref 'acr/v9.9.9' ….

Part C — Live release (optional, ~5 min)

Exercises the real tag + GitHub Release path. Safe — it only creates git tags and Releases, no cloud resources. Requires push rights + gh auth.

C1. Dry-run the release workflow

In the GitHub UI: Actions → module-release → Run workflow with dry_run = true. Expect the run to validate the registry and print the release plan (summary.md artifact) without creating tags.

C2. Publish (merge path)

Merge a PR that bumps a module's version in modules/registry.json + adds the matching CHANGELOG section. On merge to main, the workflow creates <module>/vX.Y.Z and a GitHub Release with the CHANGELOG body. Re-running is idempotent — already-tagged modules are skipped.

git fetch --tags
git tag --list "*/v*"          # the published module tags

Expect: the bumped module's tag present; gh release list shows its Release.

Sign-off

Field Value
Result ☐ PASS   ☐ FAIL
Run by
Date
Commit
Notes