Skip to content

Manual Test Runbook — H5: Service Principal Inventory + Rotation

Owner: Sagar  |  Time: ~5 min (Parts A + B, offline) · +20 min Part C (live tenant read + PR)  |  Sandbox: snowops-sandbox-tenant-01

Promotes H5 (apps/sp-inventory/ + .github/workflows/sp-inventory-rotation.yml) from 🟦 Code Complete → 🟩 Shipped. Parts A + B are 100% offline. Part C reads a real tenant's app registrations and proves the rotation PR opens.

Prerequisites

  • Local tooling: node >= 20.10, npm, az CLI >= 2.50, jq, gh (Part C)
  • (Part C) An OIDC application with Application.Read.All application permission granted + admin-consented (read-only — no write scope)
  • (Part C) A sandbox GitHub repo with the AZURE_CLIENT_ID / AZURE_TENANT_ID / AZURE_SUBSCRIPTION_ID secrets set, and the H5 caller workflow installed from templates/client-repo/
  • Working directory: repo root

Steps

Part A — typecheck + unit tests (offline, ~3 min)

  1. Install, typecheck, build:
cd apps/sp-inventory
npm ci
npx tsc -p tsconfig.json --noEmit
npx tsc -p tsconfig.test.json --noEmit
npm run build

Expected: clean (no TS errors).

  1. Run the unit suite:
npm test

Expected: 2 suites, 19 tests pass (inventory.test.ts + report.test.ts), including the "seed a stale SP → flagged → PR warranted" path and the federated-OIDC-only SP (zero credentials → never stale) path.

  1. Smoke-run the CLI against a seeded fixture (no cloud):
cat > /tmp/sp-apps.json <<'JSON'
[
  {"id":"o1","appId":"a1","displayName":"legacy-deploy-sp","credentials":[
    {"keyId":"00000000-0000-0000-0000-0000000000ab","type":"password","displayName":"ci-secret",
     "startDateTime":"2025-12-01T00:00:00Z","endDateTime":"2026-12-01T00:00:00Z"}]},
  {"id":"o2","appId":"a2","displayName":"oidc-build-sp","credentials":[]}
]
JSON
node dist/index.js --input /tmp/sp-apps.json --threshold-days 90 \
  --now 2026-05-29T00:00:00Z --out-dir /tmp/sp-out
cat /tmp/sp-out/pr-body.md

Expected: should_open_pr=true; the PR body lists legacy-deploy-sp with a checkbox; oidc-build-sp (no credentials) is NOT flagged.

Part B — workflow lint + pre-commit (offline, ~2 min)

  1. Both workflows parse + lint:
ruby -ryaml -e "YAML.load_file('.github/workflows/sp-inventory-rotation.yml')"
ruby -ryaml -e "YAML.load_file('templates/client-repo/.github/workflows/sp-inventory-rotation.yml')"
actionlint .github/workflows/sp-inventory-rotation.yml \
  templates/client-repo/.github/workflows/sp-inventory-rotation.yml   # if installed
  1. pre-commit on the touched files:
pre-commit run --files \
  apps/sp-inventory/src/*.ts \
  .github/workflows/sp-inventory-rotation.yml \
  templates/client-repo/.github/workflows/sp-inventory-rotation.yml

Expected: all hooks PASS.

Part C — live tenant read + rotation PR (~20 min)

A Graph-issued secret's startDateTime is always "now", so you can't create a genuinely 90-day-old secret on demand. Lower the threshold to 1 (or 0) so a freshly-created secret is flagged — this exercises the exact same stale-detection + PR path.

  1. Sign in + create a throwaway app registration with a secret to seed a stale credential:
az login
APP_ID=$(az ad app create --display-name "snowops-h5-throwaway" --query appId -o tsv)
az ad app credential reset --id "$APP_ID" --append --years 1 >/dev/null
  1. Live local run against the tenant (proves Graph read + classification):
cd apps/sp-inventory
node dist/index.js --threshold-days 1 --out-dir /tmp/sp-live
jq '.stale_credentials[] | select(.appId=="'"$APP_ID"'")' /tmp/sp-live/inventory.json

Expected: the throwaway app's secret appears in stale_credentials (reason: "aged"); should_open_pr=true in stdout.

  1. Full workflow drill in the sandbox repo: dispatch the H5 caller (temporarily set threshold_days: 1 in the caller, or dispatch the reusable workflow with that input) so the seeded secret is flagged. Confirm:
  2. The run authenticates via OIDC (read-only) and reads app registrations.
  3. A PR titled 🔑 Rotate N stale service-principal credential(s) opens on the automated/sp-rotation branch.
  4. The PR commits compliance/sp-inventory/inventory.json + STALE.md.
  5. The PR body lists the throwaway SP with a rotation checklist.
  6. Re-dispatching does NOT open a duplicate PR (the branch is updated).

Pass criteria

  • Part A Step 2 — 19 unit tests pass
  • Part A Step 3 — CLI flags the seeded stale SP, ignores the OIDC-only SP
  • Part B — both workflows parse + pre-commit hooks pass
  • Part C Step 7 — live Graph read flags the throwaway secret
  • Part C Step 8 — a rotation PR opens with the inventory snapshot committed; a second run updates rather than duplicates it
  • The OIDC application held only Application.Read.All (no write scope used)

Teardown

az ad app delete --id "$APP_ID"            # remove the throwaway app registration
gh pr close automated/sp-rotation --delete-branch   # in the sandbox repo

Sign-off

  • Tester: _  |  Date: _  |  Result: PASS / FAIL / N/A
  • Notes: