Manual Test Runbook — K4: Post-Incident Review (via E7)¶

Owner: Sagar | Time: ~5 min (Parts A/B offline) · +10 min (Part C live) | Cloud: none for Parts A/B · Sentinel read for Part C

Promotes K4 (apps/post-incident-review/) from 🟦 Code Complete → 🟩 Shipped. Parts A/B are offline ($0). Part C collects real closed Sentinel incidents and files post-incident-review issues through the E7 CLI — extending the proven E7 ticketing bridge (I5/R2).

Prerequisites¶

Local tooling: node >= 20, npm
(Part C only) az authenticated against a subscription with a Sentinel (J3) workspace; E7 (apps/ticket-platform/) built; tracker creds in env (e.g. GITHUB_TOKEN for --platform github)
Working directory: apps/post-incident-review

Steps¶

Part A — build, typecheck, unit tests (offline, ~3 min)¶

bash cd apps/post-incident-review npm install npm run typecheck npm test

Expected: typecheck clean; 23 tests pass across review.test.ts (normalize both Sentinel properties + flat shapes, closed+severity filter, latency metrics, dedupe), render.test.ts (real metrics, verbatim root-cause / no fabrication, action-item checklist, honest "to be completed" fallbacks), and ticket.test.ts (E7 argv construction, output parse, dry-run).

Part B — offline dry-run on the sample (~2 min, $0)¶

```bash npm run build node dist/index.js --input examples/incidents.sample.json --out-dir ./out

Expected: `dry-run — 1 of 3 closed incident(s) would get a review (min
Medium)`. The 3 sample incidents are High/Closed, Low/Closed (benign), and
Medium/Active → only the first passes (Active is excluded; Low is below the
floor). Inspect:

```bash
cat out/post-incident-reviews.json   # reviewed=1, MTTR computed from timestamps
ls out/pir-*.md                      # the rendered blameless review

(Lower the floor) Confirm the benign Low closed incident is now included:

node dist/index.js --input examples/incidents.sample.json --min-severity Low --out-dir ./out

Expected: 2 of 3 (both Closed incidents; the Active one still excluded).

Part C — live collect + file via E7 (~10 min)¶

Build E7 and confirm the CLI contract:

( cd ../ticket-platform && npm install && npm run build )
node ../ticket-platform/dist/index.js --platform dry-run --title t \
  --body "x" --dedupe-key k --output /tmp/e7.txt && cat /tmp/e7.txt

Expected: /tmp/e7.txt contains ticket_id=…, ticket_url=…, ticket_updated=… — the contract K4 parses.

Collect real closed incidents and file to a test tracker (GitHub shown):

export GITHUB_TOKEN=…   # repo-scoped
node dist/index.js \
  --resource-group <sentinel-rg> --workspace-name <law-name> \
  --subscription "$SUB_ID" \
  --platform github --repo <org>/<test-repo> \
  --ticket-cmd "node ../ticket-platform/dist/index.js" \
  --out-dir ./out

Expected: one issue per closed incident at/above the floor, labelled post-incident-review + severity:*, body = the blameless PIR with real MTTR. Re-run the same command and confirm each outcome is updated: true in out/post-incident-reviews.json (idempotent — no duplicates).

Open one issue and confirm the qualitative sections are honest "to be completed" prompts (not fabricated) when the source incident didn't carry a root cause — then complete them in-issue and re-run to confirm the update doesn't clobber your edits' identity (same dedupe key).

Pass criteria¶

Part A — typecheck clean; 23 tests pass
Part B — sample dry-run reviews 1/3 (Medium floor), 2/3 (Low floor)
Part C — a real closed incident files a review issue via E7; a second run updates it (no dup)
Generated post-incident-reviews.json + pir-*.md look correct (real metrics, no fabricated root cause)

Failure mode¶

A flood of low-value closed incidents generating review issues — mitigated by the severity floor + closed-status filter + the per-incident dedupe key (re-runs update one issue). K4 never fabricates a root cause it wasn't given; a sparse incident yields a review skeleton to fill in. Documented in the README.

Cost impact¶

$0 — pure orchestration; read-only az sentinel incident list. No resources created.

Removal path¶

Delete apps/post-incident-review/. Nothing external is provisioned.

Sign-Off¶

Field	Value
Part A (unit tests)	☐ PASS
Part B (offline dry-run)	☐ PASS
Part C (live via E7)	☐ PASS / ☐ skipped
Tester
Date
Result	☐ PASS