Skip to content

Manual Test Runbook — K5: Tabletop Exercise Pack

Owner: Sagar  |  Time: ~5 min (Part A offline) · +60 min (Part B = facilitate one tabletop)  |  Cloud: none — K5 is a discussion-based doc asset, no live phase

Promotes K5 (docs/runbooks/tabletop/) from 🟦 Code Complete → 🟩 Shipped. K5 is a doc/offline asset like K4/S4 — there is no live Azure phase. Part A is the always-on offline gate ($0). Part B is a worked walkthrough: facilitate the account-compromise tabletop end-to-end on paper and produce a sample after-action — proving the pack is actually runnable, not just present.

Prerequisites

  • Local tooling: python3 (stdlib only — the validator imports nothing extra)
  • Working directory: docs/runbooks/tabletop
  • The K1 incident runbooks present under docs/runbooks/incident/ (the scenarios cross-link them; the gate checks the links resolve)

Part A — Offline gate (the tabletop validate gate, ~3 min, $0)

Run the always-on offline gate and confirm all five scenarios + the template are present and correctly cross-linked.

  1. bash cd docs/runbooks/tabletop ./validate.sh # or: python3 validate.py echo "exit=$?" 
    Expected (exit 0):
    All 5 tabletop scenario(s) valid (+ template): scenario-account-compromise.md, scenario-data-exfiltration.md, scenario-ddos.md, scenario-ransomware.md, scenario-vendor-breach.md exit=0

The gate checks, for every scenario-*.md (and the shape of tabletop-template.md): - required sections present — Objective, Roles, Injects, Discussion Questions, Evaluation Rubric, After-Action; - Discussion Questions reference all five K1 phases (Identification → Containment → Eradication → Recovery → Post-Incident); - at least 4 timed injects; - a cross-link to a K1 incident runbook (../incident/*.md) that exists on disk (no dangling links).

  1. (Confirm the gate actually bites) Point a scenario at a non-existent runbook in a scratch copy and re-run — it must fail with a dangling-cross-link violation and exit 1:
tmp=$(mktemp -d); cp -r ../incident "$tmp/"; mkdir "$tmp/tabletop"
sed 's#\.\./incident/compromise\.md#../incident/does-not-exist.md#g' \
  scenario-account-compromise.md > "$tmp/tabletop/scenario-x.md"
cp tabletop-template.md validate.py "$tmp/tabletop/"
python3 "$tmp/tabletop/validate.py"; echo "exit=$?"   # expect violations + exit=1
rm -rf "$tmp"

Expected: Tabletop validation FAILED: listing a dangling cross-link for does-not-exist.md, exit=1.

  1. Confirm the five scenarios and the supporting docs are all present:
ls scenario-*.md README.md facilitation-guide.md tabletop-template.md after-action-template.md

Expected: five scenario-*.md plus the four supporting docs.

Part B — Worked walkthrough: facilitate the account-compromise tabletop (~60 min)

Run ../tabletop/scenario-account-compromise.md end-to-end as a real (or dry) facilitation, following ../tabletop/facilitation-guide.md, and produce a sample after-action. This is the proof a facilitator can pick the pack up cold.

  1. Prep (5 min). Read the scenario + ../incident/compromise.md. Assign roles (IC, on-call, identity SME, comms). Open ../tabletop/after-action-template.md to capture as you go.

  2. Kickoff (5 min). State the objective, scope/assumptions, and ground rules (blameless; no real consoles; cite the runbook step). Confirm everyone has compromise.md open.

  3. Run the injects (35 min). Reveal injects 1→5 in order (hold inject 6 as the curveball). After each, work the matching K1 phase's Discussion Questions and capture decisions + gaps:

  4. Inject 1 (impossible travel) → Identification: confirm out-of-band with Jane; recognise MFA-fatigue.
  5. Inject 2 (new authenticator + app secret) → Containment: revoke sessions, reset password, remove rogue MFA/app secret; decide on the K3 isolate-account playbook (dry-run vs enforce).
  6. Inject 3 (PIM activation + group add) → Eradication: run the runbook's SigninLogs/AuditLogs queries; spot the privilege escalation.
  7. Inject 4 (Key Vault reads + storage ACL change) → Recovery: revert the changes, rotate read secrets; note the pivot toward data-leak.

  8. Inject 5 (all-clear) → Post-Incident: enforce phishing-resistant MFA for privileged roles.

  9. Debrief + score (10 min). Score the scenario's Evaluation Rubric /20 with the room; surface the top gaps and what worked.

  10. Produce the sample after-action (5 min). Fill after-action-template.md into a scratch file and confirm it captures: header, rehearsal timing, what-went-well, gaps (citing runbook steps), and owned, dated action items routed to K4. A worked sample of the expected output:

Scenario: account-compromise   K1: ../incident/compromise.md
Rubric total: 15/20
What went well: out-of-band confirmation with Jane was fast; sessions revoked
  within the timebox.
Gaps found:
  1. compromise.md has no out-of-band contact list for the affected user
     (Identification "Immediate Action") — H
  2. no runbook guidance on revoking a PIM role activation (Containment) — M
Action items:
  1. Add an OOB contact roster to compromise.md — owner: CO — due: <date> —
     dest: K1 edit — tracked: <to file via K4/E7>
  2. Enforce FIDO2 for all PIM-eligible roles — owner: Identity — due: <date>
     — dest: RBAC/policy — tracked: <K4 issue>

Confirm the after-action would seed a K4 review cleanly: its timeline / contributing-factors / action-items map onto K4's blameless-PIR sections, so each material gap can be filed via the E7 ticketing bridge K4 uses (see apps/post-incident-review/). K5 does not run the K4 collector against Sentinel — a tabletop is a rehearsal, so the facilitator transcribes findings into K4-shaped issues.

Pass criteria

  • Part A — validate.sh exits 0; all five scenarios + template valid and cross-linked; the deliberately-broken scratch copy fails with exit 1
  • Part A — all five scenario-*.md + README/facilitation-guide/template/ after-action-template present
  • Part B — account-compromise tabletop facilitated through all five K1 phases; rubric scored /20
  • Part B — a sample after-action produced that captures gaps with owners + dates and is shaped to seed a K4 review

Failure mode

A tabletop pack that looks complete but is subtly unrunnable — a scenario missing a phase, or cross-linking a renamed/deleted runbook — so a facilitator hits a dead end mid-session. Mitigated by the always-on offline gate (Part A), which fails closed (exit 1) on any missing section, missing K1 phase, too-few injects, or dangling cross-link. The human walkthrough (Part B) backstops "present but not actually facilitatable".

Cost impact

$0 — pure docs + a stdlib validator. No cloud resources, no paging, no consoles.

Removal path

Delete docs/runbooks/tabletop/. Nothing external is provisioned; the K1 incident runbooks it references are unaffected.

Sign-Off

Field Value
Part A (offline gate) ☐ PASS
Part B (worked walkthrough) ☐ PASS
Tester
Date
Result ☐ PASS