Skip to content

Manual Test Runbook — G4: Discovery run workflow

Owner: Sagar  |  Time: ~30 min  |  Sandbox: SnowOps sandbox subscription (acts as the "client")

Purpose

End-to-end test of the manual-dispatch workflow: OIDC into the sandbox, collect → evaluate → render → upload artifact → Slack notify.

Prerequisites

  • G0 runbook passed against the sandbox sub
  • GitHub Environment discovery-default created on this repo with:
  • Secret AZURE_CLIENT_ID set to the APP_ID printed by G0
  • (Optional) Secret HUBSPOT_PRIVATE_APP_TOKEN
  • (Optional) Variables SNOWOPS_AUDIT_ACCOUNT_URL, SNOWOPS_AUDIT_CONTAINER, SLACK_WEBHOOK_URL
  • (Recommended) Required reviewer on the environment so production runs gate on a human

Steps

1. Dispatch the workflow

GitHub → Actions → discovery-run → Run workflow:

  • client_name: sandbox-test
  • tenant_id: <SANDBOX_TENANT_ID>
  • subscription_id: <SANDBOX_SUB_ID>
  • subscription_display_name: SnowOps Sandbox

  • environment: discovery-default

  • Workflow run starts within 30s

  • Concurrency group discovery-<sub> shown — a second dispatch with the same sub queues, not parallels

2. Job execution

  • Install pandoc step completes
  • Install discovery-auditor deps runs npm ci cleanly
  • Build TS step exits 0
  • Run tests step exits 0 (no rule has a fixture gap; chain verify passes)
  • Azure login succeeds via federated OIDC (no client secret in any step log)
  • Run discovery audit exits 0 in under 5 min

3. Artifact

  • Artifact discovery-sandbox-test-<run_id> is present
  • Contains .json, .md, .pdf
  • PDF opens; cover shows SnowOps Sandbox; severity counts non-zero
  • Retention is 90 days

4. Slack notification (if SLACK_WEBHOOK_URL is set)

  • Message lands in the configured channel
  • Contains the reviewer checklist
  • Artifact URL is clickable

5. WORM audit append (if audit account configured)

  • [discovery] audit record_hash=... line in the workflow log
  • Verify per G6.md step 4

6. HubSpot patch (optional — only if you provided a Deal ID)

  • HubSpot Deal property discovery_report_url is set to the artifact URL
  • Verify by opening the Deal in the HubSpot UI

7. Negative test — concurrency

  • Dispatch a second run with the same subscription_id while the first is still in progress
  • Second run waits in the queue rather than racing the first

Pass criteria

  • One green run end-to-end with all artifacts produced
  • No client secret in any step's log output
  • Reviewer checklist visible in Slack
  • Concurrency gate works

Failure modes & escalation

Symptom Likely cause Action
AADSTS70021: No matching federated identity record found Workflow ref or repo subject doesn't match the FIC Re-run G0 with the correct --workflow-ref
Pandoc step fails apt cache stale Add sudo apt-get update retry; usually transient
Slack 403 Webhook URL revoked Rotate webhook; update repo variable

Sign-off

  • Tester: ___  |  Date: _  |  Result: PASS / FAIL / N/A
  • Notes: