Skip to content

Manual Test Runbook — B3: Subscription Baseline (F1 + Group RBAC + MCSB)

Owner: Sagar  |  Time: ~5 min (Parts A + B) · +12 min (optional Part C sandbox apply) · +10 min (optional Part D posture verification)  |  Sandbox: snowops-sandbox-01

Promotes B3 (modules/azure/subscription-baseline/) from 🟦 Code Complete → 🟩 Shipped. Parts A + B are 100% offline. Part C costs ~$0 incremental — defender_plans is forced to {} in the fixture, so no Standard-tier billing; the only billable surface is short-lived Log Analytics ingestion (negligible on an empty workspace).

Prerequisites

  • Sandbox subscription access active (PIM activated if required)
  • az login done; az account show confirms the sandbox subscription is selected
  • Identity has Owner (or Contributor + User Access Administrator) on the sandbox subscription — needed to write the RBAC role assignments AND to create the MCSB policy assignment's system-assigned identity
  • Local tooling: terraform >= 1.6, go >= 1.22, az CLI >= 2.50
  • SNOWOPS_SANDBOX_SUBSCRIPTION_ID and SNOWOPS_SANDBOX_TENANT_ID env vars set
  • (Part C only) SNOWOPS_AAD_BREAKGLASS_GROUP_OBJECT_ID set to a real AAD group object ID in the sandbox tenant (B3 binds RBAC with principal_type = "Group"; a synthetic GUID fails with PrincipalNotFound). Reuse the same group the F3 AKS test consumes.
  • Working directory: repo root

Steps

Part A — terraform fmt + validate (offline, ~2 min)

  1. Confirm formatting + structural validity of the module on its own (note: B3 is a composition module — init loads the child F1 module from ../baseline):
terraform -chdir=modules/azure/subscription-baseline fmt -recursive -check
terraform -chdir=modules/azure/subscription-baseline init -backend=false -input=false
terraform -chdir=modules/azure/subscription-baseline validate

Expected: Success! The configuration is valid.

  1. Same for the example, which wires the azurerm provider + the RBAC + regulatory surface:
terraform -chdir=modules/azure/subscription-baseline/examples/basic fmt -check
terraform -chdir=modules/azure/subscription-baseline/examples/basic init -backend=false -input=false
terraform -chdir=modules/azure/subscription-baseline/examples/basic validate

Expected: Success! The configuration is valid.

  1. Run the B3-targeted offline Terratest:
cd tests/terratest
go test -v -timeout 5m ./modules/azure/... -run 'TestSubscriptionBaselineValidate'

Expected: --- PASS: TestSubscriptionBaselineValidate.

Part B — full offline Terratest suite + pre-commit (offline, ~3 min)

  1. Run the whole offline suite to confirm B3 hasn't regressed F0–F6 / H1–H3 / B2:
cd tests/terratest
go test -v -timeout 10m ./...

Expected: 20 top-level tests pass (the 19 from v0.26 plus TestSubscriptionBaselineValidate).

  1. Run pre-commit + pre-push hooks on every new file:
pre-commit run --files \
  modules/azure/subscription-baseline/versions.tf \
  modules/azure/subscription-baseline/variables.tf \
  modules/azure/subscription-baseline/main.tf \
  modules/azure/subscription-baseline/outputs.tf \
  modules/azure/subscription-baseline/README.md \
  modules/azure/subscription-baseline/examples/basic/versions.tf \
  modules/azure/subscription-baseline/examples/basic/variables.tf \
  modules/azure/subscription-baseline/examples/basic/main.tf \
  modules/azure/subscription-baseline/examples/basic/outputs.tf \
  tests/terratest/fixtures/subscription-baseline/main.tf \
  tests/terratest/fixtures/subscription-baseline/variables.tf \
  tests/terratest/fixtures/subscription-baseline/outputs.tf \
  tests/terratest/modules/azure/subscription_baseline_validate_test.go \
  tests/terratest/modules/azure/subscription_baseline_test.go \
  docs/runbooks/test/B3.md

pre-commit run --hook-stage pre-push --all-files

Expected: every hook PASS; FINAL_EXIT=0 from the pre-push run.

Part C — integration test (real sandbox apply + destroy, ~12 min, ~$0)

Skip if iterating on offline changes only. The fixture forces defender_plans = {} so the apply does not enable Standard-tier Defender — cost stays ~$0.

  1. Confirm sandbox env vars + the real group object ID:
export SNOWOPS_SANDBOX_SUBSCRIPTION_ID="<sandbox-subscription-guid>"
export SNOWOPS_SANDBOX_TENANT_ID="<sandbox-tenant-guid>"
export SNOWOPS_AAD_BREAKGLASS_GROUP_OBJECT_ID="<real-sandbox-group-object-id>"
  1. Run the B3 integration test:
cd tests/terratest
go test -v -tags integration -timeout 30m ./modules/azure/... -run TestSubscriptionBaselineModule
  1. Watch for key milestones:
  2. azurerm_log_analytics_workspace.this: Creation complete (via the F1 child module) — the slow step, ~1-3 min.
  3. azurerm_policy_set_definition.snowops_standard: Creation complete + azurerm_subscription_policy_assignment.snowops_standard[...]: Creation complete.
  4. azurerm_subscription_policy_assignment.regulatory[0]: Creation complete — the MCSB assignment with its system-assigned identity.
  5. azurerm_role_assignment.rbac[...]: Creation complete ×3 (Contributor, Security Reader, the freeform RG-scope Reader) — ~5-30s each as ARM propagates.
  6. All output assertions PASS.
  7. Destroy complete! — clean LIFO teardown.

Part D — posture verification (optional, ~10 min)

Confirms the catalog test criterion directly: "az policy state list shows initiative assigned; Defender plans on". Run this against a manually stood-up stack (so it isn't auto-destroyed). Easiest path: terraform apply the example under examples/basic with a real group ID, verify, then terraform destroy.

  1. Verify the SnowOps Standard custom initiative + the MCSB regulatory initiative are both assigned at subscription scope:
az policy assignment list --scope "/subscriptions/$SNOWOPS_SANDBOX_SUBSCRIPTION_ID" \
  --query "[?starts_with(name,'snowops')].{name:name, policyDef:policyDefinitionId}" -o table

Expected: two rows — the SnowOps Standard set assignment and snowops-mcsb-... pointing at the MCSB policy set definition (...1f3afdf9-d0c9-4c3d-847f-89da613e70a8).

  1. Confirm the RBAC group bindings landed:

    az role assignment list \
  --assignee "$SNOWOPS_AAD_BREAKGLASS_GROUP_OBJECT_ID" \
  --scope "/subscriptions/$SNOWOPS_SANDBOX_SUBSCRIPTION_ID" \
  --query "[].{role:roleDefinitionName, scope:scope}" -o table

    Expected: Contributor + Security Reader at subscription scope (and Reader at the workspace RG scope if you applied the fixture rather than the example).

  2. (If Defender plans were enabled in your manual apply — they are NOT in the test fixture) confirm the pricing tier:

    az security pricing list --query "value[?pricingTier=='Standard'].name" -o tsv

    Expected: the SnowOps default plan set (VirtualMachines, StorageAccounts, KeyVaults, Arm, Containers, AppServices, SqlServers, CloudPosture). Skip this step when validating via the test fixture, which uses defender_plans = {}.

  3. Tear down the manual stack:

    terraform -chdir=modules/azure/subscription-baseline/examples/basic destroy \
  -var subscription_id=$SNOWOPS_SANDBOX_SUBSCRIPTION_ID \
  -var tenant_id=$SNOWOPS_SANDBOX_TENANT_ID

Pass criteria

  • Part A — terraform validate passes for both module and example
  • Part A — TestSubscriptionBaselineValidate passes
  • Part B — full offline Terratest suite passes (20 top-level tests)
  • Part B — pre-commit + pre-push hooks all green
  • Part C — TestSubscriptionBaselineModule integration test passes end-to-end
  • Part C — regulatory_assignment_id + regulatory_assignment_principal_id (system-assigned identity GUID) both non-empty
  • Part C — all 3 RBAC role-assignment keys present in role_assignment_ids
  • Part C — identity_contract.scope_type == "azure_subscription" (no MG in scope)
  • Part C — destroy succeeds (LIFO)
  • (Part D) az policy assignment list shows both the SnowOps Standard set + MCSB assignments
  • (Part D) az role assignment list shows the group's Contributor + Security Reader bindings

Teardown

The integration test runs terraform destroy automatically. If a failure mid-run orphans resources:

# Resource group (LAW) — find the test RG and delete
az group list --query "[?starts_with(name,'snowops-b3-test-')].name" -o tsv | xargs -I{} az group delete -n {} --yes --no-wait

# Regulatory + SnowOps Standard policy assignments
az policy assignment list --scope "/subscriptions/$SNOWOPS_SANDBOX_SUBSCRIPTION_ID" \
  --query "[?starts_with(name,'snowops-mcsb-') || starts_with(name,'snowops-standard-b3-')].name" -o tsv \
  | xargs -I{} az policy assignment delete --name {} --scope "/subscriptions/$SNOWOPS_SANDBOX_SUBSCRIPTION_ID"

# Policy set definition
az policy set-definition list --query "[?starts_with(name,'snowops-standard-b3-')].name" -o tsv \
  | xargs -I{} az policy set-definition delete --name {}

# RBAC role assignments for the group at subscription scope
az role assignment delete --assignee "$SNOWOPS_AAD_BREAKGLASS_GROUP_OBJECT_ID" \
  --scope "/subscriptions/$SNOWOPS_SANDBOX_SUBSCRIPTION_ID"

Note: Defender pricing does NOT auto-revert to Free on destroy. The test fixture never enables Standard plans, so there is nothing to revert. If you ran a manual apply with Defender plans on, set them back with az security pricing create -n <plan> --tier Free.

Sign-off

  • Tester: _  |  Date: _  |  Result: PASS / FAIL / N/A
  • Notes: