Skip to content

Manual Test Runbook — D3 + X3: OPA / Conftest Policy Bundle

Owner: Sagar  |  Time: ~20 min  |  Sandbox: none (purely local)

D3 ships the SnowOps OPA bundle (policy/opa/rules/). X3 ships the test suite (policy/opa/tests/). This runbook validates both — unit tests via conftest verify, and end-to-end via conftest test against the committed fixture plans.

Prerequisites

conftest --version
# Conftest: dev (or a tagged build)
# OPA: 1.15.x

Steps

Part A — conftest verify: the bundle's own tests (~3 min)

  1. Run the policy-side unit tests:
conftest verify --policy policy/opa/rules --policy policy/opa/tests
  1. Expected: 36 tests, 36 passed, 0 warnings, 0 failures, 0 exceptions, 0 skipped. Coverage breakdown (6 + 4 + 8 + 7 + 8 + 3 = 36):
  2. tags_test.rego — 6 tests
  3. locations_test.rego — 4 tests
  4. network_test.rego — 8 tests
  5. encryption_test.rego — 7 tests
  6. cost_test.rego — 8 tests

  7. integration_test.rego — 3 cross-category tests

  8. If a test fails, fix the rule and the test before re-running.

Part B — conftest test against the clean fixture (~3 min)

  1. Confirm the bundle reports zero findings on a fully-compliant plan:
conftest test policy/opa/tests/fixtures/clean_plan.json \
  --policy policy/opa/rules
  1. Expected: 14 tests, 14 passed, 0 warnings, 0 failures, 0 exceptions. Exit code 0.

Part C — conftest test against the bad-plan fixture (~3 min)

  1. Confirm the bundle catches the planted violations:
conftest test policy/opa/tests/fixtures/bad_plan.json \
  --policy policy/opa/rules
  1. Expected: 9 failures, one or more per category:
  2. snowops.tags — 1 finding (missing tags on the untagged RG)
  3. snowops.locations — 1 finding (brazilsouth)
  4. snowops.network — 3 findings (storage public_network_access_enabled=true, storage allow_nested_items_to_be_public=true, NSG SSH-from-internet)
  5. snowops.encryption — 2 findings (storage min_tls_version=TLS1_0, storage https_traffic_only_enabled=false)
  6. snowops.cost — 2 findings (legacy azurerm_postgresql_server, M-series VM)
  7. Exit code non-zero (Conftest exits 1 whenever any deny fires).

Part D — pre-push hook smoke test (~3 min)

  1. Confirm the pre-push hook wires conftest in:
pip install pre-commit
pre-commit install --install-hooks
pre-commit run conftest-verify --all-files --hook-stage pre-push
  1. Expected: hook executes conftest verify and exits 0. If conftest is not on PATH, the hook fails immediately — install it first.

Part E — D2 CI job (~5 min, optional, requires push to GitHub)

  1. From a feature branch, push any change and open a PR. The quality-gates / conftest verify (policy/opa) job must run alongside the other five gates and report ✅.

  2. Optional negative test: introduce a deliberately broken assertion in a test file (e.g., change count(deny) == 0 to count(deny) == 1 in a clearly-passing test), push, and confirm the CI job fails on the PR. Revert before merging.

Pass criteria

  • conftest verify reports 36/36 tests passing.
  • conftest test on clean_plan.json exits 0 with no findings.
  • conftest test on bad_plan.json exits non-zero with at least one finding from each of the five snowops.* categories.
  • pre-commit run conftest-verify --hook-stage pre-push succeeds locally.
  • (Optional) quality-gates / conftest verify (policy/opa) job passes on a clean PR; fails on a planted-bad-rule PR.

Teardown

D3+X3 have no cloud side-effects. If you mutated a test file for Part E, revert it before merging:

git checkout -- policy/opa/tests

Sign-off

  • Tester: _  |  Date: _  |  Result: PASS / FAIL / N/A
  • Notes: