Skip to content

Manual Test Runbook — F0: Cloud-Agnostic Module Contracts

Owner: Sagar  |  Time: ~15 min  |  Sandbox: not required

What this runbook proves

  1. Every modules/_contracts/<domain>/ sub-module is a valid Terraform module carrying no providers and no resources.
  2. F1 (modules/azure/baseline/) emits identity_contract and observability_contract outputs whose shapes compile against the corresponding contracts.
  3. F6 (modules/azure/state-backend/) emits an object_store_contract output whose shape compiles against the object_store/ contract.
  4. The contract modules mechanically REJECT bad literal candidates — proving the gate has teeth, not just that compatible shapes happen to compile.

Apply-time conformance (the catch for drift in module outputs wired from un-applied resources) is exercised by the F1 + F6 runbooks against the sandbox subscription — see F1.md Part B and F6.md Part B.

Prerequisites

  • Local tooling: terraform >= 1.6, go >= 1.22
  • Working directory: repo root
  • No cloud credentials needed — this runbook is 100% offline

Steps

Part A — every contract module validates standalone (~2 min)

  1. Walk every contract sub-module and confirm terraform fmt -check, terraform init -backend=false, and terraform validate all succeed:
for c in identity observability object_store network cluster registry kv; do
  echo "=== _contracts/$c ==="
  terraform -chdir=modules/_contracts/$c fmt -check && \
  terraform -chdir=modules/_contracts/$c init -backend=false -input=false >/dev/null && \
  terraform -chdir=modules/_contracts/$c validate || { echo "FAIL: $c"; exit 1; }
done
echo "ALL CONTRACTS VALIDATED"
  1. Expect to see Success! The configuration is valid. seven times followed by ALL CONTRACTS VALIDATED.

Part B — F1 and F6 retrofit still validate (~2 min)

  1. Confirm the F1 retrofit (added identity_contract + observability_contract outputs and the azurerm_client_config data source) hasn't broken F1:
terraform -chdir=modules/azure/baseline fmt -check && \
terraform -chdir=modules/azure/baseline init -backend=false -input=false >/dev/null && \
terraform -chdir=modules/azure/baseline validate
  1. Same for F6 (added object_store_contract):
terraform -chdir=modules/azure/state-backend fmt -check && \
terraform -chdir=modules/azure/state-backend init -backend=false -input=false >/dev/null && \
terraform -chdir=modules/azure/state-backend validate

Part C — positive conformance tests pass (~30 s after init cache warms)

  1. Run the positive-path tests. They wire F1 + F6's contract outputs through the matching contract modules and assert terraform validate succeeds:
cd tests/terratest
go test -v -timeout 5m -run 'TestF1ContractConformance|TestF6ObjectStoreContractConformance' ./modules/azure/
  1. Expect:
--- PASS: TestF1ContractConformance (...s)
--- PASS: TestF6ObjectStoreContractConformance (...s)
PASS

Part D — negative conformance tests prove the gate has teeth (~30 s)

  1. Run the negative-path tests. They pipe deliberately-malformed literal candidates through each contract and assert terraform validate FAILS with the expected diagnostic substring:
go test -v -timeout 5m -run TestContractsRejectBadLiterals ./modules/azure/
  1. Expect four sub-tests to pass:
--- PASS: TestContractsRejectBadLiterals/identity-missing-scope-type
--- PASS: TestContractsRejectBadLiterals/identity-bad-enum
--- PASS: TestContractsRejectBadLiterals/observability-missing-name
--- PASS: TestContractsRejectBadLiterals/object-store-locked-without-enabled
--- PASS: TestContractsRejectBadLiterals

Each scenario triggers a different rejection path: - identity-missing-scope-type — missing required field rejected by type. - identity-bad-enum — scope_type outside closed set rejected by validation block. - observability-missing-name — missing required field rejected by type. - object-store-locked-without-enabled — cross-field rule rejected by validation block.

Part E — full Terratest suite stays green (~30 s)

  1. Run the entire repo's Terratest suite to confirm nothing else regressed:
go test -v -timeout 5m ./...
  1. Expect all tests to pass, including the pre-existing TestBaselineValidate, TestStateBackendValidate, TestNoopHarness, TestSandboxValidate.

Part F — spot-check the contract output shape in F1 / F6 source (~3 min)

  1. Read modules/azure/baseline/outputs.tf and confirm the two new outputs are present and that every contract field has a value:

    • identity_contract — one of two shapes depending on whether an MG is in scope. Both include scope_id, scope_type, parent_scope_id, tenant_id, display_name.
    • observability_contract — workspace_id, workspace_name, customer_id, region, retention_days, ingest_endpoint.

  2. Read modules/azure/state-backend/outputs.tf and confirm object_store_contract includes every field listed in modules/_contracts/object_store/variables.tf.

  3. Read modules/_contracts/README.md and confirm the seven-contract table matches what's on disk.

Pass criteria

  • All 7 contract sub-modules: fmt -check clean, init -backend=false OK, validate Success.
  • F1 + F6 still validate clean after retrofit.
  • TestF1ContractConformance + TestF6ObjectStoreContractConformance PASS.
  • TestContractsRejectBadLiterals PASS with all 4 sub-scenarios green.
  • Full go test ./... suite remains green (5 prior tests + 3 new = 8 top-level).
  • F1 + F6 contract outputs visually match the corresponding contract variables.tf shapes.
  • modules/_contracts/README.md table matches ls modules/_contracts/.

Teardown

Nothing to clean up — no cloud resources are created by this runbook. The .terraform/ and .terraform.lock.hcl artifacts left in each contract module directory may be removed if you want a pristine tree:

find modules/_contracts -type d -name .terraform -exec rm -rf {} +
find modules/_contracts -name .terraform.lock.hcl -delete
find tests/terratest/fixtures/baseline-contract tests/terratest/fixtures/state-backend-contract tests/terratest/fixtures/contracts-negative -type d -name .terraform -exec rm -rf {} +
find tests/terratest/fixtures/baseline-contract tests/terratest/fixtures/state-backend-contract tests/terratest/fixtures/contracts-negative -name .terraform.lock.hcl -delete

Sign-off

  • Tester: ____  |  Date: ______  |  Result: PASS / FAIL / N/A
  • Notes: