Skip to content

Manual Test Runbook — D4 + X4: Kyverno Policy Bundle

Owner: Sagar  |  Time: ~20 min (offline) · +30 min (optional kind-cluster install)  |  Sandbox: none for Parts A–B; local kind for Part C

D4 ships the SnowOps Kyverno bundle (policy/kyverno/rules/). X4 ships the test harness (policy/kyverno/tests/). This runbook validates both — offline via kyverno test, then end-to-end via an actual admission round-trip on a local kind cluster (Part C is optional).

Prerequisites

  • Local tooling:
    • kyverno >= 1.18 (brew install kyverno on macOS).
    • (Part C only) kind, kubectl, Docker daemon running.
  • Working directory: repo root.
  • Clean working tree.
kyverno version | head -1
# Version: 1.18.x

Steps

Part A — kyverno test per-rule fixtures (~2 min)

  1. Run the full X4 suite:
policy/kyverno/tests/run-tests.sh
  1. Expected:
Running 5 kyverno test suites...

── disallow-latest-tag                      ── PASS
── disallow-privileged-containers           ── PASS
── require-network-policy                   ── PASS
── require-pod-labels                       ── PASS
── require-signed-images                    ── PASS

All 5 suites passed.
  1. Per-rule breakdown:
  2. disallow-latest-tag — 6 assertions (pass + fail per rule × 3 pods × 2 rules)
  3. disallow-privileged-containers — 10 assertions (5 pods × 2 rules)
  4. require-pod-labels — 3 assertions (well-labelled / missing-owner / no-labels)
  5. require-network-policy — 1 assertion (Namespace → generated default-deny NetworkPolicy matches tests/require-network-policy/generated.yaml)

  6. require-signed-images — 1 assertion (non-ACR image short-circuits to skip; ACR signature verification is exercised in Part C only — kyverno-test cannot reach a live registry)

  7. If a suite fails, fix the rule and the fixture before re-running. The wrapper prints the full kyverno-test output for any failing suite.

Part B — per-rule deep-dive (~5 min, optional)

  1. Run a single suite with verbose output to confirm individual results:
kyverno test policy/kyverno/tests/disallow-latest-tag --detailed-results
  1. Spot-check the result table:
  2. tagged-pod × both rules → Pass / Ok
  3. untagged-pod × require-image-tag → Pass / Ok (test asserts fail)
  4. latest-tag-pod × disallow-latest-tag → Pass / Ok (test asserts fail)

"Result: Pass" in the table = "the assertion matched". "Reason: Ok" = the policy WAS evaluated (vs "Excluded" / "Not found"). Both must be present for a clean run.

Part C — live admission on a kind cluster (~30 min, optional)

Required before signing off the rule bundle for client install. Skip if only iterating on rule wording or the test harness.

  1. Spin up a local kind cluster + install Kyverno:
kind create cluster --name snowops-d4-test
kubectl create namespace kyverno

# Kyverno v1.18.x install via Helm
helm repo add kyverno https://kyverno.github.io/kyverno/
helm install kyverno kyverno/kyverno --namespace kyverno --version 3.4.x \
  --set replicaCount=1
kubectl -n kyverno rollout status deploy/kyverno-admission-controller --timeout=5m
  1. Apply the SnowOps rule bundle:
kubectl apply -f policy/kyverno/rules/
kubectl get clusterpolicy
# NAME                                          BACKGROUND   VALIDATE ACTION   READY
# snowops-disallow-latest-tag                   true         Enforce           true
# snowops-disallow-privileged-containers        true         Enforce           true
# snowops-require-default-deny-netpol           true         Enforce           true
# snowops-require-pod-labels                    true         Enforce           true
# snowops-require-signed-images                 true         Enforce           true
  1. Validate each rule end-to-end. Create a workload namespace first:
kubectl create namespace app
# Expect: kyverno generates a default-deny NetworkPolicy automatically.
kubectl -n app get networkpolicy default-deny
# NAME           POD-SELECTOR   AGE
# default-deny   <none>         5s

  1. Attempt to create a Pod with :latest:

    kubectl -n app run bad-pod --image=nginx:latest --restart=Never
# Expect:
# Error from server: admission webhook "validate.kyverno.svc-fail" denied the
# request: ... Image tag ':latest' is not allowed ...

  2. Attempt to create a privileged Pod (use a quick manifest):

    kubectl -n app apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: priv-test
  labels:
    app.kubernetes.io/name: priv-test
    app.kubernetes.io/version: 1.0.0
    app.kubernetes.io/part-of: snowops-suite
    snowops.io/owner: platform-team
spec:
  containers:
  - name: bad
    image: nginx:1.27.0
    securityContext:
      privileged: true
EOF
# Expect rejection: "Privileged containers are not allowed."

  3. Attempt to create a Pod missing the SnowOps labels:

    kubectl -n app run no-labels --image=nginx:1.27.0 --restart=Never
# Expect rejection: "Pods must carry ... snowops.io/owner labels."

  4. Create a fully-conformant Pod and confirm admission succeeds:

    kubectl -n app apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: good-pod
  labels:
    app.kubernetes.io/name: good-pod
    app.kubernetes.io/version: 1.0.0
    app.kubernetes.io/part-of: snowops-suite
    snowops.io/owner: platform-team
spec:
  containers:
  - name: web
    image: nginx:1.27.0
EOF
# Expect: pod/good-pod created

  5. Confirm the exclude block keeps system namespaces working:

    kubectl get networkpolicy -A | grep default-deny
# Expect default-deny in `app` only — NOT in kube-system, kube-public,
# kube-node-lease, kyverno, or default.

  6. Cleanup:

    kind delete cluster --name snowops-d4-test

Pass criteria

  • Part A — policy/kyverno/tests/run-tests.sh reports 5/5 suites passing.
  • Part B — per-suite kyverno test --detailed-results runs show Reason: Ok for every evaluated case (no Excluded / Not found).
  • (Part C) kind cluster boots and Kyverno admission controller is Ready.
  • (Part C) kubectl apply -f policy/kyverno/rules/ succeeds and kubectl get clusterpolicy shows all 5 policies Ready=true.
  • (Part C) Each of the four failure paths (latest tag, privileged, missing labels, untagged image) is rejected by the admission webhook with the expected error message.
  • (Part C) A fully-conformant Pod is admitted.
  • (Part C) default-deny NetworkPolicy is auto-generated in workload namespaces and NOT generated in kube-system / kyverno / default / kube-public / kube-node-lease.

Teardown

D4 + X4 have no cloud side-effects. Part C creates a local kind cluster that must be deleted with kind delete cluster --name snowops-d4-test.

If Part A flagged a regression, revert the offending file:

git checkout -- policy/kyverno/

Sign-off

  • Tester: _  |  Date: _  |  Result: PASS / FAIL / N/A
  • Notes: