Skip to content

DR Plan: <CLIENT> / <WORKLOAD>

Master disaster-recovery plan for one client/workload. Copy this file into the client repo (e.g. docs/dr/<workload>/dr-plan.md), fill every <PLACEHOLDER>, and link the scenario runbooks. This is the document an incident commander opens first.

Backing capabilities: Azure Backup Policy Module (L1), Cross-Region Replication Module (L2), Automated Restore Drill (L4). See docs/runbooks/dr/README.md for how the pieces fit. A plan with unfilled placeholders is not approved.

Field Value
Client <CLIENT>
Workload <WORKLOAD>
Environments in scope <dev / staging / prod>
Primary region <PRIMARY_REGION> (e.g. eastus)
Secondary / paired region <SECONDARY_REGION> (e.g. westus2)
Plan owner <NAME / ROLE>
Last reviewed <YYYY-MM-DD>
Last L4 drill pass <YYYY-MM-DD / RestoreDrillReport id>
Plan version <X.Y>

1. Scope

  • In scope: <list the resources this plan covers — AKS cluster(s), SQL databases, storage accounts, Key Vaults, App Gateway/Front Door endpoints>.
  • Out of scope: <SaaS dependencies, third-party services, anything covered by a separate plan>.
  • Assumptions: backups and replication are already provisioned via L1/L2 and verified monthly by L4. This plan covers recovery procedures, not the provisioning of the backup/replication estate.

2. Roles & contacts

Fill with real people. Keep an out-of-band copy (these contacts must be reachable when the primary region — and possibly the corp tenant — is down).

Role Name Primary contact Backup contact Out-of-band
Incident Commander (declares DR) <NAME> <phone/Slack> <NAME> <personal phone>
DR Operations Lead <NAME> <phone/Slack> <NAME> <personal phone>
Database Owner (SQL failover) <NAME> <phone/Slack> <NAME> <personal phone>
Networking (DNS/Front Door) <NAME> <phone/Slack> <NAME> <personal phone>
Client point of contact <NAME> <phone/email> <NAME> <email>
Communications / status page <NAME> <Slack> <NAME> <email>

On-call routing for the underlying alerts flows through the On-Call Integration Module (K2) (PagerDuty/Opsgenie + Slack).

3. RTO / RPO objectives

Defaults below mirror docs/runbooks/dr/README.md and the real L1 retention / L2 failover posture. Override per the client SLA, but keep them coherent — a sub-15-minute RTO is not credible against a Manual SQL failover policy.

Env RPO (data-loss budget) RTO (time to recover) Backup retention (L1) SQL failover posture (L2)
dev <≤ 24h / best-effort> <next business day> daily 7d Manual (no grace)
staging <≤ 1h> <≤ 4h> daily 14d + weekly 5w Automatic, 60-min grace
prod <≤ 15 min SQL / ≤ 1h blob> <≤ 2h failover / ≤ 4h restore> daily 30d + weekly 12w + monthly 12m + yearly 7y Automatic, 120-min grace

4. Dependency map

What must come back, and in what order. Recovery is bottom-up (foundations first); teardown of the failed/old region is top-down. Mirrors the F7 dependency DAG (baseline → network → kv/acr → cluster) and SEQUENCING.md.

<WORKLOAD> endpoint  (Front Door / App Gateway)         ← step LAST in recovery
        ├── AKS workload (Secure AKS Module F3)
        │        ├── depends on → SQL (failover group, L2)
        │        ├── depends on → Key Vault (F5) secrets/CMK
        │        └── depends on → Storage (object replication, L2)
        ├── Network Hub (F2) — vNet, Private DNS, firewall
        └── Azure Baseline (F1) — subscription, Log Analytics, Defender   ← step FIRST in rebuild
Dependency Recovery mechanism Scenario runbook
SQL databases L2 SQL failover group scenario-sql-failover.md
Blob/Azure Files data L2 object replication / L1 restore scenario-region-failover.md, scenario-data-restore.md
AKS / VM / app data L1 vault restore scenario-data-restore.md
Whole region combine all + IaC rebuild (F7) scenario-full-region-loss.md

5. Declaration criteria

Declare a DR event (and pick a scenario) when any holds:

  • Primary region (<PRIMARY_REGION>) is unreachable for > <N> minutes per synthetic monitoring (X8) / Azure Service Health.
  • Data loss/corruption confirmed and a point-in-time restore is required → scenario-data-restore.md.
  • SQL primary is down or degraded beyond RTO → scenario-sql-failover.md.
  • Total/extended region loss (Azure-declared regional outage) → scenario-full-region-loss.md.
  • A security incident (ransomware/data-leak per K1) requires recovery from clean backups — coordinate with the Incident Response Runbooks (K1).

Who can declare: the Incident Commander (Section 2). Record the declaration time — it starts the RTO clock.

6. Communication plan

When Audience Channel Owner Template
On declaration Internal responders <#dr-warroom Slack> IC "DR declared for <WORKLOAD> at <time>, scenario <X>, IC is <name>."
Within <15 min> Client PoC <phone + email> Client contact <status + next update time>
Every <30 min> War room + client Slack + status page Comms rolling status
On recovery All Slack + status page + email IC "Service restored at <time>, RPO impact <X>, post-DR review to follow."

Status page / Trust Center updates go through the Communications role. Keep client-facing language free of internal resource IDs.

8. Post-DR review

Within <5 business days> of recovery, complete a review (pairs with the Post-Incident Review asset K4 when security-related):

  • Timeline: declaration → first action → recovery → all-clear (with times).
  • Objectives met? Measured RTO <actual> vs target <target>; measured RPO <actual data loss> vs target.
  • What worked / what didn't: tooling, contacts, doc accuracy.
  • Action items: owner + due date for each (update this plan, fix L1/L2 config, add an L4 drill case, etc.).
  • Did L4 predict this? If the failed path had never passed a drill, that is a finding — add it to the monthly drill scope.
Item Owner Due Status
<action> <name> <date> <open/done>

Sign-off

Field Value
DR event / drill ref <incident id / drill id / "review only">
Scenario(s) executed <region-failover / data-restore / sql-failover / full-region-loss>
Date / time (UTC) <YYYY-MM-DD HH:MM>
Incident Commander <NAME>
DR Operations Lead <NAME>
Outcome <recovered / partial / failed>
Measured RTO / RPO <actual> / <actual>
Follow-up issues <links>
Plan reviewed & current ☐ yes ☐ needs update