DR Plan: <CLIENT> / <WORKLOAD>¶
Master disaster-recovery plan for one client/workload. Copy this file into the client repo (e.g.
docs/dr/<workload>/dr-plan.md), fill every<PLACEHOLDER>, and link the scenario runbooks. This is the document an incident commander opens first.Backing capabilities: Azure Backup Policy Module (
L1), Cross-Region Replication Module (L2), Automated Restore Drill (L4). Seedocs/runbooks/dr/README.mdfor how the pieces fit. A plan with unfilled placeholders is not approved.
| Field | Value |
|---|---|
| Client | <CLIENT> |
| Workload | <WORKLOAD> |
| Environments in scope | <dev / staging / prod> |
| Primary region | <PRIMARY_REGION> (e.g. eastus) |
| Secondary / paired region | <SECONDARY_REGION> (e.g. westus2) |
| Plan owner | <NAME / ROLE> |
| Last reviewed | <YYYY-MM-DD> |
Last L4 drill pass |
<YYYY-MM-DD / RestoreDrillReport id> |
| Plan version | <X.Y> |
1. Scope¶
- In scope:
<list the resources this plan covers — AKS cluster(s), SQL databases, storage accounts, Key Vaults, App Gateway/Front Door endpoints>. - Out of scope:
<SaaS dependencies, third-party services, anything covered by a separate plan>. - Assumptions: backups and replication are already provisioned via
L1/L2and verified monthly byL4. This plan covers recovery procedures, not the provisioning of the backup/replication estate.
2. Roles & contacts¶
Fill with real people. Keep an out-of-band copy (these contacts must be reachable when the primary region — and possibly the corp tenant — is down).
| Role | Name | Primary contact | Backup contact | Out-of-band |
|---|---|---|---|---|
| Incident Commander (declares DR) | <NAME> |
<phone/Slack> |
<NAME> |
<personal phone> |
| DR Operations Lead | <NAME> |
<phone/Slack> |
<NAME> |
<personal phone> |
| Database Owner (SQL failover) | <NAME> |
<phone/Slack> |
<NAME> |
<personal phone> |
| Networking (DNS/Front Door) | <NAME> |
<phone/Slack> |
<NAME> |
<personal phone> |
| Client point of contact | <NAME> |
<phone/email> |
<NAME> |
<email> |
| Communications / status page | <NAME> |
<Slack> |
<NAME> |
<email> |
On-call routing for the underlying alerts flows through the On-Call Integration
Module (K2) (PagerDuty/Opsgenie + Slack).
3. RTO / RPO objectives¶
Defaults below mirror
docs/runbooks/dr/README.mdand the realL1retention /L2failover posture. Override per the client SLA, but keep them coherent — a sub-15-minute RTO is not credible against aManualSQL failover policy.
| Env | RPO (data-loss budget) | RTO (time to recover) | Backup retention (L1) |
SQL failover posture (L2) |
|---|---|---|---|---|
| dev | <≤ 24h / best-effort> |
<next business day> |
daily 7d | Manual (no grace) |
| staging | <≤ 1h> |
<≤ 4h> |
daily 14d + weekly 5w | Automatic, 60-min grace |
| prod | <≤ 15 min SQL / ≤ 1h blob> |
<≤ 2h failover / ≤ 4h restore> |
daily 30d + weekly 12w + monthly 12m + yearly 7y | Automatic, 120-min grace |
4. Dependency map¶
What must come back, and in what order. Recovery is bottom-up (foundations first); teardown of the failed/old region is top-down. Mirrors the
F7dependency DAG (baseline → network → kv/acr → cluster) andSEQUENCING.md.
<WORKLOAD> endpoint (Front Door / App Gateway) ← step LAST in recovery
│
├── AKS workload (Secure AKS Module F3)
│ ├── depends on → SQL (failover group, L2)
│ ├── depends on → Key Vault (F5) secrets/CMK
│ └── depends on → Storage (object replication, L2)
│
├── Network Hub (F2) — vNet, Private DNS, firewall
│
└── Azure Baseline (F1) — subscription, Log Analytics, Defender ← step FIRST in rebuild
| Dependency | Recovery mechanism | Scenario runbook |
|---|---|---|
| SQL databases | L2 SQL failover group |
scenario-sql-failover.md |
| Blob/Azure Files data | L2 object replication / L1 restore |
scenario-region-failover.md, scenario-data-restore.md |
| AKS / VM / app data | L1 vault restore |
scenario-data-restore.md |
| Whole region | combine all + IaC rebuild (F7) |
scenario-full-region-loss.md |
5. Declaration criteria¶
Declare a DR event (and pick a scenario) when any holds:
- Primary region (
<PRIMARY_REGION>) is unreachable for> <N>minutes per synthetic monitoring (X8) / Azure Service Health. - Data loss/corruption confirmed and a point-in-time restore is required →
scenario-data-restore.md. - SQL primary is down or degraded beyond RTO →
scenario-sql-failover.md. - Total/extended region loss (Azure-declared regional outage) →
scenario-full-region-loss.md. - A security incident (ransomware/data-leak per
K1) requires recovery from clean backups — coordinate with the Incident Response Runbooks (K1).
Who can declare: the Incident Commander (Section 2). Record the declaration time — it starts the RTO clock.
6. Communication plan¶
| When | Audience | Channel | Owner | Template |
|---|---|---|---|---|
| On declaration | Internal responders | <#dr-warroom Slack> |
IC | "DR declared for <WORKLOAD> at <time>, scenario <X>, IC is <name>." |
Within <15 min> |
Client PoC | <phone + email> |
Client contact | <status + next update time> |
Every <30 min> |
War room + client | Slack + status page | Comms | rolling status |
| On recovery | All | Slack + status page + email | IC | "Service restored at <time>, RPO impact <X>, post-DR review to follow." |
Status page / Trust Center updates go through the Communications role. Keep client-facing language free of internal resource IDs.
7. Scenario runbooks (this plan links to)¶
- Region failover →
scenario-region-failover.md - Point-in-time data restore →
scenario-data-restore.md - SQL failover-group →
scenario-sql-failover.md - Full region loss (worst case) →
scenario-full-region-loss.md
8. Post-DR review¶
Within <5 business days> of recovery, complete a review (pairs with the
Post-Incident Review asset K4 when security-related):
- Timeline: declaration → first action → recovery → all-clear (with times).
- Objectives met? Measured RTO
<actual>vs target<target>; measured RPO<actual data loss>vs target. - What worked / what didn't: tooling, contacts, doc accuracy.
- Action items: owner + due date for each (update this plan, fix
L1/L2config, add anL4drill case, etc.). - Did
L4predict this? If the failed path had never passed a drill, that is a finding — add it to the monthly drill scope.
| Item | Owner | Due | Status |
|---|---|---|---|
<action> |
<name> |
<date> |
<open/done> |
Sign-off¶
| Field | Value |
|---|---|
| DR event / drill ref | <incident id / drill id / "review only"> |
| Scenario(s) executed | <region-failover / data-restore / sql-failover / full-region-loss> |
| Date / time (UTC) | <YYYY-MM-DD HH:MM> |
| Incident Commander | <NAME> |
| DR Operations Lead | <NAME> |
| Outcome | <recovered / partial / failed> |
| Measured RTO / RPO | <actual> / <actual> |
| Follow-up issues | <links> |
| Plan reviewed & current | ☐ yes ☐ needs update |