VxRail High Availability and Disaster Recovery Design

By Brendan Finley, Maven IT Solutions

Most VxRail buyers evaluate high availability and disaster recovery as if they were one feature. They aren’t. HA is a default, largely automatic capability built into vSphere and vSAN. DR is a design decision that has to be made deliberately, sized correctly, and tested. Conflating the two is where a lot of organizations discover, mid-incident, that the protection they assumed they had doesn’t cover the failure they’re actually facing. This article breaks down what VxRail’s HA layer actually does, where its limits are, and what stretched clusters, replication, and backup add on top of it.

How vSphere HA and vSAN fault domains protect workloads

vSphere HA is the cluster service that watches for node failure and restarts affected virtual machines on the nodes still running. It works off a heartbeat between hosts. When a node stops responding, HA declares it failed and powers the affected VMs back on elsewhere in the cluster, typically within one to a few minutes depending on cluster size and load.

vSAN fault domains determine how data gets distributed so a hardware failure doesn’t also take out the data a VM needs in order to restart. By default, each VxRail node is its own fault domain, and vSAN places redundant copies of a VM’s data across separate fault domains. A single node failure shouldn’t strand both the compute and the storage a workload depends on, provided the storage policy protecting that VM accounts for it.

These two systems work as a pair, not independently. HA can restart a VM in seconds, but the VM only comes back online if vSAN still has an accessible, healthy copy of its data. If the storage policy protecting that VM doesn’t account for the failure that just occurred, HA has nothing to restart onto.

None of this happens automatically without configuration. Admission control has to reserve enough spare capacity across the cluster for HA to have somewhere to restart workloads. Undersized clusters, or clusters running close to capacity, can have HA enabled and still fail to restart every VM during a real event.

What happens during a single node failure

In a healthy, properly sized cluster, a single node failure is close to a non-event. vSphere HA detects the lost heartbeat within seconds and begins restarting affected VMs on the remaining nodes. Most production environments see workloads back online within one to three minutes, depending on how many VMs are involved and how quickly vCenter can place them.

Behind the restart, vSAN starts a resync operation. Every object that had a data copy on the failed node needs a new copy rebuilt elsewhere to restore the storage policy’s full protection level. Resync duration depends on how much data lived on that node and how much spare I/O and network bandwidth the surviving nodes have available. On a cluster running with thin headroom, resync competes with production traffic and slows both.

The operational takeaway is straightforward. A single node failure in a well-designed cluster is something HA is built to absorb without becoming an incident. The real exposure isn’t the failure itself. It’s the window between when that failure happens and when resync finishes restoring full redundancy. During that window, the cluster is running with less protection than its storage policy assumes, which is exactly the scenario the next section covers.

Tolerating multiple simultaneous failures

How many failures a VxRail cluster can survive at once is set by the vSAN storage policy’s Failures to Tolerate (FTT) value, not by cluster size on its own. FTT defines how many redundant copies of a VM’s data vSAN keeps. Each additional failure tolerated costs usable capacity and requires more nodes in the cluster.

The tradeoff is usable capacity versus resilience. RAID-1 mirroring consumes more raw capacity to protect data but has less rebuild overhead than erasure coding. RAID-5 and RAID-6 use capacity more efficiently but demand more nodes and more compute during a rebuild. Neither option is universally correct. The right policy depends on the cluster size already in place and how much usable capacity the workloads on it need.

The scenario that matters operationally is a second failure landing before resync from the first one finishes. If a cluster is running FTT=1 and a node fails, vSAN can absorb that one failure, but the cluster is temporarily down to zero spare protection for any object that had a copy on that node. A second failure during the resync window, on a different node holding the only remaining copy of that object, can make it inaccessible. This isn’t a theoretical edge case. It’s the direct consequence of running FTT=1 without enough spare capacity to resync quickly.

Stretched clusters: active-active across two sites

A vSAN stretched cluster splits a single VxRail cluster’s nodes across two physical sites, running active-active, with a witness host or appliance at a third location to break ties if the link between sites goes down. Instead of protecting against a node failure inside one data center, a stretched cluster protects against losing an entire site.

Data written to a stretched cluster is mirrored synchronously between the two data sites, so a workload can fail over to the surviving site with no data loss for VMs that were healthy at the moment of failure. That synchronous requirement is also the design’s biggest constraint. VMware requires round-trip latency between the two data sites to stay at or under 5 milliseconds, which in practice limits how far apart the sites can realistically be, usually within the same metro area or campus rather than across a region. The link to the witness site is far more forgiving, tolerating up to 200 milliseconds round-trip, since the witness stores only small amounts of metadata rather than VM data.

What a stretched cluster does not do matters just as much. It protects against a site-level infrastructure failure: power loss, cooling failure, a network outage. It does not protect against data corruption, a ransomware event, or an application-level failure, because whatever gets written propagates to both sites almost immediately. A stretched cluster with a healthy, current copy of corrupted or encrypted data is still down. That’s the gap that trips up buyers who assume a stretched cluster is their disaster recovery plan. It’s an availability architecture, not a recovery plan. The two solve different problems, and most enterprise environments need both.

Backup and replication integration options

VxRail supports two additional layers of protection beyond HA and stretched clusters, and both matter for a complete DR posture.

Native vSAN replication runs asynchronously between two vSAN clusters, on a schedule rather than continuously, giving VMs a defined recovery point objective that can range from a few minutes to several hours depending on configuration and link bandwidth. It’s a lighter-weight option than a stretched cluster, useful when the two sites don’t meet the latency requirements for synchronous mirroring, or when the workload doesn’t need zero data loss.

For most enterprise environments, native replication is only part of the picture. VxRail integrates with third-party backup platforms through vSphere’s APIs, enabling VM-level backup, point-in-time recovery, and immutable, off-cluster copies that a stretched cluster or native replication alone can’t provide. Immutable, off-cluster backup is specifically what protects against ransomware and logical corruption, since it exists outside the live, continuously replicated environment.

The distinction worth repeating: HA and stretched clusters protect availability. Backup and replication protect recoverability. A DR posture that leans entirely on one or the other has a gap, and that gap doesn’t usually surface until the day it matters. The other variable that never shows up on an architecture diagram is whether the plan has actually been tested. A replication schedule or backup job that has never been through a real failover test is an assumption, not a plan.

Where VxRail’s HA design ends and a recovery partner starts

VxRail’s built-in high availability works well for what it’s designed to catch: single node failures, and multiple failures up to whatever the storage policy accounts for. Organizations get exposed just outside that design: a second failure during a resync window, a stretched cluster whose witness link was never properly validated, a lifecycle update that leaves the cluster in a state HA wasn’t built to resolve, or a DR runbook that looks complete on paper and has never been tested under real failure conditions.

Maven has been the team called in after that gap opened, not before. In one recent engagement, a VxRail Manager update left a production cluster in an inconsistent state that Dell’s standard support process could not resolve inside the customer’s required timeframe. Maven engineers recovered the cluster and stabilized the lifecycle process going forward. Read the full VxRail lifecycle upgrade failure case study.

If your VxRail environment’s HA and DR design hasn’t been stress-tested against a real failure scenario, or a past incident exposed a gap between what you assumed the platform covered and what it actually did, contact Maven for an engineering-led conversation. The full scope of Maven’s VxRail recovery work is on the HCI Recovery and Support page, alongside more detail on Maven’s Dell VxRail support program.

FAQ: VxRail High Availability and Disaster Recovery