Virtual SAN - Fault Domain Design Considerations

Fault Domains (FD) in Virtual SAN environment is a concept introduced to provide a Rack/Chassis level redundancy.
In case your Virtual SAN Cluster spans across racks and server chassis (Converged or Hyper-Converged) and you want to hosts to be protected against rack or chassis failure, then you must create Fault Domains and add one or more hosts to each fault domain.

VSAN follows a distributed data locality algorithm to distribute the constructs of an object (replicas/witness/parity etc.) across disk groups of different servers. 

But with additional configuration of Fault Domains, we can enforce VSAN to also distribute the constructs across servers placed in different racks a.k.a Fault Domains than just distributing it across servers within same rack.
So with an appropriate FD design, we can achieve resiliency across rack failures as well.

What do we need to do this.
Firstly we need to ensure that our VSAN Fault Domain configuration matches our server placement in racks.
Let me explain with an example.

Consider below scenario. We have 8 Rack servers forming a VSAN cluster.
I would need a minimum of 3 Fault Domains to be begin with. That's because Fault Domain count = 2 x n + 1
where, n = No. of failure to tolerate

However for best results 4 or more Fault Domains are recommended.

So let's take 4 Fault Domains in the example.

The Blue VM is running with a Storage Policy of FTT = 1
The Orange VM is running with a Storage Policy of Erasure Coding = RAID 5

Since we have configured Fault Domains to include respective ESXis as shown on the right side of the image, you can see the data block for an object are written across the fault domains.
However in absence of FD configuration matching server placement, the data might have got written across servers within the same rack.

This is just an illustration, but to know the exact distribution of the Object blocks, use vSphere Web client go to Virtual Machine -> Monitor -> Policy...Click the Object and check Physical Disk Placement.


For balanced storage loads consider below points as guidelines for configuring Fault Domains

• Minimum no. of Fault Domains in a VSAN Cluster is 3. For best results configure 4 or more fault domains
• Possibly assign same no. of hosts to each fault domain
• Use Hosts with uniform configuration (from disk and compute perspective)
• Dedicate one Fault Domain of free capacity for rebuilding data after a failure.