DELL EMC VxRAIL™ PLANNING GUIDE FOR

VIRTUAL SAN STRETCHED CLUSTER

ABSTRACT
This planning guide provides best practices and requirements for using stretched
cluster with an external vCenter Server specifically with VxRail appliances.

November 2016

WHITE PAPER
Table of Contents
Audience 3
Overview 3
Environment Requirements 4
VxRail and vSphere versions Requirements 4
Network topology 4
Layer 2 and Layer 3 support ............................................................................. 4
vSphere & Virtual SAN ...................................................................................... 5
Fault Domains ................................................................................................... 5
VxRail Cluster Nodes ........................................................................................ 5
Witness host ...................................................................................................... 5

Networking & Latency 6

Supported geographical distances .................................................................... 6
Data site to data site network latency ................................................................ 6
Data site to data site bandwidth ........................................................................ 6
Data Site to witness network latency ................................................................. 6
Data Site to witness network bandwidth ............................................................ 6
Inter-site MTU consistency ................................................................................ 6
Connectivity ....................................................................................................... 7

Conclusion 7

2
Audience
This guide is intended for customers, EMC and Business Partner Sales teams, and implementation
professionals to understand the requirements for Stretched Cluster support with the Dell EMC VxRail
Appliance. EMC services are required for implementation of Stretched Clusters.

This document is not intended to replace the implementation guide or bypass the service
implementation required for Stretched Clusters. Customers who attempt to set-up Stretch Clusters on
their own will invalidate support.

Overview
This planning guide provides best practices and requirements for using stretched cluster with an
external vCenter Server specifically with VxRail appliances. Additional technical information and
overview of on the benefits of Stretched Cluster can be found in the VSAN 6.2 Stretched Cluster
Guide.

http://www.vmware.com/files/pdf/products/vsan/VMware-Virtual-SAN-6.2-Stretched-Cluster-Guide.pdf

3
Environment Requirements
 Minimum of two VxRail clusters running VxRail 3.5 release or higher
 Each cluster has minimum of 4 nodes (VxRail 3.5) or 3 nodes (VxRail 4.0 or higher).
 Each data site must have an identical number of nodes
 Each VxRail appliance must be connected to an external vCenter Server version 6.0 U2 on a
separate, non-VxRail host
o External VC required customer provide additional virtual center license or running at
customer existing virtual center, VxRail embedded virtual center license cannot transfer
to external virtual center.
 Before VxRail first run initial configuration:
o You must create a vCenter User Account for VxRail via Web Client (16 characters or
less)
o You must create a Datacenter Name on the vCenter Server for VxRail
o A Witness Host (OVA version 6.2) must be installed on a separate site as part of the
installation engagement

VxRail and vSphere versions Requirements

 VxRail 3.5 requires vSphere 6.0 Update 2 or greater.
o vSphere 6.0 U2 integration
o ESXi 6.0.0 U2 GA ISO build 3620759
o VCSA or Windows version 6.0 U2 GA ISO build 3634788
o VxRail Manager 3.5 build 3936871 (or higher)
o vSphere Enterprise Plus (or Enterprise) Licenses

 VxRail 4.0 requires vSphere 6.0 Update 3

o vSphere6.0 U2 integration
o ESXi 6.0.0 U2(Patch 3) GA ISO build 4192238
o VCSA 6.0 U2 GA ISO build 3634788
o VxRail Manager 4.0 build 4550943
o vSphere Enterprise Plus (or Enterprise) Licenses

Network topology
Layer 2 and Layer 3 support
A stretched cluster in VxRail requires Layer 2 connectivity between date sites. Connectivity between
the data sites and the witness may be Layer 3. Figure 1 illustrates a supported configuration.

4
 Figure 1 VxRail Supported Topology

Note: Other topologies are noted in the VSAN 6.2 Stretched Cluster Guide. At this time, only the
above topology is supported with VxRail appliance.

vSphere & Virtual SAN

For Virtual SAN stretched cluster functionality on VxRail, vSphere Distributed Resource Scheduler
(DRS) is required. DRS will provide initial placement assistance, and automatically migrate virtual
machines to their corrected site in accordance to Host/VM affinity rules. It can also help locate virtual
machines to their correct site when a site recovers after a failure.

Fault Domains
Fault domains (FD) provide the core functionality of Virtual SAN Stretched Cluster. The maximum
number of fault domains in a Virtual SAN Stretched Cluster is 3. The first Fault Domain can be
referred as “Preferred” data site, the second Fault Domain can be referred as “Secondary” data site
and the third Fault Domain is the witness host site. It is important to keep utilization per data site
below 50% to ensure proper availability should either the Preferred or Secondary site go offline.
Additionally, Virtual SAN storage efficiency feature RAID-6 erasure coding is not supported with
stretch cluster.

VxRail Cluster Nodes

Virtual SAN Stretched Cluster are deployed across 2 sites in an Active/Active configuration. An
identical number of ESXi hosts are required to ensure a balanced distribution of resources.
Each data site is configured as a Fault Domain. An externally available third site, houses a Witness
appliance, which makes up the third Fault Domain.

The minimum supported configuration is 3+3+1 (6 nodes+1 witness). This configuration is comprised
of 3 VxRail nodes in the Preferred Site, 3 VxRail nodes in the Secondary Site, and a Witness node in
a third site. (Note- Customers running VxRail 3.5 the minimum supported configuration is 4+4+1.) The
maximum supported configuration is 15+15+1 (31 nodes+1 witness).

Witness host
Each Virtual SAN Stretched Cluster configuration requires a Witness host. The Witness must reside
on a third site that has independent paths to each data site. While the Witness host must be part of
5
the same vCenter as the hosts in the data sites, it must not be part of the cluster that the hosts are
part of.

A Virtual SAN Witness Appliance, or a physical host, can be used for the Witness function. The
Virtual SAN Witness Appliance includes licensing, while a physical host would still need to be
licensed accordingly. If you are using the Witness host OVA file it came with its own license thus it will
not consume vSphere license but the physical host itself need its own license. The Witness host can
be installed on vSphere 5.5.

Networking & Latency

Supported geographical distances
For VMware Virtual SAN Stretched Clusters, support is based on network latency and bandwidth
requirements, rather than distance. The key requirement is the actual latency numbers between sites.

Data site to data site network latency

Latency or RTT (Round Trip Time) between sites hosting virtual machine objects should not be
greater than 5msec (< 2.5msec one-way).

Data site to data site bandwidth

Bandwidth between sites hosting virtual machine objects will be workload dependent. For most
workloads, VMware recommends a minimum of 10Gbps or greater bandwidth between sites.

Data Site to witness network latency

In most Virtual SAN Stretched Cluster configurations, latency or RTT (Round Trip Time) between
sites hosting VM objects and the witness nodes should not be greater than 200msec (100msec one-
way).

The latency to the witness is dependent on the number of objects in the cluster. VMware
recommends that on Virtual SAN Stretched Cluster configurations up to 10+10+1, a latency of less
than or equal to 200 milliseconds is acceptable, although if possible, a latency of less than or equal to
100 milliseconds is preferred. For configurations that are greater than 10+10+1, VMware
recommends a latency of less than or equal to 100 milliseconds is required.

Data Site to witness network bandwidth

Bandwidth between sites hosting VM objects and the witness nodes are dependent on the number of
objects residing on Virtual SAN. It is important to size data site to witness bandwidth appropriately for
both availability and growth. A standard rule of thumb is 2Mbps for every 1000 objects on Virtual
SAN.

Inter-site MTU consistency

It is important to maintain a consistent MTU (maximum transmission unit) size between data nodes
and the witness in a Stretched Cluster configuration. Ensuring that each VMkernel interface
designated for Virtual SAN traffic, is set to the same MTU size will prevent traffic fragmentation. The
Virtual SAN Health Check checks for a uniform MTU size across the Virtual SAN data network, and
reports on any inconsistencies.

6
Connectivity
 Management network: connectivity to all 3 sites
 VM network: connectivity between the data sites (the witness will not run virtual machines that
are deployed on the VSAN cluster)
 vMotion network: connectivity between the data sites (virtual machines will never be migrated
from a data host to the witness host)
 Virtual SAN network: connectivity to all 3 sites

Conclusion
This document provides quick guidance on stretched cluster pertaining to VxRail. For more
information, we encourage to read VSAN 6.2 Stretched Cluster Guide.

In short, Virtual SAN stretched cluster feature is available in VxRail Appliance version 3.5 and later. It
creates a stretched cluster between two geographically separate sites, synchronously replicating data
between sites, and enabling enterprise-level availability. The stretched cluster feature allows for an
entire site failure to be tolerated, with no data loss and near zero downtime.

7
The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect
to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

EMC2, EMC, the EMC logo, VxRail are registered trademarks or trademarks of EMC Corporation in the United States and other
countries. All other trademarks used herein are the property of their respective owners. © Copyright 2016 EMC Corporation. All rights
reserved. Published in the USA. 10/16. Technical Guide H15275.2

EMC believes the information in this document is accurate as of its publication date. The information is subject to change without
notice.

EMC is now part of the Dell group of companies.