Vce 340 Gen3 3 Architecture Overview

www.vce.
com
VCE Vblock® and VxBlock™

Systems 340 Gen 3.3
Architecture Overview
Document revision 3.8

August 2015
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Contents
Contents
Revision history..........................................................................................................................................4
Introduction................................................................................................................................................. 5
Accessing VCE documentation.................................................................................................................6
System overview.........................................................................................................................................7
System architecture and components.................................................................................................... 7
Base configurations and scaling.............................................................................................................9
Connectivity overview...........................................................................................................................11
Segregated network architecture................................................................................................... 12
Unified network architecture.......................................................................................................... 17
Compute layer overview...........................................................................................................................24

Compute overview................................................................................................................................24
Cisco Unified Computing System.........................................................................................................24
Cisco Unified Computing System fabric interconnects.........................................................................25
Cisco Trusted Platform Module............................................................................................................ 25
Scaling up compute resources............................................................................................................. 26
VCE bare metal support policy.............................................................................................................27
Disjoint layer 2 configuration................................................................................................................ 28
Storage layer............................................................................................................................................. 30
Storage overview..................................................................................................................................30
EMC VNX series storage arrays.......................................................................................................... 30
Replication............................................................................................................................................32
Scaling up storage resources...............................................................................................................32
Storage features support......................................................................................................................35
Network layer............................................................................................................................................ 37
Network overview................................................................................................................................. 37
IP network components........................................................................................................................37
Port utilization.......................................................................................................................................38
Cisco Nexus 5548UP Switch - segregated networking..................................................................39
Cisco Nexus 5596UP Switch - segregated networking..................................................................40
Cisco Nexus 5548UP Switch – unified networking........................................................................ 40
Cisco Nexus 5596UP - unified networking.....................................................................................42
Cisco Nexus 9396PX Switch - segregated networking..................................................................43
Storage switching components............................................................................................................ 44
Virtualization layer....................................................................................................................................46
Virtualization overview..........................................................................................................................46
2
© 2013-2015 VCE Company, LLC.
All Rights Reserved.
Contents VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
VMware vSphere Hypervisor ESXi.......................................................................................................46

VMware vCenter Server....................................................................................................................... 47
Management..............................................................................................................................................50
Management components overview.....................................................................................................50
Management hardware components....................................................................................................50
Management software components..................................................................................................... 51
Management network connectivity....................................................................................................... 52
System infrastructure...............................................................................................................................57
VCE Systems descriptions................................................................................................................... 57
Cabinets overview................................................................................................................................ 58
Intelligent Physical Infrastructure appliance......................................................................................... 58
Power options.......................................................................................................................................58
Configuration descriptions...................................................................................................................... 60
VCE Systems with EMC VNX8000...................................................................................................... 60
Sample configurations............................................................................................................................. 75
Sample VCE System with EMC VNX8000........................................................................................... 75
Sample VCE System with EMC VNX5800........................................................................................... 81
Sample VCE System with EMC VNX5800 (ACI ready)........................................................................86
Additional references............................................................................................................................... 92
Virtualization components.................................................................................................................... 92
Compute components.......................................................................................................................... 92
Network components............................................................................................................................93
Storage components............................................................................................................................ 93
3
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Revision history
Revision history
Date VCE System Document Description of changes
revision
August 2015 Gen 3.3 3.8 Updated to include the VxBlock System 340. Added support
for VMware vSphere 6.0 with VMware VDS on the VxBlock
System and for existing Vblock Systems.
Added information on Intelligent Physical Infrastructure (IPI)
appliance
February 2015 Gen 3.2 3.7 Added support for Cisco B200 M4 Blade.
December 2014 Gen 3.2 3.6 Added support for AMP-2HA.
September 2014 Gen 3.2 3.5 Modified elevations and removed aggregate section.
July 2014 Gen 3.2 3.4 Added support for VMware VDS.
May 2014 Gen 3.2 3.3 • Updated for Cisco Nexus 9396 Switch and 1500 drives
for EMC VNX8000
• Added support for VMware vSphere 5.5
January 2014 Gen 3.1 3.2 Updated elevations for AMP-2 reference.
November 2013 Gen 3.1 3.1 Updated network connectivity management illustration.
October 2013 Gen 3.1 3.0 Gen 3.1 release
4
Introduction VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
Introduction
This document describes the high-level design of the VCE System. This document also describes the
hardware and software components that VCE includes in the VCE System.
In this document, the Vblock System and VxBlock System are referred to as VCE Systems.
The VCE Glossary provides terms, definitions, and acronyms that are related to VCE.
To suggest documentation changes and provide feedback on this book, send an e-mail to
docfeedback@vce.com. Include the name of the topic to which your feedback applies.
Related information
Accessing VCE documentation (see page 6)
5
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Accessing VCE documentation
Accessing VCE documentation

Select the documentation resource that applies to your role.
Role Resource
Customer support.vce.com
A valid username and password are required. Click VCE Download Center to access the
technical documentation.
Cisco, EMC, VMware partner.vce.com

employee, or VCE A valid username and password are required.
Partner
VCE employee sales.vce.com/saleslibrary

or
vblockproductdocs.ent.vce.com
6
System overview VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
System overview
System architecture and components

This section describes the architecture and components of the VCE Systems.
VCE Systems have a number of features including:
• Optimized, fast delivery configurations based on the most commonly purchased components
• Standardized cabinets with multiple North American and international power solutions
• Block (SAN) and unified storage options (SAN and NAS)
• Support for multiple features of the EMC operating environment for EMC VNX arrays
• Granular, but optimized compute and storage growth by adding predefined kits and packs
• Second generation of the Advanced Management Platform (AMP-2) for management
• Unified network architecture provides the option to leverage Cisco Nexus switches to support IP
and SAN without the use of Cisco MDS switches.
VCE Systems contain the following key hardware and software components:
Resource Components
VCE Systems • VCE Vision™ Intelligent Operations System Library

management • VCE Vision™ Intelligent Operations Plug-in for vCenter
• VCE Vision™ Intelligent Operations Compliance Checker
• VCE Vision™ Intelligent Operations API for System Library
• VCE Vision™ Intelligent Operations API for Compliance Checker
7
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview System overview
Resource Components
Virtualization • VMware vSphere Server Enterprise Plus

and • VMware vSphere ESXi
management
• VMware vCenter Server
• VMware vSphere Web Client
• VMware Single Sign-On (SSO) Service (version 5.1 and higher)
• Cisco UCS C220 Server for AMP-2
• EMC PowerPath/VE
• Cisco UCS Manager
• EMC Unisphere Manager
• EMC VNX Local Protection Suite
• EMC VNX Remote Protection Suite
• EMC VNX Application Protection Suite
• EMC VNX Fast Suite
• EMC VNX Security and Compliance Suite
• EMC Secure Remote Support (ESRS)
• EMC PowerPath Electronic License Management Server (ELMS)
• Cisco Data Center Network Manager for SAN
Compute • Cisco UCS 5108 Server Chassis

• Cisco UCS B-Series M3 Blade Servers with Cisco UCS VIC 1240, optional port expander or
Cisco UCS VIC 1280
• Cisco UCS B-Series M4 Blade Servers with Cisco UCS VIC 1340, optional port expander or
Cisco UCS VIC 1380
• Cisco UCSB-MLOM-PT-01 - Port Expander for 1240 VIC
• Cisco UCS 2208XP fabric extenders or Cisco UCS 2204XP fabric extenders
• Cisco UCS 2208XP Fabric Extenders with FET Optics or Cisco UCS 2204XP Fabric
Extenders with FET Optics
• Cisco UCS 6248UP Fabric Interconnects or Cisco UCS 6296UP Fabric Interconnects
Network • Cisco Nexus 3048 Switches

• Cisco Nexus 5548UP Switches, Cisco Nexus 5596UP switches, or Cisco Nexus 9396PX
Switches
• (Optional) Cisco MDS 9148 Multilayer Fabric Switch
• (Optional) Cisco Nexus 1000V Series Switches
• (Optional) VMware vSphere Distributed Switch (VDS) (VMware vSphere version 5.5 and
higher)
• (Optional) VMware NSX Virtual Networking
Storage • EMC VNX storage array (5400, 5600, 5800, 7600, 8000) running the VNX Operating
Environment
• (Optional) EMC unified storage (NAS)
8
VCE Systems have a different scale point based on compute and storage options. VCE Systems can
support block and/or unified storage protocols.
The VCE Release Certification Matrix provides a list of the certified versions of components for VCE
Systems. For information about VCE System management, refer to the VCE Vision™ Intelligent
Operations Technical Overview.
The VCE Integrated Data Protection Guide provides information about available data protection solutions.
Related information
EMC VNX series storage arrays (see page 30)
Base configurations and scaling

VCE Systems have a base configurations that contain a minimum set of compute and storage
components, as well as fixed network resources that are integrated within one or more 19 inch, 42U
cabinets.
Within the base configuration, the following hardware aspects can be customized:
Hardware How it can be customized
Compute blades Cisco UCS B-Series blade types include all supported VCE blade
configurations.
Compute chassis Cisco UCS Server Chassis

SIxteen chassis maximum for VCE Systems with EMC VNX8000, VCE
Systems with EMC VNX7600, VCE Systems with EMC VNX5800
Eight chassis maximum for VCE Systems with EMC VNX5600
Two chassis maximum for VCE Systems with EMC VNX5400
Edge servers Four to six Cisco UCS B-series Blade Servers, including the B200 M4 with VIC
(with optional VMware NSX) 1340 and VIC 1380.
For more information, see the VCE VxBlock™ Systems for VMware NSX
Architecture Overview.
Storage hardware Drive flexibility for up to three tiers of storage per pool, drive quantities in each
tier, the RAID protection for each pool, and the number of disk array enclosures
(DAEs).
Storage EMC VNX storage - block only or unified (SAN and NAS)
9
Hardware How it can be customized
Supported disk drives FastCache

100/200GB SLC SSD
Tier 0
100/200GB SLC SSD
100/200/400GB eMLC SSD
Tier 1
300/600GB 15K SAS
600/900GB 10K SAS
Tier 2
1/2/3/4 TB 7.2K NL-SAS
Supported RAID types Tier 0: RAID 1/0 (4+4), RAID 5 (4+1) or (8+1)
Tier 1: RAID 1/0 (4+4), RAID 5 (4+1) or (8+1), RAID 6 (6+2), (12 +2)*, (14+2)**
Tier 2: RAID 1/0 (4+4), RAID 5 (4+1) or (8+1), RAID 6 (6+2), (12 +2)*, (14+2)**
*file virtual pool only
**block virtual pool only
Management hardware options The second generation of the Advanced Management Platform (AMP-2)
centralizes management of VCE System components. AMP-2 offers minimum
physical, redundant physical, and highly available models. The standard option
for this platform is the minimum physical model.
The optional VMware NSX feature requires AMP-2HA Performance.
Data Mover enclosure (DME) Available on all VCE Systems. Additional enclosure packs can be added for
packs additional X-Blades on VCE Systems with EMC VNX8000, VCE Systems with
EMC VNX7600, and VCE Systems with EMC VNX5800.
Together, the components offer balanced CPU, I/O bandwidth, and storage capacity relative to the
compute and storage arrays in the system. All components have N+N or N+1 redundancy.
These resources can be scaled up as necessary to meet increasingly stringent requirements. The
maximum supported configuration differs from model to model. To scale up compute resources, add
blade packs and chassis activation kits.
To scale up storage resources, add RAID packs, DME packs, and DAE packs. Optionally, expansion
cabinets with additional resources can be added.
VCE Systems are designed to keep hardware changes to a minimum if the storage protocol is changed
after installation (for example, from block storage to unified storage). Cabinet space can be reserved for
all components that are needed for each storage configuration (Cisco MDS switches, X-Blades, etc.)
ensuring that network and power cabling capacity for these components is in place.
Related information
EMC VNX series storage arrays (see page 30)
10
Scaling up compute resources (see page 26)
Scaling up storage resources (see page 32)
Management components overview (see page 50)
Replication (see page 32)
Connectivity overview
This topic describes the components and interconnectivity within the VCE Systems.
These components and interconnectivity are conceptually subdivided into the following layers:
Layer Description
Compute Contains the components that provide the computing power within a VCE System. The Cisco UCS blade
servers, chassis, and fabric interconnects belong to this layer.
Storage Contains the EMC VNX storage component.
Network Contains the components that provide switching between the compute and storage layers within a VCE
System, and between a VCE System and the network. Cisco MDS switches and the Cisco Nexus
switches belong to this layer.
All components incorporate redundancy into the design.
Segregated network architecture and unified network architecture
In the segregated network architecture, LAN and SAN connectivity is segregated into separate switches
within the VCE System. LAN switching uses the Cisco Nexus switches. SAN switching uses the Cisco
MDS 9148 Multilayer Fabric Switch.
In the unified network architecture, LAN and SAN switching is consolidated onto a single network device
(Cisco Nexus 5548UP switches or Cisco Nexus 5596UP switches) within the VCE System. This removes
the need for a Cisco MDS SAN switch.
Note: The optional VMware NSX feature uses the Cisco Nexus 9396 switches for LAN switching. For
more information, see the VCE VxBlock™ Systems for VMware NSX Architecture Overview.
In addition, all management interfaces for infrastructure power outlet unit (POU), network, storage, and
compute devices are connected to redundant Cisco Nexus 3048 switches. These switches provide
connectivity for Advanced Management Platform (AMP-2) and egress points into the management stacks
for the VCE Systems components.
Related information
11
Segregated network architecture (see page 12)
Unified network architecture (see page 17)
Segregated network architecture

This topic shows VCE Systems segregated network architecture for block, SAN boot, and unified storage.
Block storage configuration
The following illustration shows a block-only storage configuration for VCE Systems with the X-Blades
absent from the cabinets. However, space can be reserved in the cabinets for these components
12
(including optional EMC RecoverPoint Appliances). This design makes it easier to add the components
later if there is an upgrade to unified storage.
13
SAN boot storage configuration
In all VCE Systems configurations, the VMware vSphere ESXi blades boot over the Fibre Channel (FC)
SAN. In block-only configurations, block storage devices (boot and data) are presented over FC through
the SAN. In a unified storage configuration, the boot devices are presented over FC and data service can
be either block devices (SAN) or presented as NFS data stores (NAS). In a file-only configuration, the
boot devices are presented over FC and data devices are through NFS shares. Storage can also be
presented directly to the VMs as CIFS shares.
14
The following illustration shows the components (highlighted in a red, dotted line) that are leveraged to
support SAN booting in VCE Systems:
15
Unified storage configuration
In a unified storage configuration, the storage processors also connect to X-Blades over FC. The X-
Blades connect to the Cisco Nexus switches within the network layer over 10 GbE, as shown in the
following illustration:
16
Related information
Connectivity overview (see page 11)
Unified network architecture

The topic provides an overview of the block storage, SAN boot storage, and unified storage
configurations for the unified network architecture.
With unified network architecture, access to both block and file services on the EMC VNX is provided
using the Cisco Nexus 5548UP Switch or Cisco Nexus 5596UP Switch. The Cisco Nexus 9396PX Switch
is not supported in unified network architecture.
17
Block storage configuration
The following illustration shows a block-only storage configuration in VCE Systems:
In this example, there are no X-Blades providing NAS capabilities. However, space can be reserved in
the cabinets for these components (and including the optional EMC RecoverPoint Appliance). This design
makes it easier to add the components later if there is an upgrade to unified storage.
18
In a unified storage configuration for block and file, the storage processors also connect to X-Blades over
Fibre Channel (FC). The X-Blades connect to the Cisco Nexus switches within the network layer over 10
GbE.
SAN boot storage configuration
In all VCE Systems configurations, VMware vSphere ESXi blades boot over the FC SAN. In block-only
configurations, block storage devices (boot and data) are presented over FC through the Cisco Nexus
unified switch. In a unified storage configuration, the boot devices are presented over FC and data
devices can be either block devices (SAN) or presented as NFS data stores (NAS). In a file-only
configuration, boot devices are presented over FC, and data devices over NFS shares. The remainder of
the storage can be presented either as NFS or as VMFS data stores. Storage can also be presented
directly to the VMs as CIFS shares.
19
The following illustration shows the components that are leveraged to support SAN booting in VCE
Systems:
20
Unified storage configuration
In a unified storage configuration, the storage processors also connect to X-Blades over FC. The X-
Blades connect to the Cisco Nexus switches within the network layer over 10 GbE.
21
The following illustration shows a unified storage configuration for VCE Systems:
22
Related information
Segregated network architecture (see page 12)
23
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Compute layer
Compute layer
Compute overview
This topic provides an overview of the compute components for the VCE Systems.
Cisco UCS B-Series Blades installed in the Cisco UCS chassis provide computing power within VCE
Systems.
Fabric extenders (FEX) within the Cisco UCS chassis connect to Cisco fabric interconnects over
converged Ethernet. Up to eight 10 GbE ports on each Cisco UCS fabric extender connect northbound to
the fabric interconnects, regardless of the number of blades in the chassis. These connections carry IP
and storage traffic.
VCE has reserved some of these ports to connect to upstream access switches within the VCE Systems.
These connections are formed into a port channel to the Cisco Nexus switch and carry IP traffic destined
for the external network 10 GbE links. In a unified storage configuration, this port channel can also carry
NAS traffic to the X-Blades within the storage layer.
Each fabric interconnect also has multiple ports reserved by VCE for Fibre Channel (FC) ports. These
ports connect to Cisco SAN switches. These connections carry FC traffic between the compute layer and
the storage layer. In a unified storage configuration, port channels carry IP traffic to the X-Blades for NAS
connectivity. For SAN connectivity, SAN port channels carrying FC traffic are configured between the
fabric interconnects and upstream Cisco MDS or Cisco Nexus switches.
Cisco Unified Computing System

This topic provides an overview of the Cisco Unified Compute System (UCS) data center platform that
unites compute, network, and storage access.
Optimized for virtualization, the Cisco UCS integrates a low-latency, lossless 10 Gb Ethernet unified
network fabric with enterprise-class, x86-based servers (the Cisco B-Series).
VCE Systems powered by Cisco UCS feature:
• Built-in redundancy for high availability
• Hot-swappable components for serviceability, upgrade, or expansion
• Fewer physical components than in a comparable system built piece by piece
• Reduced cabling
• Improved energy efficiency over traditional blade server chassis
The Vblock System Blade Pack Reference provides a list of supported Cisco UCS blades.
24
Compute layer VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
Related information
Cisco Unified Computing System fabric interconnects

The Cisco Unified Computing System (UCS) fabric interconnects provide network connectivity and
management capabilities to the Cisco UCS blades and chassis.
The Cisco UCS fabric interconnects provide the management and communication backbone for the
blades and chassis. The Cisco UCS fabric interconnects provide LAN and SAN connectivity for all blades
within their domain. Cisco UCS fabric interconnects are used for boot functions and offer line-rate, low-
latency, lossless 10 Gigabit Ethernet and Fibre Channel over Ethernet (FCoE) functions.
VCE Systems use Cisco UCS 6248UP Fabric Interconnects and Cisco UCS 6296UP Fabric
Interconnects. Single domain uplinks of 2, 4, or 8 between the fabric interconnects and the chassis are
provided with the Cisco UCS 6248UP Fabric Interconnects. Single domain uplinks of 4 or 8 between the
fabric interconnects and the chassis are provided with the Cisco UCS 6296UP Fabric Interconnects.
The optional VMware NSX feature uses Cisco UCS 6296UP Fabric Interconnects to accommodate the
port count needed for VMware NSX external connectivity (edges). For more information, see the VCE
VxBlock™ Systems for VMware NSX Architecture Overview.
Related information
Cisco Trusted Platform Module

Cisco Trusted Platform Module (TPM) is a computer chip that securely stores artifacts such as
passwords, certificates, or encryption keys that authenticate the VCE System.
Cisco TPM provides authentication and attestation services that provide safer computing in all
environments. Cisco TPM is a computer chip that securely stores artifacts such as passwords,
certificates, or encryption keys that authenticate the VCE System. Cisco TPM provides authentication and
attestation services that provide safer computing in all environments.
Cisco TPM is available by default within the VCE System as a component within the Cisco UCS B-Series
M3 Blade Servers and Cisco UCS B-Series M4 Blade Servers, and is shipped disabled. The Vblock
System Blade Pack Reference contains additional information about Cisco TPM.
VCE supports only the Cisco TPM hardware. VCE does not support the Cisco TPM functionality. Because
making effective use of the Cisco TPM involves the use of a software stack from a vendor with significant
experience in trusted computing, VCE defers to the software stack vendor for configuration and
operational considerations relating to the Cisco TPMs.
25
Related information
www.cisco.com
Scaling up compute resources

This topic describes what you can add to your VCE System to scale up compute resources.
To scale up compute resources, you can add uplinks, blade packs, and chassis activation kits to enhance
Ethernet and Fibre Channel (FC) bandwidth either when VCE Systems are built, or after they are
deployed.
The following table shows the maximum chassis and blade quantities that are supported for VCE
Systems with EMC VNX5400, VCE Systems with EMC VNX5600, VCE Systems with EMC VNX5800,
VCE Systems with EMC VNX7600, and VCE Systems with EMC VNX8000:
VCE Systems 2-link Cisco UCS 4-link Cisco 4-link Cisco 8-link Cisco 8-link Cisco
with 6248UP Cisco UCS 6248UP UCS 6296UP UCS 6248UP UCS 6296UP
UCS 2204XP IOM Cisco UCS Cisco UCS Cisco UCS Cisco UCS
2204XP IOM 2204XP IOM 2208XP IOM 2208XP IOM
EMC VNX8000 16(128) 8(64) 16(128) 4(32) 8(64)
EMC VNX7600 16(128) 8(64) 16(128) 4(32) 8(64)
EMC VNX5800 16(128) 8(64) 16(128) 4(32) 8(64)
EMC VNX5600 N/A 8(64) 8(64) 4(32) 8(64)
EMC VNX5400 N/A 2(16) N/A N/A N/A
Ethernet and FC I/O bandwidth enhancement
For VCE Systems with EMC VNX5600, EMC VNX5800, EMC VNX7600, and EMC VNX8000, the
Ethernet I/O bandwidth enhancement increases the number of Ethernet uplinks from the Cisco UCS
6296UP fabric interconnects to the network layer to reduce oversubscription. To enhance Ethernet I/O
bandwidth performance increase the uplinks between the Cisco UCS 6296UP fabric interconnects and
the Cisco Nexus 5548UP Switch for segregated networking, or the Cisco Nexus 5596UP Switch for
unified networking.
FC I/O bandwidth enhancement increases the number of FC links between the Cisco UCS 6248UP fabric
interconnects or Cisco UCS 6296UP fabric interconnects and the SAN switch, and from the SAN switch to
the EMC VNX storage array. The FC I/O bandwidth enhancement feature is supported on VCE Systems
with EMC VNX5800, EMC VNX7600, and EMC VNX8000.
Blade packs
Cisco UCS blades are sold in packs of two and include two identical Cisco UCS blades. The base
configuration of each VCE System includes two blade packs. The maximum number of blade packs
26
depends on the type of VCE System. Each blade type must have a minimum of two blade packs as a
base configuration and then can be increased in single blade pack increments thereafter.
Each blade pack is added along with the following license packs:
• VMware vSphere ESXi
• Cisco Nexus 1000V Series Switches (Cisco Nexus 1000V Advanced Edition only)
• EMC PowerPath/VE
Note: License packs for VMware vSphere ESXi, Cisco Nexus 1000V Series Switches, and EMC
PowerPath are not available for bare metal blades.
The Vblock System Blade Pack Reference provides a list of supported Cisco UCS blades.
Chassis activation kits
The power supplies and fabric extenders for all chassis are populated and cabled, and all required
Twinax cables and transceivers are populated.
As more blades are added and additional chassis are required, chassis activation kits (CAK) are
automatically added to an order. The kit contains software licenses to enable additional fabric
interconnect ports.
Only enough port licenses for the minimum number of chassis to contain the blades are ordered. Chassis
activation kits can be added up-front to allow for flexibility in the field or to initially spread the blades
across a larger number of chassis.
Related information
VCE bare metal support policy

Since many applications cannot be virtualized due to technical and commercial reasons, VCE Systems
support bare metal deployments, such as non-virtualized operating systems and applications.
While it is possible for VCE Systems to support these workloads (with caveats noted below), due to the
nature of bare metal deployments, VCE is able to provide only “reasonable effort" support for systems
that comply with the following requirements:
• VCE Systems contain only VCE published, tested, and validated hardware and software
components. The VCE Release Certification Matrix provides a list of the certified versions of
components for VCE Systems.
• The operating systems used on bare-metal deployments for compute and storage components
must comply with the published hardware and software compatibility guides from Cisco and EMC.
27
• For bare metal configurations that include other hypervisor technologies (Hyper-V, KVM, etc.),
those hypervisor technologies are not supported by VCE. VCE Support is only provided on
VMware Hypervisors.
VCE reasonable effort support includes VCE acceptance of customer calls, a determination of whether a
VCE Systems is operating correctly, and assistance in problem resolution to the extent possible.
VCE is unable to reproduce problems or provide support on the operating systems and applications
installed on bare metal deployments. In addition, VCE does not provide updates to or test those operating
systems or applications. The OEM support vendor should be contacted directly for issues and patches
related to those operating systems and applications.
Related information
Disjoint layer 2 configuration

In the disjoint layer 2 configuration, traffic is split between two or more different networks at the fabric
interconnect to support two or more discrete Ethernet clouds. The Cisco UCS servers connect to two
different clouds.
Upstream disjoint layer 2 networks allow two or more Ethernet clouds that never connect to be accessed
by servers or VMs located in the same Cisco UCS domain.
28
The following illustration provides an example implementation of disjoint layer 2 networking into a Cisco
UCS domain:
Virtual port channels (VPCs) 101 and 102 are production uplinks that connect to the network layer of VCE
Systems. Virtual port channels 105 and 106 are external uplinks that connect to other switches.
If you use Ethernet performance port channels (PC 103 and 104 by default), port channels 101 through
104 are assigned to the same VLANs.
29
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Storage layer
Storage layer
Storage overview
EMC VNX series are fourth-generation storage platforms that deliver industry-leading capabilities. They
offer a unique combination of flexible, scalable hardware design and advanced software capabilities that
enable them to meet the diverse needs of today’s organizations.
EMC VNX series platforms support block storage and unified storage. The platforms are optimized for
VMware virtualized applications. They feature flash drives for extendable cache and high performance in
the virtual storage pools. Automation features include self-optimized storage tiering, and application-
centric replication.
Regardless of the storage protocol implemented at startup (block or unified), VCE Systems can include
cabinet space, cabling, and power to support the hardware for all of these storage protocols. This
arrangement makes it easier to move from block storage to unified storage with minimal hardware
changes.
VCE Systems are available with:
• EMC VNX5400
• EMC VNX5600
• EMC VNX5800
• EMC VNX7600
• EMC VNX8000
Note: In all VCE Systems, all EMC VNX components are installed in VCE cabinets in VCE-specific
layout.
EMC VNX series storage arrays

This topic provides an overview of the EMC VNX series storage arrays.
The EMC VNX series storage arrays connect to dual storage processors (SPs) using 6Gb/s four-lane
serial attached SCSI (SAS). Each storage processor connects to one side of each two, four, eight, or
sixteen (depending on the VCE System) redundant pairs of four-lane x 6Gb/s serial attached SCSI (SAS)
buses, providing continuous drive access to hosts in the event of a storage processor or bus fault. Fibre
Channel (FC) expansion cards within the storage processors connect to the Cisco MDS switches in the
network layer over FC.
30
Storage layer VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
The storage layer in the VCE System consists of an EMC VNX storage array. Each EMC VNX model
contains some or all of the following components:
• The disk processor enclosure (DPE) houses the service processors for the EMC VNX5400, EMC
VNX5600, EMC VNX5800 and EMC VNX7600. The DPE provides slots for two service
processors, two battery backup units (BBU), and an integrated 25 slot disk array enclosure (DAE)
for 2.5" drives. Each SP provides support for up to 5 SLICs (small I/O cards).
• The EMC VNX8000 uses a service processor enclosure (SPE) and standby power supplies
(SPS). The SPE is a 4U enclosure with slots for two service processors, each supporting up to 11
SLICs. Each EMC VNX8000 includes two 2U SPS' that power the SPE and the vault DAE. Each
SPS contains two Li-ION batteries that require special shipping considerations.
• X-Blades (also known as data movers) provide file-level storage capabilities. These are housed in
data mover enclosures (DME). Each X-Blade connects to the network switches using 10G links
(either Twinax or 10G fibre).
• DAEs contain individual disk drives and are available in the following configurations:
— 2U model that can hold 25 2.5" disks
— 3U model that can hold 15 3.5" disks
EMC VNX5400
The EMC VNX5400 is a DPE-based array with two back-end SAS buses, up to four slots for front-end
connectivity, and support for up to 250 drives. It is available in both unified (NAS) and block
configurations.
EMC VNX5600
The EMC VNX5600 is a DPE-based array with up to six back-end SAS buses, up to five slots for front-
end connectivity, and support for up to 500 drives. It is available in both unified (NAS) and block
configurations.
EMC VNX5800
The EMC VNX5800 is a DPE-based array with up to six back-end SAS buses, up to five slots for front-
end connectivity, and support for up to 750 drives. It is available in a block configuration.
EMC VNX7600
The EMC VNX7600 is a DPE-based array with six back-end SAS buses, up to four slots for front-end
connectivity, and support for up to 1000 drives. It is available in a block configuration.
EMC VNX8000
The EMC VNX8000 comes in a different form factor from the other EMC VNX models. The EMC
VNX8000 is an SPE-based model with up to 16 back-end SAS buses, up to nine slots for front-end
connectivity, and support for up to 1500 drives. It is available in a block configuration.
31
Related information
Storage features support (see page 35)
Replication
This section describes how VCE Systems can be upgraded to include EMC RecoverPoint.
For block storage configurations, VCE Systems can be upgraded to include EMC RecoverPoint. This
replication technology provides continuous data protection and continuous remote replication for on-
demand protection and recovery to any point in time. EMC RecoverPoint advanced capabilities include
policy-based management, application integration, and bandwidth reduction. RecoverPoint is included in
the EMC Local Protection Suite and EMC Remote Protection Suite.
To implement EMC RecoverPoint within a VCE System, add two or more EMC RecoverPoint Appliances
(RPA) in a cluster to the VCE System. This cluster can accommodate approximately 80 MB/s sustained
throughput through each EMC RPA.
To ensure proper sizing and performance of an EMC RPA solution, VCE works with an EMC Technical
Consultant. They collect information about the data to be replicated, as well as data change rates, data
growth rates, network speeds, and other information that is needed to ensure that all business
requirements are met.
Scaling up storage resources

This topic describes what you can add to your VCE System to scale up storage resources.
To scale up storage resources, you can expand block I/O bandwidth between the compute and storage
resources, add RAID packs, and add disk-array enclosure (DAE) packs. I/O bandwidth and packs can be
added when VCE Systems are built and after they are deployed.
I/O bandwidth expansion
Fibre channel (FC) bandwidth can be increased in the VCE Systems with EMC VNX8000, VCE Systems
with EMC VNX7600, and VCE Systems with EMC VNX5800. This option adds an additional four FC
interfaces per fabric between the fabric interconnects and the Cisco MDS 9148 Multilayer Fabric Switch
(segregated network architecture) or Cisco Nexus 5548UP Switch or Switch Cisco Nexus 5596UP Switch
(unified network architecture). It also adds an additional four FC ports from the EMC VNX to each SAN
fabric.
This option is available for environments that require high bandwidth, block-only configurations. This
configuration requires the use of four storage array ports per storage processor that are normally
reserved for unified connectivity of the X-Blades.
32
RAID packs
Storage capacity can be increased by adding RAID packs. Each pack contains a number of drives of a
given type, speed, and capacity. The number of drives in a pack depends upon the RAID level that it
supports.
The number and types of RAID packs to include in VCE Systems are based upon the following:
• The number of storage pools that are needed.
• The storage tiers that each pool contains, and the speed and capacity of the drives in each tier.
The following table lists tiers, supported drive types, and supported speeds and capacities.
Note: The speed and capacity of all drives within a given tier in a given pool must be the same.
Tier Drive type Supported speeds and capacities
1 Solid-state Enterprise Flash drives (EFD) 100 GB SLC EFD

200 GB SLC EFD
100 GB eMLC EFD
200 GB eMLC EFD
400 GB eMLC EFD
2 Serial attached SCSI (SAS) 300 GB 10K RPM

600 GB 10K RPM
900 GB 10K RPM
300 GB 15K RPM
600 GB 15K RPM
3 Nearline SAS 1 TB 7.2K RPM

2 TB 7.2K RPM
3 TB 7.2K RPM
• The RAID protection level for the tiers in each pool. The following table describes each
supported RAID protection level. The RAID protection level for the different pools can vary.
RAID Description
protection
level
RAID 1/0 • A set of mirrored drives.

• Offers the best overall performance of the three supported RAID protection levels.
• Offers robust protection. Can sustain double-drive failures that are not in the same mirror set.
• Lowest economy of the three supported RAID levels since usable capacity is only 50% of raw
capacity.
33
RAID Description
protection
level
RAID 5 • Block-level striping with a single parity block, where the parity data is distributed across all of the
drives in the set.
• Offers the best mix of performance, protection, and economy.
• Has a higher write performance penalty than RAID 1/0 because multiple I/Os are required to
perform a single write.
• With single parity, can sustain a single drive failure with no data loss. Vulnerable to data loss or
unrecoverable read errors on a track during a drive rebuild.
• Highest economy of the three supported RAID levels. Usable capacity is 80% of raw capacity or
better.
RAID 6 • Block-level striping with two parity blocks, distributed across all of the drives in the set.
• Offers increased protection and read performance comparable to RAID 5.
• Has a significant write performance penalty because multiple I/Os are required to perform a
single write.
• Economy is very good. Usable capacity is 75% of raw capacity or better.
• EMC best practice for SATA and NL-SAS drives.
There are RAID packs for each RAID protection level/tier type combination. The RAID levels dictate the
number of drives that are included in the packs. RAID 5 or RAID 1/0 is for performance and extreme
performance tiers and RAID 6 is for the capacity tier. The following table lists RAID protection levels and
the number of drives in the pack for each level:
RAID protection level Number of drives per RAID pack
RAID 1/0 8 (4 data + 4 mirrors)
RAID 5 5 (4 data + 1 parity) or 9 (8 data + 1 parity)
RAID 6 8 (6 data + 2 parity), 14 (12 data + 2 parity)* or 16 (14 data + 2 parity)**
* file virtual pool only
**block virtual pool only
Disk array enclosure packs
If the number of RAID packs in VCE Systems is expanded, more disk array enclosures (DAEs) might be
required. DAEs are added in packs. The number of DAEs in each pack is equivalent to the number of
back-end buses in the EMC VNX array in the VCE System. The following table lists the number of buses
in the array and the number of DAEs in the pack for each VCE System:
VCE System Number of buses in the array Number of DAEs in the DAE pack
EMC VNX8000 8 or 16 8 or 16
EMC VNX7600 6 6
34
VCE System Number of buses in the array Number of DAEs in the DAE pack
EMC VNX5800 6 6
EMC VNX5600 2 or 6 2 or 6 (base includes DPE as the first DAE)
EMC VNX5400 2 2 (base includes DPE as the first DAE)
There are two types of DAEs: a 2U 25 slot DAE for 2.5" disks and 3U 15 slot DAE for 3.5" disks. A DAE
pack can contain a mix of DAE sizes, as long as the total DAEs in the pack equals the number of buses.
To ensure that the loads are balanced, physical disk will be spread across the DAEs in accordance with
best practice guidelines.
Storage features support

This topic presents additional storage features available on VCE Systems.
Support for array hardware or capabilities
The following table provides an overview of the support provided for EMC VNX operating environment for
new array hardware or capabilities:
Feature Description
NFS Virtual X-Blades – Provides security and segregation for service provider environmental clients.
VDM (Multi-LDAP
Support)
Data-in-place block When compression is enabled, thick LUNs are converted to thin and compressed in
compression place. RAID group LUNs are migrated into a pool during compression. There is no need
for additional space to start compression. Decompression temporarily requires additional
space, since it is a migration, and not an in-place decompression.
Compression for file/ Available file compression types:

display compression • Fast compression (default)
capacity savings
• Deep compression (up to 30% more space efficient, but slower and with higher
CPU usage)
Displays capacity savings due to compression to allow a cost/benefit comparison (space
savings versus performance impact).
EMC VNX snapshots EMC VNX snapshots are only for storage pools, not for RAID groups. Storage pools can
use EMC SnapView snapshots and EMC VNX snapshots at the same time.
Note: This feature is optional. Both types of snapshots have a seamless support
perspective. VCE relies on guidance from EMC best practices for different use
cases of EMC SnapView snapshots versus EMC VNX snapshots.
35
Hardware features
VCE supports the following hardware features:
• Dual 10 GE Optical/Active Twinax IP IO/SLIC for X-Blades
• 2 ½ inch vault drives
• 2 ½ inch DAEs and drive form factors
• 3 ½ inch DAEs and drive form factors
File deduplication
File deduplication is supported, but is not enabled by default. Enabling this feature requires knowledge of
capacity and storage requirements.
Block compression
Block compression is supported but is not enabled by default. Enabling this feature requires knowledge of
capacity and storage requirements.
External NFS and CIFS access
VCE Systems can present CIFS and NFS shares to external clients provided that these provisions are
followed:
• VCE Systems shares cannot be mounted internally by VCE Systems hosts and external to the
VCE Systems at the same time.
• In a configuration with two X-Blades, mixed internal and external access is supported.
• In a configuration with more than two X-Blades, external NFS and CIFS access can run on one or
more X-Blades that are physically separate from the X-Blades serving VMFS data stores to the
VCE Systems compute layer.
Snapshots
EMC VNX snapshots are only for storage pools; not for RAID groups. Storage pools can use EMC
SnapView snapshots and EMC VNX snapshots at the same time.
Note: EMC VNX snapshot is an optional feature. Both types of snapshots have a seamless support
perspective. VCE relies on guidance from EMC best practices for different use cases of EMC
SnapView snapshots versus EMC VNX snapshots.
Replicas
For VCE Systems NAS configurations, EMC VNX Replicator is supported. This software can create local
clones (full copies) and replicate file systems asynchronously across IP networks. EMC VNX Replicator is
included in the EMX VNX Remote Protection Suite.
36
Network layer VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
Network layer
Network overview
This topic provides an overview of the network components for the VCE System.
The Cisco Nexus Series Switches in the network layer provide 10 or 40 GbE IP connectivity between the
VCE System and the external network. In unified storage architecture, the switches also connect the
fabric interconnects in the compute layer to the X-Blades in the storage layer.
In the segregated architecture, the Cisco MDS 9000 series switches in the network layer provide Fibre
Channel (FC) links between the Cisco fabric interconnects and the EMC VNX array. These FC
connections provide block level devices to blades in the compute layer. In unified network architecture,
there are no Cisco MDS series storage switches. FC connectivity is provided by the Cisco Nexus 5548UP
Switches or Cisco Nexus 5596UP Switches.
Ports are reserved or identified for special services such as backup, replication, or aggregation uplink
connectivity.
The VCE System contains two Cisco Nexus 3048 switches to provide management network connectivity
to the different components of the VCE System. These connections include the EMC VNX service
processors, Cisco UCS fabric interconnects, Cisco Nexus 5500UP switches or Cisco Nexus 9396PX
switches, and power output unit (POU) management interfaces.
IP network components
This topic describes the IP network components used by VCE Systems.
VCE Systems use Cisco UCS 6200 series fabric interconnects. VCE Systems with EMC VNX5400 use
the Cisco UCS 6248UP fabric switches. All other VCE Systems use the Cisco UCS 6248UP Fabric
Interconnects or the Cisco UCS 6296UP Fabric Interconnects.
VCE Systems include two Cisco Nexus 5548UP switches, Cisco Nexus 5596UP switches, or Cisco
Nexus 9396PX switches to provide 10 or 40 GbE connectivity:
• Between the VCE Systems internal components
• To the site network
• To the second generation Advanced Platform (AMP-2) through redundant connections between
AMP-2 and the Cisco Nexus 5548UP switches, Cisco Nexus 5596UP switches, or Cisco Nexus
9396PX switches
37
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Network layer
To support the Ethernet and SAN requirements in the traditional, segregated network architecture, two
Cisco Nexus 5548UP switches or Cisco Nexus 9396PX switches provide Ethernet connectivity, and a pair
of Cisco MDS switches provide Fibre Channel (FC) connectivity.
The Cisco Nexus 5548UP Switch is available as an option for all segregated network VCE Systems. It is
also an option for unified network VCE Systems with EMC VNX5400 and EMC VNX5600.
Cisco Nexus 5500 series switches
The two Cisco Nexus 5500 series switches support low latency, line-rate, 10 Gb Ethernet and FC over
Ethernet (FCoE) connectivity for up to 96 ports. Unified port expansion modules are available and provide
an extra 16 ports of 10 GbE or FC connectivity. The FC ports are licensed in packs of eight in an on-
demand basis.
The Cisco Nexus 5548UP switches have 32 integrated, low-latency, unified ports. Each port provides
line-rate, 10 Gb Ethernet or eight Gb/s FC connectivity. The Cisco Nexus 5548UP switches have one
expansion slot that can be populated with a 16 port unified port expansion module. The Cisco Nexus
5548UP Switch is the only network switch supported for VCE Systems data connectivity in VCE Systems
(5400).
The Cisco Nexus 5596UP switches have 48 integrated, low-latency, unified ports. Each port provides
line-rate 10 GB Ethernet or eight Gbs FC connectivity. The Cisco Nexus 5596UP switches have three
expansion slots that can be populated with 16 port, unified, port expansion modules. The Cisco Nexus
5596UP Switch is available as an option for both network topologies for all VCE Systems except VCE
Systems (5400).
Cisco Nexus 9396PX Switch
The Cisco Nexus 9396PX Switch supports both 10 Gbps SFP+ ports and 40 Gbps QSFP+ ports. The
Cisco Nexus 9396PX Switch is a two rack unit (2RU) appliance with all ports licensed and available for
use. There are no expansion modules available for the Cisco Nexus 9396PX Switch.
The Cisco Nexus 9396PX Switch provide 48 integrated, low-latency SFP+ ports. Each port provides line-
rate 1/10 Gbps Ethernet. There are also 12 QSFP+ ports that provide line-rate 40Gbps Ethernet.
Related information
Management hardware components (see page 50)
Management software components (see page 51)
Port utilization
This section describes the switch port utilization for Cisco Nexus 5548UP Switch and Cisco Nexus
5596UP Switch in segregated networking and unified networking configurations, as well as the Cisco
Nexus switches in a segregated networking configuration.
38
Cisco Nexus 5548UP Switch - segregated networking

This section describes port utilization for a Cisco Nexus 5548UP Switch segregated networking
configuration.
The base Cisco Nexus 5548UP Switch provides 32 SFP+ ports used for 1G or 10G connectivity for LAN
traffic.
The following table shows the core connectivity for the Cisco Nexus 5548UP Switch (no module) with
segregated networking:
Feature Used ports Port speeds Media
Uplinks from fabric interconnect (FI) 8* 10G Twinax
Uplinks to customer core 8** Up to 10G SFP+
Uplinks to other Cisco Nexus 5000 Series Switches 2 10G Twinax
AMP-2 ESX management 3 10G SFP+
*VCE Systems with VNX5400 only support four links between the Cisco UCS FIs and Cisco Nexus
5548UP switches.
**VCE Systems with VNX5400 only support four links between the Cisco Nexus 5548UP Switch and
customer core network.
The remaining ports in the base Cisco Nexus 5548UP Switch (no module) provide support for the
following additional connectivity option:
Feature Available ports Port speeds Media
Customer IP backup 3 1G or 10G SFP+
If an optional 16 unified port module is added to the Cisco Nexus 5548UP Switch, there are 28 additional
ports (beyond the core connectivity requirements) available to provide additional feature connectivity.
Actual feature availability and port requirements are driven by the model that is selected.
The following table shows the additional connectivity for Cisco Nexus 5548UP Switch with a 16UP
module:
Uplinks from Cisco UCS FI for Ethernet bandwidth (BW) enhancement 8 10G Twinax
39
Cisco Nexus 5596UP Switch - segregated networking

This section describes port utilization for a Cisco Nexus 5596UP Switch segregated networking
configuration.
traffic.
The following table shows core connectivity for the Cisco Nexus 5596UP Switch (no module) with
segregated networking:
Uplinks from Cisco UCS FI 8 10G Twinax
Uplinks to customer core 8 Up to 10G SFP+
Uplinks to other Cisco Nexus 5000 Series Switches 2 10G Twinax
following additional connectivity option:
If an optional 16 unified port module is added to the Cisco Nexus 5596UP Switch, additional ports
(beyond the core connectivity requirements) are available to provide additional feature connectivity.
Actual feature availability and port requirements are driven by the model that is selected.
The following table shows the additional connectivity for the Cisco Nexus 5596UP Switch with one 16UP
module:
Note: Cisco Nexus 5596UP Switch with two or three 16UP modules is not supported with segregated
networking.
Uplinks from Cisco UCS FIs for Ethernet BW enhancement 8 10G Twinax
Cisco Nexus 5548UP Switch – unified networking

This section describes port utilization for a Cisco Nexus 5548UP Switch unified networking configuration.
traffic or 2/4/8 Gbps FC traffic.
40
The following table shows the core connectivity for the Cisco Nexus 5548UP Switch (no module) with
unified networking for VCE Systems with EMC VNX5400 only.
Uplinks to other Cisco Nexus 5K 2 10G Twinax
FC uplinks from Cisco UCS FI 4 8G SFP+
FC links to EMC VNX array 6 8G SFP+
The following table shows the core connectivity for the Cisco Nexus 5548UP Switch with unified
networking for VCE Systems with EMC VNX5600:
FC uplinks from UCS FI 4 8G SFP+
following additional connectivity options for VCE Systems with EMC VNX5400 only.
X-Blade connectivity 2 10G EMC Active Twinax
X-Blade NDMP connectivity 2 8G SFP+
The remaining ports in the base Cisco Nexus 5548UP Switch provide support for the following additional
connectivity options for the other VCE Systems:
EMC RecoverPoint WAN links (one per EMC RecoverPoint 2 1G GE_T SFP+
Appliance pair)
41
If an optional 16 unified port module is added to the Cisco Nexus 5548UP Switch, additional ports
(beyond the core connectivity requirements) available to provide additional feature connectivity. Actual
feature availability and port requirements are driven by the model that is selected.
The following table shows the additional connectivity for the Cisco Nexus 5548UP Switch with one 16UP
module:
EMC RecoverPoint WAN links (one per EMC RecoverPoint 4 1G GE_T SFP+
Appliance pair)
Uplinks from Cisco UCS FIs for Ethernet BW Enhancement 8 10G Twinax
Cisco Nexus 5596UP Switch - unified networking

This section describes port utilization for a Cisco Nexus 5596UP Switch unified networking configuration.
The base Cisco Nexus 5596UP Switch provides 48 SFP+ ports used for 1/10G connectivity for LAN traffic
or 2/4/8 Gbps Fibre Channel (FC) traffic.
The following table shows the core connectivity for the Cisco Nexus 5596UP Switch (no module):
FC links to EMC VNX Array 6 8G SFP+
following additional connectivity options:
Feature Minimum ports Port speeds Media

required for feature
X-Blade NDMP connectivity 2 8G SFP+
IP backup solutions 4 1 or 10G SFP+
42
Feature Minimum ports Port speeds Media

required for feature
EMC RecoverPoint WAN links (one per EMC 2 1G GE_T SFP+

RecoverPoint Appliance pair)
EMC RecoverPoint SAN links (two per EMC 4 8G SFP+

RecoverPoint Appliance)
Up to three additional 16 unified port modules can be added to the Cisco Nexus 5596UP Switch
(depending on the selected VCE System). Each module has 16 ports to enable additional feature
connectivity. Actual feature availability and port requirements are driven by the model that is selected.
The following table shows the connectivity options for Cisco Nexus 5596UP Switch for slots 2-4:
Feature Ports available Port speeds Media Default

for feature module
Uplinks from Cisco UCS FI for Ethernet BW 8 10G Twinax 1

enhancement
EMC VPLEX SAN connections (4 per engine) 8 8G SFP+ 1
X-Blade connectivity 12 10G EMC Active 3

Twinax
X-Blade NDMP connectivity 6 8G SFP+ 3,4
EMC RecoverPoint WAN links (1 per EMC 2 1G GE_T SFP+ 4

RecoverPoint Appliance pair)
EMC RecoverPoint SAN links 4 8G SFP+ 4

(2 per EMC RecoverPoint Appliance)
FC links from Cisco UCS fabric interconnect 4 8G SFP+ 4

for FC BW enhancement
FC links from EMC VNX array for FC BW 4 8G SFP+ 4

enhancement
Cisco Nexus 9396PX Switch - segregated networking

This section describes port utilization for a Cisco Nexus 9396PX Switch segregated networking
configuration.
The base Cisco Nexus 9396PX Switch provides 48 SFP+ ports used for 1G or 10G connectivity and 12
40G QSFP+ ports for LAN traffic.
43
The following table shows core connectivity for the Cisco Nexus 9396PX Switch with segregated
networking:
Uplinks from fabric 8* 10G Twinax

interconnect (FI)
Uplinks to customer 8(10G)**/2(40G) Up to 40G SFP+/QSFP+

core***
VPC peer links 2 40G Twinax
AMP-2 ESX 3 10G SFP+

management
*VCE Systems with EMC VNX5400 only support four links between the Cisco UCS FIs and Cisco Nexus
9396PX switches.
** VCE Systems with EMC VNX5400 only support four links between the Cisco Nexus 9396PX Switch
and customer core network.
*** VCE Systems and Cisco Nexus 9396PX support 40G or 10G SFP+ uplinks to customer core.
The remaining ports in the Cisco Nexus 9396PX Switch provide support for a combination of the following
additional connectivity options:
EMC RecoverPoint 4 1G GE T SFP+

WAN links (one per
EMC RecoverPoint
Appliance pair)
Uplinks from Cisco 8 10G Twinax

UCS FIs for Ethernet
BW enhancement*
*Not supported on VCE Systems with EMC VNX5400
Storage switching components

This section describes how VCE Systems include redundant Cisco SAN fabric switches.
In a segregated networking model, there are two Cisco MDS 9148 multilayer fabric switches. In a unified
networking model, Fibre Channel (FC) based features are provided by the two Cisco Nexus 5548UP
switches or Cisco Nexus 5596UP switches that are also used for LAN traffic.
44
In the VCE Systems, these switches provide:
• FC connectivity between the compute layer components and the storage layer components
• Connectivity for backup, business continuity (EMC RecoverPoint Appliance), and storage
federation requirements when configured.
Note: Inter-Switch Links (ISLs) to the existing SAN are not permitted.
The Cisco MDS 9148 Multilayer Fabric Switch provides from 16 to 48 line-rate ports for non-blocking
8 Gbps throughput. The port groups are enabled on an as needed basis.
The Cisco Nexus 5548UP switches or Cisco Nexus 5596UP switches provide a number of line-rate ports
for non-blocking 8 Gbps throughput. Expansion modules can be added to the Cisco Nexus 5596UP
Switch that provide 16 additional ports operating at line-rate.
The following tables define the port utilization for the SAN components when using a Cisco MDS 9148:
Feature Available ports
Backup 2
FC links from Cisco UCS fabric interconnect (FI) for FC Bandwidth (BW) enhancement 4
FC links from EMC VNX array for FC BW enhancement 4
FC links to EMC VNX array dedicated for replication 2
EMC RecoverPoint SAN links (two per EMC RecoverPoint Appliance) 8
SAN aggregation 2
EMC VPLEX SAN connections (four per engine) 8
X-Blade network data management protocol (NDMP) connectivity 2
45
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Virtualization layer
Virtualization layer
Virtualization components
VMware vSphere is the virtualization platform that provides the foundation for the private cloud. The core
VMware vSphere components are the VMware vSphere ESXi and VMware vCenter Server for
management. VMware vSphere 5.x includes a Single Sign-on (SSO) component as a standalone
Windows server and an embedded service on the vCenter server. VMware vSphere 6.0 includes a pair of
Platform Service Controller Linux appliances to provide the SSO service.
The hypervisors are deployed in a cluster configuration. The cluster allows dynamic allocation of
resources, such as CPU, memory, and storage. The cluster also provides workload mobility and flexibility
with the use of VMware vMotion and Storage vMotion technology.
VMware vSphere Hypervisor ESXi

This topic describes the VMware vSphere Hypervisor ESXi that runs on the second generation of the
Advanced Management Platform (AMP-2) and in a VCE System utilizing VMware vSphere Server
Enterprise Plus.
This lightweight hypervisor requires very little space to run (less than six GB of storage required to install)
and has minimal management overhead.
VMware vSphere ESXi does not contain a console operating system. The VMware vSphere Hypervisor
ESXi boots from Cisco FlexFlash (SD card) on AMP-2. For the compute blades, ESXi boots from the SAN
through an independent Fibre Channel (FC) LUN presented from the EMC VNX storage array. The FC
LUN also contains the hypervisor's locker for persistent storage of logs and other diagnostic files to
provide stateless computing within VCE Systems. The stateless hypervisor is not supported.
Cluster configuration
VMware vSphere ESXi hosts and their resources are pooled together into clusters. These clusters
contain the CPU, memory, network, and storage resources available for allocation to virtual machines
(VMs). Clusters can scale up to a maximum of 32 hosts for VMware vSphere 5.1/5.5 and 64 hosts for
VMware vSphere 6.0. Clusters can support thousands of VMs.
The clusters can also support a variety of Cisco UCS blades running inside the same cluster.
Note: Some advanced CPU functionality might be unavailable if more than one blade model is running in
a given cluster.
Data stores
VCE Systems support a mixture of data store types: block level storage using VMFS or file level storage
using NFS.
46
Virtualization layer VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
The maximum size per VMFS5 volume is 64 TB (50 TB VMFS3 @ 1 MB). Beginning with VMware
vSphere 5.5, the maximum VMDK file size is 62 TB. Each host/cluster can support a maximum of 255
volumes.
VCE optimizes the advanced settings for VMware vSphere ESXi hosts that are deployed in VCE Systems
to maximize the throughput and scalability of NFS data stores. VCE Systems support a maximum of 256
NFS data stores per host.
Virtual networks
Virtual networking in the Advanced Management Platform (AMP-2) uses standard virtual switches. Virtual
networking in VCE Systems is managed by the Cisco Nexus 1000V Series Switch. The Cisco Nexus
1000V Series Switch ensures consistent, policy-based network capabilities to all servers in the data
center by allowing policies to move with a VM during live migration. This provides persistent network,
security, and storage compliance.
Alternatively, virtual networking in VCE Systems is managed by a VMware vCenter Virtual Distributed
Switch (version 5.5 or higher) with comparable features to the Cisco Nexus 1000V where applicable. The
VMware VDS option consists of both a VMware Standard Switch (VSS) and a VMware vSphere
Distributed Switch (VDS) and will use a minimum of four uplinks presented to the hypervisor.
The implementation of Cisco Nexus 1000V Series Switch for VMware vSphere 5.1/5.5 and VMware VDS
for VMware vSphere 5.5 use intelligent network Class of Service (CoS) marking and Quality of Service
(QoS) policies to appropriately shape network traffic according to workload type and priority. With
VMware vSphere 6.0, QoS is set to Default (Trust Host). The vNICs are equally distributed across all
available physical adapter ports to ensure redundancy and maximum bandwidth where appropriate. This
provides general consistency and balance across all Cisco UCS blade models, regardless of the Cisco
UCS Virtual Interface Card (VIC) hardware. Thus, VMware vSphere ESXi has a predictable uplink
interface count. All applicable VLANs, native VLANs, MTU settings, and QoS policies are assigned to the
virtual network interface cards (vNIC) to ensure consistency in case the uplinks need to be migrated to
the VMware vSphere Distributed Switch (VDS) after manufacturing.
Related information
Management hardware components (see page 50)
Management software components (see page 51)
VMware vCenter Server

This topic describes the VMware vCenter Server which is a central management point for the hypervisors
and VMs.
VMware vCenter Server is a central management point for the hypervisors and virtual machines. VMware
vCenter Server is installed on a 64-bit Windows Server. VMware Update Manager is installed on a 64-bit
Windows Server and runs as a service to assist with host patch management.
47
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Virtualization layer
Second generation of the Advanced Management Platform with redundant physical servers (AMP-2RP)
and the VCE System each have a unified VMware vCenter Server Appliance instance. Each of these
systems resides in the AMP-2RP.
VMware vCenter Server provides the following functionality:
• Cloning of VMs
• Creating templates
• VMware vMotion and VMware Storage vMotion
• Initial configuration of VMware Distributed Resource Scheduler (DRS) and VMware vSphere
high-availability clusters
VMware vCenter Server provides monitoring and alerting capabilities for hosts and VMs. VCE System
administrators can create and apply the following alarms to all managed objects in VMware vCenter
Server:
• Data center, cluster, and host health, inventory, and performance
• Data store health and capacity
• VM usage, performance, and health
• Virtual network usage and health
Databases
The backend database that supports VMware vCenter Server and VMware Update Manager (VUM) is
remote Microsoft SQL Server 2008 (vSphere 5.1) and Microsoft SQL 2012 (vSphere 5.5/6.0). The SQL
Server service requires a dedicated service account.
Authentication
VCE Systems support the VMware Single Sign-On (SSO) Service capable of the integration of multiple
identity sources including Active Directory, Open LDAP, and local accounts for authentication. VMware
SSO is available in VMware vSphere 5.1 and higher. VMware vCenter Server, Inventory, Web Client,
SSO, Core Dump Collector, and Update Manager run as separate Windows services, which can be
configured to use a dedicated service account depending on the security and directory services
requirements.
VCE supported features
VCE supports the following VMware vCenter Server features:
• VMware Single Sign-On (SSO) Service (version 5.1 and higher)
• VMware vSphere Web Client (used with VCE Vision™ Intelligent Operations)
48
Virtualization layer VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
• VMware vSphere Distributed Switch (VDS)
• VMware vSphere High Availability
• VMware DRS
• VMware Fault Tolerance
• VMware vMotion
• VMware Storage vMotion
— Layer 3 capability available for compute resources (version 6.0 and higher)
• Raw Device Mappings
• Resource Pools
• Storage DRS (capacity only)
• Storage driven profiles (user-defined only)
• Distributed power management (up to 50 percent of VMware vSphere ESXi hosts/blades)
• VMware Syslog Service
• VMware Core Dump Collector
• VMware vCenter Web Client
49
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Management
Management
Management components overview

This topic describes the second generation of the Advanced Management Platform (AMP-2) components.
AMP-2 provides a single management point for VCE Systems and provides the ability to:
• Run the core and VCE Optional Management Workloads
• Monitor and manage VCE System health, performance, and capacity
• Provide network and fault isolation for management
• Eliminate resource overhead on VCE Systems
The Core Management Workload is the minimum required set of management software to install, operate,
and support a VCE System. This includes all hypervisor management, element managers, virtual
networking components (Cisco Nexus 1000v or VMware vSphere Distributed Switch (VDS)), and VCE
Vision™ Intelligent Operations software.
The VCE Optional Management Workload is non-Core Management Workloads that are directly
supported and installed by VCE whose primary purpose is to manage components within a VCE System.
The list would be inclusive of, but not limited to, Data Protection, Security or Storage management tools
such as, EMC Unisphere for EMC RecoverPoint or EMC VPLEX, Avamar Administrator, EMC InsightIQ
for Isilon, or VMware vCNS appliances (vShield Edge/Manager).
Related information
Management hardware components

This topic describes the second generation of the Advanced Management Platform (AMP-2) hardware.
AMP-2 is available with one to three physical servers. All options use their own resources to run
workloads without consuming resources:
AMP-2 option Physical server Description
AMP-2P One Cisco UCS C220 Default configuration for VCE Systems that use a dedicated
server Cisco UCS C220 Server to run management workload
applications.
50
Management VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
AMP-2 option Physical server Description
AMP-2RP Two Cisco UCS C220 Adds a second Cisco UCS C220 Server to support application
servers and hardware redundancy.
AMP-2HA Baseline Two Cisco UCS C220 Implements VMware vSphere HA/DRS with shared storage
servers provided by EMC VNXe3200 storage.
AMP-2HA Three Cisco UCS C220 Adds a third Cisco UCS C220 Server and additional storage
Performance servers for EMC FAST VP.
Management software components

This topic describes the software that is delivered pre-configured with the second generation of the
Advanced Management Platform (AMP-2).
AMP-2 is delivered pre-configured with the following software components which are dependent on the
selected VCE Release Certification Matrix:
• Microsoft Windows Server 2008 R2 SP1 Standard x64
• Microsoft Windows Server 2012 R2 Standard x64
• VMware vSphere Enterprise Plus
• VMware vSphere Hypervisor ESXi
• VMware Single Sign-On (SSO) Service
• VMware vSphere Web Client Service
• VMware vSphere Inventory Service
• VMware vCenter Server
• VMware vCenter Database using Microsoft SQL Server Standard Edition
• VMware vCenter Update Manager
• VMware vSphere client
• VMware vSphere Syslog Service (optional)
• VMware vSphere Core Dump Service (optional)
• VMware vCenter Server Appliance (AMP-2RP) - a second instance of VMware vCenter Server is
required to manage the replication instance separate from the production VMware vCenter Server
• VMware vSphere Replication Appliance (AMP-2RP)
51
• VMware vSphere Distributed Switch (VDS) or Cisco Nexus 1000V virtual switch (VSM)
• EMC PowerPath/VE Electronic License Management Server (ELMS)
• EMC Secure Remote Support (ESRS)
• Array management modules, including but not limited to, EMC Unisphere Client, EMC Unisphere
Service Manager, EMC VNX Initialization Utility, EMC VNX Startup Tool, EMC SMI-S Provider,
EMC PowerPath Viewer
• Cisco Prime Data Center Network Manager and Device Manager
• (Optional) EMC RecoverPoint management software that includes EMC RecoverPoint

Management Application and EMC RecoverPoint Deployment Manager
Management network connectivity

This topic provides the second generation of the Advanced Management Platform network connectivity
and server assignment illustrations.
52
AMP-2HA network connectivity
The following illustration provides an overview of the network connectivity for the AMP-2HA:
53
AMP-2HA server assignments
The following illustration provides an overview of the VM server assignment for AMP-2HA:
VCE Systems that use VMware vSphere Distributed Switch (VDS) do not include Cisco Nexus1000V
VSM VMs.
The Performance option of AMP-2HA leverages the DRS functionality of VMware vCenter to optimize
resource usage (CPU/memory) so that VM assignment to a VMware vSphere ESXi host will be managed
automatically
54
AMP-2P server assignments
The following illustration provides an overview of the VM server assignment for AMP-2P:
55
AMP-2RP server assignments
The following illustration provides an overview of the VM server assignment for AMP-2RP:
VCE Systems that use VMware VDS do not include Cisco Nexus1000V VSM VMs.
56
System infrastructure VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
System infrastructure
VCE Systems descriptions

This topic provides a comparison of the compute, network, and storage architecture for the VCE Systems.
The following table shows a comparison of the compute architecture:
VCE Systems with: EMC VNX8000 EMC VNX7600 EMC VNX5800 EMC VNX5600 EMC VNX5400
Cisco B-series blade 16 maximum 8 maximum 2 maximum

chassis
B-series blades Half-width = 128, Full-width = 64 Half-width = 64 Half-width = 16

(maximum) Full-width = 32 Full-width = 8
Fabric interconnects Cisco Nexus 6248UP or Cisco Nexus 6296UP Cisco Nexus
6248UP
The following table shows a comparison of the network architecture:
Network Cisco Nexus 5548UP or Cisco Nexus 5596UP Cisco Nexus 5548UP
SAN Cisco MDS 9148 (segregated)
The following table shows a comparison of the storage architecture:
Storage access Block or unified
Back-end SAS buses 8 or 16 6 6 2 or 6 2
Storage protocol (block) FC
Storage protocol (file) NFS and CIFS
Data store type (block) VMFS
Data store type (file) NFS
Boot path SAN
Maximum drives 1500 1000 750 500 250
X-Blades (min/max) 2/8 2/4 2/3 2/2 2/2
57
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview System infrastructure
Cabinets overview
This topic describes the VCE cabinets.
In each VCE System, the compute, storage, and network layer components are distributed within two or
more 42U cabinets. Distributing the components this way balances out the power draw and reduces the
size of the power outlet units (POUs) that are required.
Each cabinet conforms to a standard predefined layout. Space can be reserved for specific components
even if they are not present or required for the external configuration. This design makes it easier to
upgrade or expand each VCE System as capacity needs increase.
VCE System cabinets are designed to be installed next to one another within the data center (that is,
contiguously). If a customer requires the base and expansion cabinets to be physically separated,
customized cabling is needed, which incurs additional cost and can increase delivery time.
Note: The cable length is NOT the same as distance between cabinets. The cable must route through
the cabinets and through the cable channels overhead or in the floor.
Intelligent Physical Infrastructure appliance

The Intelligent Physical Infrastructure (IPI) appliance provides an intelligent gateway to gather information
about power, thermals, security, alerts, and all components in the physical infrastructure for each cabinet.
For more information about the IPI appliance, refer to the administration guide for your VCE System.
Power options
This topic describes the power outlet unit (POU) options inside and outside of North America.
VCE Systems support several POU options inside and outside of North America.
North America power options
The NEMA POU is standard; other POUs add time to assembly and delivery. The following table lists the
POUs available for VCE Systems in North America:
POU Power specifications
NEMA L15-30P 3-phase Delta / 30A / 208V
IEC 60309 3-phase Delta / 60A / 208V

EC 309 3P4W SPLASH PROOF 460P9S
IEC 60309 Single phase / 60A / 208V

IEC309 2P3W SPLASH PROOF 360P6S
NEMA L6-30P Single phase / 30A / 208V (half-height)
58
System infrastructure VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
Europe power options
The IEC 309 POU is standard; other POUs add time to assembly and delivery. The following table lists
the POUs available for VCE Systems in Europe:
IEC 60309, SPLASH PROOF 3-phase WYE/ 32A / 230 / 400V
IEC 60309, SPLASH PROOF 3-phase WYE / 16A / 230 / 400V
IEC 60309, SPLASH PROOF Single phase / 32A / 230V (half height)
Japan power options
The following table lists the POUs available for VCE Systems in Japan:
JIS C8303 L15-30P 3-phase Delta / 30A / 208V
IEC 60309 SPLASH PROOF 3-phase Delta / 60A / 208V
IEC 60309 SPLASH PROOF Single phase / 60A / 208V
JIS C8303 L15-30P Single phase / 30A / 208V (half-height)
The VCE Vblock® and VxBlock™ Systems 340 Physical Planning Guide provides more information about
power requirements.
Related information
59
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Configuration descriptions
Configuration descriptions
VCE Systems with EMC VNX8000

This topic provides an overview of the VCE Systems with EMC VNX8000 components.
Array options
VCE Systems (8000) are available as block only or unified storage. Unified storage VCE Systems (8000)
support up to eight X-Blades and ships with two X-Blades and two control stations. Each X-Blade
provides four 10G front-end network connections. An additional data mover enclosure (DME) supports
the connection of two additional X-Blades with the same configuration as the base data movers.
The following table shows the available array options:
Array Bus Supported X-Blades
Block 8/16 N/A
Unified 8/16 2
Unified 8/16 3
Unified 8/16 4
Unified 8/16 5
Unified 8/16 6
Unified 8/16 7
Unified 8/16 8
Each X-Blade contains:
• One 6 core 2.8 GHz Xeon processor
• 24 GB RAM
• One Fibre Channel (FC) storage line card (SLIC) for connectivity to array
• Two 2-port 10 GB SFP+ compatible SLICs
Feature options
VCE Systems (8000) support both Ethernet and FC bandwidth (BW) enhancement. Ethernet BW
enhancement is available with Cisco Nexus 5596UP switches only. FC BW enhancement requires that
SAN connectivity is provided by Cisco MDS 9148 multilayer fabric switches or Cisco Nexus 5596UP
switches, depending on topology.
60
Configuration descriptions VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
The following table shows the feature options:
Array Topology FC BW enhancement Ethernet BW enhancement
Block Segregated Y Y
Unified Segregated Y Y
Block Unified network Y Y
Unified Unified network Y Y
Unified networking is supported only on the VCE Systems (8000) with Cisco Nexus 5596UP switches.
Ethernet BW enhancement is supported only on the VCE Systems (8000) with Cisco Nexus 5596UP
switches.
Disk array enclosure configuration
VCE Systems (8000) include two 25 slot 2.5" disk array enclosures (DAEs). An additional six DAEs are
required beyond the two base DAEs. Additional DAEs can be added in either 15 slot 3.5" DAEs or 25 slot
2.5" DAEs. Additional DAEs (after initial eight) are added in multiples of eight. If there are 16 buses, then
DAEs must be added in multiples of 16. DAEs are interlaced when racked, and all 2.5" DAEs are first
racked on the buses, then 3.5" DAEs.
SLIC configuration
The EMC VNX8000 provides slots for 11 SLICs in each service processor (SP).
• Two slots in each SP are populated with back-end SAS bus modules by default.
• Two additional back-end SAS bus modules support up to 16 buses. If this option is chosen, all
DAEs are purchased in groups of 16.
• VCE Systems (8000) support two FC SLICs per SP for host connectivity. Additional FC SLICs are
included to support unified storage.
• If FC BW enhancement is configured, an additional FC SLIC is added to the array.
• The remaining SLIC slots are reserved for future VCE configuration options.
• VCE only supports the four port FC SLIC for host connectivity.
• By default, six FC ports per SP are connected to the SAN switches for VCE Systems host
connectivity. The addition of FC BW Enhancement provides four additional FC ports per SP.
As the EMC VNX8000 has multiple CPUs, SLIC arrangements should be balanced across CPUs.
61
The following table shows the SLIC configurations per SP (eight bus):
Array FC BW enhancement SL 0 SL 1 SL 2 SL 3 SL 4 SL 5 SL 6 SL 7 SL 8 SL 9 SL 10
Block Y FC Res Res FC Res Bus Res Res Res FC Bus
Unified Y FC Res Res FC Res Bus Res Res FC/U FC Bus
Block N FC Res Res Res Res Bus Res Res Res FC Bus
Unified N FC Res Res Res Res Bus Res Res FC/U FC Bus
Unified -> 4 DM N FC Res FC/U Res Res Bus Res Res FC/U FC Bus
Unified -> 4 DM Y FC Res FC/U FC Res Bus Res Res FC/U FC Bus
Res: slot reserved for future VCE configuration options.
FC: 4xFC port input/output module (IOM): Provides four 8G FC connections.
FC/U: 4xFC port IOM dedicated to unified X-Blade connectivity: provides four 8G FC connections.
Bus: Four port - 4x lane/port 6 Gb/s SAS: provides additional back-end bus connections.
The following table shows the SLIC configurations per SP (16 bus):
Array FC BW SL 0 SL 1 SL 2 SL 3 SL 4 SL 5 SL 6 SL 7 SL 8 SL 9 SL 10
Block Y FC Res Res FC Bus Bus Bus Res Res FC Bus
Unified Y FC Res Res FC Bus Bus Bus Res FC/U FC Bus
Block N FC Res Res Res Bus Bus Bus Res Res FC Bus
Unified N FC Res Res Res Bus Bus Bus Res FC/U FC Bus
Unified -> 4 DM N FC Res FC/U Res Bus Bus Bus Res FC/U FC Bus
Unified -> 4 DM Y FC Res FC/U FC Bus Bus Bus Res FC/U FC Bus
N/A: not available for this configuration.

Res: slot reserved for future VCE configuration options.
FC: 4xFC port IOM: provides four 8G FC connections.
FC/U: 4xFC port IOM dedicated to unified X-Blade connectivity: provides four 8G FC connections.
Two additional back-end SAS bus modules are available to support up to 16 buses. If this option is
chosen, all DAEs are purchased in groups of 16.
Compute
VCE Systems (8000) support between two to 16 chassis, and up to 128 half-width blades. Each chassis
can be connected with two links (Cisco UCS 2204XP fabric extenders IOM only), four links (Cisco UCS
2204XP fabric extenders IOM only), or eight links (Cisco UCS 2208XP fabric extenders IOM only) per
IOM.
62
The following table shows the compute options that are available for the fabric interconnects:
Fabric interconnect Min chassis 2-link max chassis 4-link max chassis 8-link max
(blades) (blades) (blades) chassis (blades)
Cisco UCS 6248UP 2 (2) 16 (128) 8 (64) 4 (32)
Cisco UCS 6296UP 2 (2) N/A 16 (128) 8 (64)
Connectivity
VCE Systems (8000) support the Cisco UCS 6248UP fabric interconnects and Cisco UCS 6296UP fabric
interconnects. These uplink to the Cisco Nexus 5548UP switches or Cisco Nexus 5596UP switches for
Ethernet connectivity. SAN connectivity is provided by the Cisco Nexus 5500 Series Switches or Cisco
MDS 9148 multilayer fabric switches, based on topology.
The following table shows the available switch combinations that are available for the fabric
interconnects:
Fabric interconnect Topology Ethernet SAN
Cisco UCS 6248UP Segregated Cisco Nexus 5548UP Cisco MDS 9148 Multilayer
switches Fabric Switch
Segregated Cisco Nexus 5596UP Cisco MDS 9148 Multilayer

Unified Cisco Nexus 5596UP switches

Note: The default is unified network with Cisco Nexus 5596UP switches.

This topic provides an overview of VCE Systems with EMC VNX7600 components.
Array options
support up to eight X-Blades and ships with two X-Blades and two control stations. Each X-Blade
provides four 10G front-end connections to the network. An additional data mover enclosure (DME)
supports the connection of two additional X-Blades with the same configuration as the base X-Blades.
63
Block 6 N/A
Unified 6 2*
Unified 6 3*
Unified 6 4*
Unified 6 5*
Unified 6 6*
Unified 6 7*
Unified 6 8*
*VCE supports two to eight X-Blades in VCE Systems (7600).
• 12 GB RAM
Feature options
VCE Systems (7600) support both Ethernet and FC bandwidth (BW) enhancement. Ethernet BW
SAN connectivity is provided by Cisco MDS 9148 multilayer fabric switches or the Cisco Nexus 5596UP
switches, depending on topology. Both block and unified arrays use FC BW enhancement.
Unified networking is only supported on VCE Systems (7600) with Cisco Nexus 5596UP switches.
64
VCE Systems (7600) have two 25 slot 2.5" disk array enclosures (DAEs). The EMC VNX7600 data
processor enclosure (DPE) provides the DAE for bus 0, and provides the first DAE on bus 1. An
additional four DAEs are required beyond the two base DAEs. Additional DAEs can be added in either 15
slot 3.5" DAEs or 25 slot 2.5" DAEs. Additional DAEs (after initial six) are added in multiples of six. DAEs
are interlaced when racked, and all 2.5" DAEs are racked first on the buses, then 3.5" DAEs.
SLIC configuration
The EMC VNX7600 provides slots for five SLICs in each service processor (SP). Slot 0 in each SP is
populated with a back-end SAS bus module. VCE Systems (7600) support two FC SLICs per SP for host
connectivity. A third is reserved to support unified storage. If FC BW enhancement is configured, an
additional FC SLIC is added to the array. VCE only supports the four port FC SLIC for host connectivity.
By default, six FC ports per SP are connected to the SAN switches for VCE Systems host connectivity.
The addition of FC BW enhancement provides four additional FC ports per SP.
The following table shows the SLIC configurations per SP:
Array FC BW enhancement SLIC 0 SLIC 1 SLIC 2 SLIC 3 SLIC 4
Block Y Bus FC FC FC N/A
Unified (<5DM)* Y Bus FC FC FC FC/U
Block N Bus FC FC N/A N/A
Unified N Bus FC FC FC/U FC/U
Greater than four X-Blades prohibits FC BW enhancement feature
FC 4xFC port IO module: provides four 8G FC connections.
FC/U 4xFC port IO module dedicated to unified X-Blade connectivity: provides four 8G FC connections.
Bus four port - 4x lane/port six GB SAS: provides additional back-end bus connections.
Compute
VCE Systems (7600) support two to 16 chassis, and up to 128 half-width blades. Each chassis can be
connected with two links (Cisco UCS 2204XP fabric extenders input/output module (IOM) only), four links
(Cisco UCS 2204XP fabric extenders IOM only), or eight links (Cisco UCS 2208XP fabric extenders IOM
only) per IOM.
65
The following table shows the compute options available for the fabric interconnects:
Cisco UCS 6248UP 2 (2) 16 (128) 8 (64) 4 (32)
Cisco UCS 6296UP 2 (2) N/A 16 (128) 8 (64)
Connectivity
MDS 9148 multilayer fabric switches, based on the topology.
The following table shows the available switch combinations available for the fabric interconnects:



Array options
support up to six X-Blades and ships with two X-Blades and two control stations. Each X-Blade provides
four 10G front-end connections to the network. An additional data mover enclosure (DME) supports the
connection of one additional X-Blade with the same configuration as the base data movers.
66
Block 6 N/A
Unified 6 2
Unified 6 3*
Unified 6 4*
Unified 6 5*
Unified 6 6*
VCE supports two to six X-Blades in VCE Systems (5800).
• 12 GB RAM
Feature options
The VCE Systems (5800) support both Ethernet and FC bandwidth (BW) enhancement. Ethernet BW
SAN connectivity is provided by Cisco MDS 9148 multilayer fabric switches or the Cisco Nexus 5596UP
switches, depending on topology. Both block and unified arrays use FC BW enhancement.
Note: Unified networking is supported only on VCE Systems (5800) with Cisco Nexus 5596UP switches.
VCE Systems (5800) have two 25 slot 2.5" disk array enclosure (DAEs). The EMC VNX5800 data
processor enclosure (DPE) provides the DAE for bus 0, and the second provides the first DAE on bus 1.
67
An additional four DAEs are required beyond the base two DAEs. Additional DAEs can be added in either
15 slot 3.5" DAEs or 25 slot 2.5" DAEs. Additional DAEs (after initial six) are added in multiples of six.
DAEs are interlaced when racked, and all 2.5" DAEs are racked first on the buses, then 3.5" DAEs.
SLIC configuration
The EMC VNX5800 provides slots for five SLICs in each service processor. Slot 0 is populated with a
back-end SAS bus module. VCE Systems (5800) support two FC SLICs per SP for host connectivity. A
third is reserved to support unified storage. If FC BW enhancement is configured, an additional FC SLIC
is added to the array. VCE only supports the four-port FC SLIC for host connectivity. By default, six FC
ports per SP are connected to the SAN switches for VCE Systems host connectivity. The addition of FC
BW enhancement provides four additional FC ports per SP.
Block Y Bus FC FC FC N/A
Unified (<5DM)* Y Bus FC FC FC FC/U
Unified N Bus FC FC FC/U FC/U
Greater than four X-Blades prohibits FC BW enhancement.
FC 4xFC port input/output module (IOM): provides four 8G FC connections.
FC/U 4xFC port IOM dedicated to unified X-Blade connectivity: provides four 8G FC connections.
Compute
VCE Systems (5800) support two to 16 chassis, and up to 128 half-width blades. Each chassis can be
connected with two links (Cisco UCS 2204XP fabric extenders IOM only), four links (Cisco UCS 2204XP
fabric extenders IOM only) or eight links (Cisco UCS 2208XP fabric extenders IOM only) per IOM.
Fabric interconnect Min chassis 2-link max chassis 4-link max chassis 8-link max chassis
(blades) (blades) (blades) (blades)
Cisco UCS 6248UP 2 (2) 16 (128) 8 (64) 4 (32)
Cisco UCS 6296UP 2 (2) N/A 16 (128) 8 (64)
68
Connectivity
Ethernet connectivity. SAN connectivity is provided by the Cisco Nexus 5500 switches or Cisco MDS
9148 multilayer fabric switches, based on topology.
The following table shows all the available switch combinations that are available for the fabric
interconnects:



Array options
support one to four X-Blades and two control stations. Each X-Blade provides two 10G front-end
connections to the network.
Block 2 or 6 N/A
Unified 2 or 6 1
Unified 2 or 6 2*
Unified 2 or 6 3*
Unified 2 or 6 4*
69
*VCE supports one to four X-Blades in VCE Systems (5600).
• Six GB RAM
• One 2-port 10 GB SFP+ compatible SLICs
Feature options
VCE Systems (5600) use the Cisco Nexus 5596UP switches. VCE Systems (5600) do not support FC
bandwidth (BW) enhancement in block or unified arrays.
Array Topology Ethernet BW enhancement
Block Segregated Y
Unified Segregated Y
Block Unified network Y
Unified Unified network Y
DAE configuration
VCE Systems (5600) have two 25 slot 2.5" disk array enclosure (DAEs). The EMCVNX 5600 disk
processor enclosure (DPE) provides the DAE for bus 0, the second provides the first DAE on bus 1.
Additional DAEs can be in either 15 slot 3.5" DAEs or 25 slot 2.5? DAEs. Additional DAEs are added in
multiples of two. DAEs are interlaced when racked, and all 2.5" DAEs are racked first on the buses, then
3.5" DAEs.
An additional four port SAS bus expansion SLIC is an option with VCE Systems (5600). If more than 19
DAEs are required, the addition of a four port expansion bus card is required. If the card is added, DAEs
are purchased in groups of six.
SLIC configuration
The EMC VNX5600 provides slots for five SLICs in each service processor. VCE Systems (5600) have
two FC SLICs per SP for host connectivity. A third FC SLIC can be ordered to support unified storage.
The remaining SLIC slots are reserved for future VCE configuration options. VCE only supports the four
port FC SLIC for host connectivity. Six FC ports per SP are connected to the SAN switches for VCE
Systems host connectivity.
70
Array FC bandwidth enhancement SLIC 0 SLIC 1 SLIC 2 SLIC 3 SLIC 4
Unified N Bus FC FC N/A FC/U
The FC 4xFC port I/O module provides four 8G FC connections. The FC/U 4xFC port IO module (IOM)
dedicated to unified X-Blade connectivity provides four 8G FC connections. Bus four port - 4x lane/port
six GB SAS: provides additional back-end bus connections.
Compute
VCE Systems (5600) support two to eight chassis and up to 64 half-width blades. Each chassis can be
connected with four links (Cisco UCS 2204XP fabric extenders IOM only) or eight links (Cisco UCS
2208XP fabric extenders IOM only) per IOM.
Cisco UCS 6248UP 2 (2) N/A 8 (64) 4 (32)
Cisco UCS 6296UP 2 (2) N/A 16 (128) 8 (64)
Connectivity
MDS 9148 multilayer fabric switches, based on topology.
The following table shows the switch options that are available for the fabric interconnects:
Fabric Interconnect Topology Ethernet SAN
Unified network Cisco Nexus 5548UP switches

71
Fabric Interconnect Topology Ethernet SAN


Array options
support one to four X-Blades and two control stations. Each X-Blade provides two 10G front-end
connections to the network.
Block 2 N/A
Unified 2 1*
Unified 2 2*
Unified 2 3*
Unified 2 4*
*VCE supports one to four X-Blades in VCE Systems (5400).
• Six GB RAM
• One 2-port 10 GB SFP+ compatible SLICs
Feature options
VCE Systems (5400) use the Cisco UCS 6248UP fabric interconnects. VCE Systems (5400) do not
support FC bandwidth (BW) enhancement or Ethernet BW enhancement in block or unified arrays.
72
VCE Systems (5400) have two 25 slot 2.5" disk array enclosure (DAEs). The EMC VNX5400 disk
processor enclosure (DPE) provides the DAE for bus 0, the second provides the first DAE on bus 1.
Additional DAEs can be in either 15 slot 3.5" DAEs or 25 slot 2.5" DAEs. Additional DAEs are added in
multiples of two. DAEs are interlaced when racked, and all 2.5" DAEs are racked first on the buses, then
3.5" DAEs.
SLIC configuration
EMC VNX5400 provides slots for five SLICs in each service processor (SP), although only four are
enabled. VCE Systems (5400) have two FC SLICs per SP for host connectivity. A third FC SLIC can be
ordered to support unified storage. The remaining SLIC slots are reserved for future VCE configuration
options. VCE only supports the four-port FC SLIC for host connectivity. Six FC ports per SP are
connected to the SAN switches for VCE Systems host connectivity.
Block N N/A FC FC N/A N/A
Unified N N/A FC FC N/A FC/U
The FC 4xFC port I/O module (IOM) provides four 8G FC connections. The FC/U 4xFC port IOM
dedicated to unified X-Blade connectivity provides four 8G FC connections.
Compute
VCE Systems (5400) are configured with two chassis that support up to 16 half-width blades. Each
chassis is connected with four links per fabric extender I/O module (IOM). VCE Systems (5400) support
Cisco UCS 2204XP Fabric Extenders IOM only.
The following table shows the compute options that are available for the Cisco UCS 6248UP fabric
interconnects:
Fabric interconnect Min chassis 2-link max chassis 4-link max chassis 8-link max chassis
(blades) (blades) (blades) (blades)
Cisco UCS 6248UP 2 (2) N/A 2 (16) N/A
Connectivity
VCE Systems (5400) contain the Cisco UCS 6248UP fabric interconnects that uplink to Cisco UCS Nexus
5548UP switches for Ethernet connectivity. SAN connectivity is provided by the Cisco Nexus 5548UP
switches or Cisco MDS 9148 multilayer fabric switches.
73
The following table shows the switch options that are available for the fabric interconnects:

Note: The default is Cisco Nexus 5596UP switches.
74
Sample configurations VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
Sample configurations
Sample VCE System with EMC VNX8000

VCE Systems with EMC VNX8000 cabinet elevations vary based on the specific configuration
requirements.
These elevations are provided for sample purposes only. For specifications for a specific VCE System
design, consult your vArchitect.
VCE System with EMC VNX8000 front view
75
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Sample configurations
VCE System with EMC VNX8000 rear view
76
VCE System with EMC VNX8000 cabinet 1
77
78
79
80
Sample VCE System with EMC VNX5800

VCE Systems with EMC VNX5800 cabinet elevations vary based on the specific configuration
requirements.
81
VCE System with EMC VNX5800 front view
82
VCE System with EMC VNX5800 rear view
83
84
85
Sample VCE System with EMC VNX5800 (ACI ready)

VCE Systems with EMC VNX5800 elevations for a cabinet that is Cisco Application Centric Infrastructure
(ACI) vary based on the specific configuration requirements.
86
VCE System with EMC VNX5800 (ACI ready) front view
87
VCE System with EMC VNX5800 (ACI ready) rear view
88
VCE System with EMC VNX5800 (ACI ready) cabinet 1
89
90
91
VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview Additional references
Additional references
Virtualization components
This topic provides a description of the virtualization components.
Product Description Link to documentation
VMware vCenter Provides a scalable and extensible platform that forms http://www.vmware.com/products/
Server the foundation for virtualization management. vcenter-server/
VMware vSphere Virtualizes all application servers and provides VMware www.vmware.com/products/
ESXi high availability (HA) and dynamic resource scheduling vsphere/
(DRS).
Compute components
This topic provides a description of the compute components.
Product Description Link
Cisco UCS B-Series Servers that adapt to application demands, www.cisco.com/en/US/products/

Blade Servers intelligently scale energy use, and offer best- ps10280/index.html
in-class virtualization.
Cisco UCS Manager Provides centralized management capabilities www.cisco.com/en/US/products/

for the Cisco Unified Computing System ps10281/index.html
(UCS).
Cisco UCS 2200 Series Bring unified fabric into the blade-server www.cisco.com/en/US/prod/collateral/
Fabric Extenders chassis, providing up to eight 10 Gbps ps10265/ps10276/
connections each between blade servers and data_sheet_c78-675243.html
the fabric interconnect.
Cisco UCS 5100 Series Chassis that supports up to eight blade www.cisco.com/en/US/products/
Blade Server Chassis servers and up to two fabric extenders in a six ps10279/index.html
rack unit (RU) enclosure.
Cisco UCS 6200 Series Cisco UCS family of line-rate, low-latency, www.cisco.com/en/US/products/
Fabric Interconnects lossless, 10 Gigabit Ethernet, Fibre Channel ps11544/index.html
over Ethernet (FCoE), and Fibre Channel
functions. Provide network connectivity and
management capabilities.
92
Additional references VCE™ Vblock® and VxBlock™ Systems 340 Architecture Overview
Network components
This topic provides a description of the network components.
Cisco Nexus 1000V A software switch on a server that delivers Cisco www.cisco.com/en/US/products/
Series Switches VN-Link services to virtual machines hosted on that ps9902/index.html
server.
VMware vSphere A VMware vCenter-managed software switch that http://www.vmware.com/products/

Distributed Switch delivers advanced network services to virtual vsphere/features/distributed-
(VDS) machines hosted on that server. switch.html
Cisco MDS 9148 Provides 48 line-rate 16-Gbps ports and offers www.cisco.com/en/US/products/
Multilayer Fabric cost-effective scalability through on-demand ps10703/index.html
Switch activation of ports.
Cisco Nexus 3048 Provides local switching that connects transparently http://www.cisco.com/c/en/us/
Switch to upstream Cisco Nexus switches, creating an products/switches/nexus-3048-switch/
end-to-end Cisco Nexus fabric in data centers. index.html
Cisco Nexus 5000 Simplifies data center transformation by enabling a http://www.cisco.com/c/en/us/

Series Switches standards-based, high-performance unified fabric. products/switches/nexus-5000-series-
switches/index.html
Cisco Nexus 7000 A single end-to-end platform designed around http://www.cisco.com/c/en/us/

Series Switches infrastructure scalability, operational continuity, and products/switches/nexus-7000-series-
transport flexibility. switches/index.html
Cisco Nexus 9396PX Provides high scalability, performance, and http://www.cisco.com/c/en/us/support/

Switch exceptional energy efficiency in a compact form switches/nexus-9396px-switch/
factor. Designed to support Cisco Application model.html
Centric Infrastructure (ACI).
Storage components
This topic provides a description of the storage components.
EMC VNX8000, EMC VNX7600, High-performing unified storage with www.emc.com/products/series/

EMC VNX5800, EMC VNX5600, unsurpassed simplicity and efficiency, vnx-series.htm
EMC VNX5400 storage arrays optimized for virtual applications.
93
www.vce.com
About VCE
VCE, an EMC Federation Company, is the world market leader in converged infrastructure and converged solutions. VCE
accelerates the adoption of converged infrastructure and cloud-based computing models that reduce IT costs while
improving time to market. VCE delivers the industry's only fully integrated and virtualized cloud infrastructure
systems, allowing customers to focus on business innovation instead of integrating, validating, and managing IT
infrastructure. VCE solutions are available through an extensive partner network, and cover horizontal applications, vertical
industry offerings, and application development environments, allowing customers to focus on business innovation instead
of integrating, validating, and managing IT infrastructure.
For more information, go to http://www.vce.com.
Copyright © 2013-2015 VCE Company, LLC. All rights reserved. VCE, VCE Vision, VCE Vscale, Vblock, VxBlock, VxRack,
and the VCE logo are registered trademarks or trademarks of VCE Company LLC. All other trademarks used herein are
the property of their respective owners.
94

Vce 340 Gen3 3 Architecture Overview

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vce 340 Gen3 3 Architecture Overview

Uploaded by

Copyright:

Available Formats

www.vce.

VCE Vblock® and VxBlock™

Document revision 3.8

Accessing VCE documentation.................................................................................................................6

Compute layer overview...........................................................................................................................24

VMware vSphere Hypervisor ESXi.......................................................................................................46

December 2014 Gen 3.2 3.6 Added support for AMP-2HA.

October 2013 Gen 3.1 3.0 Gen 3.1 release

Accessing VCE documentation (see page 6)

Accessing VCE documentation

Cisco, EMC, VMware partner.vce.com

VCE employee sales.vce.com/saleslibrary

System architecture and components

VCE Systems have a number of features including:

• Block (SAN) and unified storage options (SAN and NAS)

• Second generation of the Advanced Management Platform (AMP-2) for management

VCE Systems • VCE Vision™ Intelligent Operations System Library

Virtualization • VMware vSphere Server Enterprise Plus

Compute • Cisco UCS 5108 Server Chassis

Network • Cisco Nexus 3048 Switches

Accessing VCE documentation (see page 6)

EMC VNX series storage arrays (see page 30)

Base configurations and scaling

Hardware How it can be customized

Compute chassis Cisco UCS Server Chassis

Hardware How it can be customized

Supported disk drives FastCache

EMC VNX series storage arrays (see page 30)

Scaling up compute resources (see page 26)

Scaling up storage resources (see page 32)

Management components overview (see page 50)

Replication (see page 32)

Storage Contains the EMC VNX storage component.

All components incorporate redundancy into the design.

Segregated network architecture and unified network architecture

Accessing VCE documentation (see page 6)

Management components overview (see page 50)

Segregated network architecture (see page 12)

Unified network architecture (see page 17)

Segregated network architecture

Block storage configuration

SAN boot storage configuration

Unified storage configuration

Connectivity overview (see page 11)

Unified network architecture (see page 17)

Unified network architecture

Block storage configuration

The following illustration shows a block-only storage configuration in VCE Systems:

SAN boot storage configuration

Unified storage configuration

Connectivity overview (see page 11)

Management components overview (see page 50)

Segregated network architecture (see page 12)

Cisco Unified Computing System

VCE Systems powered by Cisco UCS feature:

• Built-in redundancy for high availability

• Hot-swappable components for serviceability, upgrade, or expansion

• Fewer physical components than in a comparable system built piece by piece

• Improved energy efficiency over traditional blade server chassis

Accessing VCE documentation (see page 6)

Cisco Unified Computing System fabric interconnects

Accessing VCE documentation (see page 6)

Cisco Trusted Platform Module

Scaling up compute resources