N5k N2k Ethernet FCoE Lab Guide Student v2 1

Nexus 5000/Nexus 2000 Lab Guide
Overview
This lab is designed to help attendees understand how to deploy Nexus 5000 and Nexus
2000 in the datacenter. It covers the configuration and management perspective for Nexus
5000 and Nexus 2000. Attendees will learn how to design and deploy

Lab List
This guide includes these activities:
! Lab 1: Basic configuration of N5k
! Lab 2: Deploy N5k/N2k using EtherChannel
! Lab 3: Deploy N5k/N2k using static pinning
! Lab 4: Configuration of FCoE

Nexus 5000 and Nexus 2000 Overview

The Cisco Nexus 5000 Series Switches comprise a family of line-rate, low-latency,
lossless 10 Gigabit Ethernet, Cisco
Data Center Ethernet, and Fibre Channel over Ethernet

(FCoE) switches for data center applications. The switch family is highly serviceable, with
redundant, hot-pluggable power supplies and fan modules. Its software is based on data
center-class Cisco NX-OS Software for high reliability and ease of management.

The Cisco Nexus 2148T provides 48 Gigabit Ethernet server ports and 4 10 Gigabit Ethernet
uplink ports in a compact one rack-unit (1RU) form factor. The Cisco Nexus 2000 Series
Fabric Extender integrates with the parent switch, allowing "zero-touch" provisioning as
well automatic configuration and software upgrade when a rack is connected to the parent
switch. This integration allows large numbers of servers to be supported with the same
feature set of the parent Cisco Nexus 5000 Series switch, along with specific configuration
parameters including security and quality of service (QoS). The FEX does not run Spanning
Tree Protocol even when multiple links are connected to the parent switch, thus giving
customers loop-free, active-active connectivity between the two.

Lab Topology and Access

Lvery one or Lwo sLudenLs wlll share one Cu. Lach one Cu lncludes one nexus 3000 and one nexus
2000 swlLch. 1he n2k ls connecLed Lo lnLerface e1/19 and e1/20 of nexus 3000.
Lab Topology

IP address and console assignment

Network: 128.107.65.192/27 255.255.255.224
Gateway 128.107.65.193
Terminal
Server: 128.107.65.194

Servers IP Address Subnet Console APC APC Port

Server-1 128.107.65.211 128.107.65.192 128.107.65.221 22,23
Server-2 128.107.65.212 128.107.65.192 128.107.65.221 20,21
Server-3 128.107.65.213 128.107.65.192 128.107.65.221 18,19
Server-4 128.107.65.214 128.107.65.192 128.107.65.221 15,17
Server-5 128.107.65.215 128.107.65.192 128.107.65.221 7,9
Server-6 128.107.65.216 128.107.65.192 128.107.65.221 5,6
Server-7 128.107.65.217 128.107.65.192 128.107.65.221 3,4
Server-8 128.107.65.218 128.107.65.192 128.107.65.221 1,2

N5k IP Address Subnet Console APC APC Port

TM1 128.107.65.201 128.107.65.192 2033 128.107.65.220 23
TM2 128.107.65.202 128.107.65.192 2034 128.107.65.220 22
TM3 128.107.65.203 128.107.65.192 2035 128.107.65.220 21
TM4 128.107.65.204 128.107.65.192 2036 128.107.65.220 20
TM5 128.107.65.205 128.107.65.192 2037 128.107.65.220 19
TM6 128.107.65.206 128.107.65.192 2038 128.107.65.220 18
TM7 128.107.65.207 128.107.65.192 2039 128.107.65.220 17
TM8 128.107.65.208 128.107.65.192 2040 128.107.65.220 15

Core IP Address Subnet Console APC APC Port

6506-1 N/A N/A 2042 128.107.65.221 8
Terminal
Server 128.107.65.194 login --> labops 128.107.65.220 6

Password:
Terminal server Login password: labops. Enable password: nuovanuova
Nexus 5000: admin/nbv12345
Windows Server: administrator/nuovanuova
APC: apc/apc
Lab 1 Basic configuration of Nexus 5000

Lab Objective
ln Lhls exerclse you wlll conflgure Lhe baslc connecLlvlLy Lo nexus 3000 and geL famlllar wlLh Lhe mosL
commonly used nx-CS CLl
Lab Procedure
Step 1. Access the switch
Telnet to 128.107.65.194, sign on with a password of labops
Issue the show host command:
Nuova_TermServer>show host
[snip]
Host Port Flags Age Type Address(es)
tm1 2033 (perm, OK) 21 IP 1.1.1.1
tm2 2034 (perm, OK) 21 IP 1.1.1.1
tm3 2035 (perm, OK) 21 IP 1.1.1.1
tm4 2036 (perm, OK) 21 IP 1.1.1.1
tm5 2037 (perm, OK) 21 IP 1.1.1.1
tm6 2038 (perm, OK) 21 IP 1.1.1.1
tm7 2039 (perm, OK) 21 IP 1.1.1.1
tm9 2044 (perm, OK) 21 IP 1.1.1.1
tm8 2040 (perm, OK) 21 IP 1.1.1.1
tm6500 2042 (perm, OK) 21 IP 1.1.1.1
mds 2047 (perm, OK) 21 IP 1.1.1.1

Console in to the 5020 for your pod, for example, POD 1 would be TM1
Nuova_TermServer>tm1
Translating "tm1"
Trying tm1 (128.107.65.194, 2033)...
% Connection refused by remote host
Deleting login session
If you see the above error message you will need to get enable access to the terminal server
and clear the line as shown below. The enable password is nuovanuova. If you do not see the
error message above you can proceed to step 2
Nuova_TermServer>ena
Password:
Nuova_TermServer#clear line 33
[confirm] <- Hit enter
[OK]
Nuova_TermServer#tm1
Translating "tm1"
Trying tm1 (128.107.65.194, 2033)... Open " Hit enter here
switch# <- Should see this prompt

Step 2. Conflgure Lhe swlLch name for Lhe 3020 ln your pod, managemenL v8l, enable SSP server,
and verlfy connecLlvlLy Lhrough Lhe managemenL porL.

switch(config)# hostname TM1 " POD specific
TM1(config)# int mgmt 0
TM1(config-if)# ip address 128.107.65.201/24 " Replace with the
address for the 5020 in
TM1(config-if)# vrf context management
TM1(config-vrf)# ip route 0.0.0.0/0 128.107.65.193
TM1(config-vrf)# sh run int mgmt0
version 4.0(1a)N2(1)

interface mgmt0
ip address 128.107.65.201/24

TM1(config-vrf)# exit
TM1(config)# ssh key rsa 1024
rsa keys already present, use force option to overwrite them
TM1(config)# no ssh server enable
XML interface to system may become unavailable since ssh is disabled
TM1(config)# no ssh key
TM1(config)# ssh key rsa 1024
generating rsa key(1024 bits).....
.
generated rsa key
TM1(config)# ssh server enable
TM1(config)# end
LxecuLe show command
dlrecLly from conflguraLlon
mode
ulsable SSP and deleLe SSP
key lf Lhls error message ls
prlnLed

Ping the default gateway for the management network.
TM1# ping 128.107.65.193 vrf management
PING 128.107.65.193 (128.107.65.193): 56 data bytes
64 bytes from 128.107.65.193: icmp_seq=0 ttl=254 time=2.167 ms

--- 128.107.65.193 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 1.748/1.984/2.167 ms
TM1#

!"#$ &' 1elneL or SSP Lo Lhe nexus 3000 ln your Cu and geL famlllar wlLh nx-CS CLl

Check Lhe sofLware verslon and lmage flle locaLlon
TM1# sh version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2009, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.

Software
BIOS: version 1.2.0
loader: version N/A
kickstart: version 4.0(1a)N2(1)
system: version 4.0(1a)N2(1)
BIOS compile time: 06/19/08
kickstart image file is: bootflash:/n5000-uk9-kickstart.4.0.1a.N2.1.bin
kickstart compile time: 2/25/2009 0:00:00 [02/25/2009 08:29:12]
nx-CS sofLware verslon
lmage flle locaLlon
system image file is: bootflash:/n5000-uk9.4.0.1a.N2.1.bin
system compile time: 2/25/2009 0:00:00 [02/25/2009 08:56:57]

Hardware
cisco Nexus5020 Chassis ("40x10GE/Supervisor")
Intel(R) Celeron(R) M CPU with 2074308 kB of memory.
Processor Board ID JAB123900G8

Device name: TM1
bootflash: 974848 kB

Kernel uptime is 2 day(s), 22 hour(s), 51 minute(s), 8 second(s)

Last reset at 340459 usecs after Tue Mar 10 23:55:39 2009

Reason: Reset by installer
System version: 4.0(0)N1(2)
Service:

plugin
Core Plugin, Ethernet Plugin
Check Lhe expanslon module of nexus 3000
TM1# show module
Mod Ports Module-Type Model Status
--- ----- -------------------------------- ---------------------- ------------
1 40 40x10GE/Supervisor N5K-C5020P-BFS active *
2 8 4x10GE + 4x1/2/4G FC Module N5K-M1404 ok
3 8 4x10GE + 4x1/2/4G FC Module N5K-M1404 ok

Mod Sw Hw World-Wide-Name(s) (WWN)
--- -------------- ------ --------------------------------------------------
1 4.0(1a)N2(1) 1.0 --
2 4.0(1a)N2(1) 1.0 20:41:00:0d:ec:a3:db:80 to 20:44:00:0d:ec:a3:db:80
Lxpanslon modules
3 4.0(1a)N2(1) 1.0 20:81:00:0d:ec:a3:db:80 to 20:84:00:0d:ec:a3:db:80

Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 000d.eca3.db88 to 000d.eca3.dbaf JAB123900G8
2 000d.eca3.dbb0 to 000d.eca3.dbb7 JAB1250004E
3 000d.eca3.dbb8 to 000d.eca3.dbbf JAB12500048
TM1#

Lab 2 Deploy N5k/N2k with EtherChannel

Lab Objective
ln Lhls exerclse you wlll learn how Lo add nexus 2000 Lo nexus 3000 uslng LLherChannel as connecLlon
beLween nexus 2000 and nexus 3000
Lab Procedure
n2k ls managed by Lhe parenL swlLch n3k. AfLer Lhe n3k dlscovers Lhe connecLed n2k lL flrsL checks Lhe
sofLware compaLlblllLy beLween n2k and n3k. 1he conflguraLlon daLa ls Lhen pushed Lo n2k. 1he
conflguraLlon ls malnLalned by n3k. 1here ls no local conflguraLlon daLa sLored on n2k. n3k collecLs and
malnLaln Lhe Lrafflc sLaLlsLlcs as well.

1here are Lwo ways Lo connecL n2k Lo n3k:
LLherchannel:llrsL ls Lo bundle all Lhe llnks beLween n2k and n3k(up Lo 4) as an LLherChannel
and Lhe LLherChannel wlll carry Lhe Lrafflc for all Lhe 48 n2k porLs.
SLaLlc plnnlng": Lach of Lhe llnks beLween n3k and n2k carrles Lrafflc for a group of n2k porLs.
ln Lhls lab exerclse we wlll learn how Lo connecL n2k Lo n3k uslng LLherChannel
!"#$ ( Add n2k relaLed conflguraLlon on n3k
TM1(config)# fex 100
TM1(config-fex)# pinning max-links 1
TM1(config-fex)# description "fex 100 pod1"
TM1(config-fex)# int e1/19-20
TM1(config-if-range)# channel-group 100
TM1(config-if-range)# int port-channel 100
TM1(config-if)# switchport mode fex-fabric
TM1(config-if)# fex associate 100
TM1(config-if)# end

!"#$ ) Check n2k sLaLus and verlfy LhaL n2k ls up and onllne.

ulsplay all n2ks connecLed Lo Lhls n3k
TM1# sh fex
FEX FEX FEX FEX
Number Description State Model Serial
--------------------------------------------------------------------
----
LLherChannel mode ls Cn. no
LAC beLween n3k and n2k

LLherChannel ls conslders as
one loglcal llnk

100 fex 100 TM1 Online N2K-C2148T-1GE
JAF1244ATEQ
ulsplay Lhe deLall lnformaLlon abouL a n2k
TM1# sh fex 100 detail
FEX: 100 Description: fex 100 TM1 state: Online
FEX version: 4.0(1a)N2(1) [Switch version: 4.0(1a)N2(1)]
FEX Interim version: 4.0(1a)N2(1)
Switch Interim version: 4.0(1a)N2(1)
Extender Model: N2K-C2148T-1GE, Extender Serial: JAF1244ATEQ
Part No: 73-12009-02
Card Id: 70, Mac Addr: 00:0d:ec:b1:3e:82, Num Macs: 64
Module Sw Gen: 19 [Switch Sw Gen: 19]
pinning-mode: static Max-links: 1
Fabric port for control traffic: Eth1/1
Fabric interface state:
Po100 - Interface Up. State: Active
Eth1/1 - Interface Up. State: Active
Fex Port State Fabric Port Primary Fabric
Eth100/1/1 Down Po100 Po100
Eth100/1/5 Up Po100 Po100
Eth100/1/10 Down Po100 Po100
Eth100/1/11 Down Po100 Po100
Eth100/1/12 Down Po100 Po100
Eth100/1/13 Down Po100 Po100
Eth100/1/14 Down Po100 Po100
Eth100/1/15 Down Po100 Po100
Eth100/1/16 Down Po100 Po100
Eth100/1/17 Down Po100 Po100
Eth100/1/18 Down Po100 Po100
Eth100/1/19 Down Po100 Po100
Eth100/1/20 Down Po100 Po100
Eth100/1/21 Down Po100 Po100
Eth100/1/22 Down Po100 Po100
Eth100/1/23 Down Po100 Po100
Eth100/1/24 Down Po100 Po100
Eth100/1/25 Down Po100 Po100
Eth100/1/26 Down Po100 Po100
Eth100/1/27 Down Po100 Po100
Eth100/1/28 Down Po100 Po100
Eth100/1/29 Down Po100 Po100
Eth100/1/30 Down Po100 Po100
Eth100/1/31 Down Po100 Po100
Eth100/1/32 Down Po100 Po100
Eth100/1/33 Down Po100 Po100
Eth100/1/34 Down Po100 Po100
Eth100/1/35 Down Po100 Po100
Eth100/1/36 Down Po100 Po100
Eth100/1/37 Down Po100 Po100
Eth100/1/38 Down Po100 Po100
Eth100/1/39 Down Po100 Po100
Eth100/1/40 Down Po100 Po100
Eth100/1/41 Down Po100 Po100
Eth100/1/42 Down Po100 Po100
Eth100/1/43 Down Po100 Po100
Eth100/1/44 Down Po100 Po100
Eth100/1/45 Down Po100 Po100
Eth100/1/46 Down Po100 Po100
Eth100/1/47 Down Po100 Po100
Eth100/1/48 Down Po100 Po100
Logs:
[03/18/2009 00:01:47.921366] Module register received
[03/18/2009 00:01:47.922299] Registration response sent
[03/18/2009 00:01:47.950304] Module Online Sequence
[03/18/2009 00:01:49.252440] Module Online
TM1#

!"#$ & Conflgure n2k lnLerfaces
TM1(config)# vlan 10X " Replace X with your POD number
TM1(config-vlan)# int ethernet 100/1/1
TM1(config-if)# switchport access vlan 10X " Replace X with
your POD number
lease noLe LhaL Lhe fex lnLerfaces supporL 802.1q Lrunklng and can be conflgured as Lrunk
porL.
!"#$ * Conflgure lnLerface connecLed Lo CaLalysL 6300
TM1(config)# feature lacp
TM1(config)# interface port-channel 1
TM1(config-if-range)# exit
TM1(config)# interface ethernet 1/1-2
TM1(config-if-range)# channel-group 1 mode active
Ethernet1/1 Ethernet1/2 added to port-channel1
TM1(config-if-range)# interface port-channel 1
TM1(config-if)# switchport mode trunk
TM1(config-if)# switchport trunk allowed vlan 10X " Again POD specific
TM1(config-if)# no shut

Verify the Port channel is up and both interfaces are up and active in the port channel
TM1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
--------------------------------------------------------------------
-----------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------
-----------
1 Po100(SU) Eth LACP Eth1/1(P) Eth1/2(P)
!"#$ + verlfy server connecLlvlLy
llnd ouL Lhe l address of Lhe server ln your Cu. 8emoLe ueskLop Lo Lhe server and asslgn l address
for Lhe nlC porL connecLed Lo lLx. 1he l address for Lhe server ls 100.100.x.10 wlLh x" belng your Cu
lu.

llrsL sLarL Lhe neLwork connecLlon wlndow by cllcklng Lhe neLwork lcon on Lhe rlghL boLLom of Lhe
deskLop.

1hen double cllck Lhe neLwork connecLlon named nexus2000".

Asslgn l address Lo Lhe 1CL nlC porL named nexus2000"

1he l address should be 100.100.x.10" x" belng Lhe Cu lu.
Cu Server nlC address CaLalysL 6300 address
1 100.100.1.10 100.100.1.1
2 100.100.2.10 100.100.2.1
3 100.100.3.10 100.100.3.1
4 100.100.4.10 100.100.4.1
3 100.100.3.10 100.100.3.1
6 100.100.6.10 100.100.6.1
7 100.100.7.10 100.100.7.1
8 100.100.8.10 100.100.8.1

lng Lhe CaLalysL 6300 Svl lp address: 100.100.x.1

!"#$ , SlmulaLe Lhe replacemenL of n2k by shuL down/no shuL down Lhe fabrlc llnk beLween n3k
and n2k. ln Lhls sLep we wlll slmulaLe Lhe scenarlo where we need Lo replace an exlsLlng n2k by
dlsconnecL and reconnecL Lhe n2k Lo n3k. noLe LhaL Lhe conflguraLlon of n2k are resLored auLomaLlcally
afLer Lhe new n2k comes up and onllne.
Check Lhe n2k lnLerfaces conflguraLlon
TM1(config-if-range)# sh run int ethernet 100/1/1

interface Ethernet100/1/1
spanning-tree port type edge
switchport access vlan 101

ShuLdown all Lhe fabrlc lnLerfaces
TM1(config)# int ethernet 1/19-20
TM1(config-if-range)# shut
2009 Mar 24 22:33:28 TM1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-
channel1: Ethernet1/2 is down
2009 Mar 24 22:33:28 TM1 %ETH_PORT_CHANNEL-5-FOP_CHANGED:
port-channel1: first operational port changed from
Ethernet1/2 to Ethernet1/1
2009 Mar 24 22:33:28 TM1 %NOHMS-2-NOHMS_ENV_FEX_OFFLINE: FEX-
100 Off-line (Serial Number JAF1241BLGA)
2009 Mar 24 22:33:28 TM1 %ETHPORT-5-IF_DOWN_MODULE_REMOVED:
Interface Ethernet100/1/1 is down (module removed)
channel1: Ethernet1/1 is down
channel1: port-channel1 is down
2009 Mar 24 22:33:31 TM1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-
channel1: first operational port changed from Ethernet1/1 to none
Check all Lhe n2k lnLerfaces are no long exlsL on n3k. nx-CS malnLalns Lhe conflguraLlon so LhaL when
Lhe lLx 100 comes up onllne Lhe conflguraLlon wlll be resLored.
^
% invalid command detected at '^' marker.
Invalid range
8rlng up Lhe fabrlc lnLerfaces
TM1(config-if-range)# no shut
TM1(config-if-range)# 2009 Mar 24 22:33:59 TM1
%ETH_PORT_CHANNEL-5-PORT_UP: port-channel1: Ethernet1/2 is up
2009 Mar 24 22:33:59 TM1 %ETH_PORT_CHANNEL-5-FOP_CHANGED:
port-channel1: first operational port changed from none to
Ethernet1/2
2009 Mar 24 22:34:00 TM1 %ETH_PORT_CHANNEL-5-PORT_UP: port-
channel1: Ethernet1/1 is up
2009 Mar 24 22:34:09 TM1 %SATCTRL-FEX100-2-SOHMS_ENV_ERROR:
FEX-100 Module 1: Check environment alarms.
2009 Mar 24 22:34:10 TM1 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-
100 On-line (Serial Number JAF1241BLGA)

Check Lhe n2k comes onllne and all Lhe n2k lnLerfaces relaLed conflguraLlon are resLored auLomaLlcally


interface Ethernet100/1/1
spanning-tree port type edge
switchport access vlan 101
TM1(config-if-range)# sh int ethernet 100/1/1
Ethernet100/1/1 is up
Hardware is 1000 Ethernet, address is 000d.ecb1.2cc2 (bia
000d.ecb1.2cc2)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA

!"#$ - Check Lhe Lrafflc mapplng beLween fabrlc lnLerface and server faclng lnLerfaces.
TM1# sh interface Po100 fex-intf
Fabric FEX
Interface Interfaces
---------------------------------------------------
Po100 Eth100/1/1 Eth100/1/2 Eth100/1/3 Eth100/1/4
Eth100/1/5 Eth100/1/6 Eth100/1/7 Eth100/1/8
Eth100/1/9 Eth100/1/10 Eth100/1/11 Eth100/1/12
Eth100/1/13 Eth100/1/14 Eth100/1/15 Eth100/1/16
Eth100/1/17 Eth100/1/18 Eth100/1/19 Eth100/1/20
Eth100/1/21 Eth100/1/22 Eth100/1/23 Eth100/1/24
Eth100/1/25 Eth100/1/26 Eth100/1/27 Eth100/1/28
Eth100/1/29 Eth100/1/30 Eth100/1/31 Eth100/1/32
Eth100/1/33 Eth100/1/34 Eth100/1/35 Eth100/1/36
Eth100/1/37 Eth100/1/38 Eth100/1/39 Eth100/1/40
Eth100/1/41 Eth100/1/42 Eth100/1/43 Eth100/1/44
Eth100/1/45 Eth100/1/46 Eth100/1/47 Eth100/1/48
Lab 3 Deploy N2k with Static Pinning
When n2k ls connecLed Lo n3k wlLh sLaLlc plnnlng, Lhe Lrafflc of 48 server faclng porL are
dlsLrlbuLed among Lhe upllnks ln a flxed fashlon. 1he 48 porLs are dlvlded lnLo Lhe same
number of groups as Lhe number of upllnks beLween n2k and n3k. 1rafflc for each seL of
Lhese porLs ls carrled by one upllnk, unllke Lhe n2k/n3k deploymenL scenarlo wlLh
orLChannel where Lrafflc dlsLrlbuLlon ls deLermlned by Lhe hash algorlLhm resulL.

!"#$ ( llrsL remove Lhe conflguraLlon of prevlous lab.
TM1(config)# no int Po100
TM1(config-fex)# int e1/19-20
TM1(config-if-range)# no switchport mode
TM1(config-if-range)# no fex associate 100
TM1(config-if-range)# no fex 100
TM1(config)#

Add n2k relaLed conflguraLlon on n3k
TM1(config)# fex 100
TM1(config-fex)# pinning max-links 2
Change in Max-links will cause traffic disruption.
TM1(config-fex)# description "POD1 FEX 100"
TM1(config-fex)# int ethernet 1/19-20
TM1(config-if-range)# switchport mode fex-fabric
TM1(config-if-range)# fex associate 100
!"#$ ) Check n2k sLaLus and verlfy LhaL n2k ls up and onllne.
TM1# sh fex 100
FEX: 100 Description: POD1 FEX 100 state: Online
FEX version: 4.1(3)N1(1) [Switch version: 4.1(3)N1(1)]
Extender Model: N2K-C2148T-1GE, Extender Serial: JAF1307BHCE
Part No: 73-12009-05
pinning-mode: static Max-links: 2
Fabric port for control traffic: Eth1/19
Fabric interface state:
TM1#
!"#$ & Conflgure n2k lnLerfaces
1hls number has Lo maLch
wlLh Lhe acLual llnks beLween
n2k and n3k.

TM1(config)# vlan 10X " Replace X with your POD number
TM1(config-vlan)# int ethernet 100/1/1
TM1(config-if)# switchport access vlan 10X " Replace X with
your POD number

!"#$ * Check Lhe Lrafflc mapplng beLween fabrlc lnLerface and server faclng lnLerfaces.
TM1# sh interface ethernet 1/19 fex-intf
Fabric FEX
---------------------------------------------------
Eth1/19 Eth100/1/24 Eth100/1/23 Eth100/1/22 Eth100/1/21
Eth100/1/20 Eth100/1/19 Eth100/1/18 Eth100/1/17
Eth100/1/16 Eth100/1/15 Eth100/1/14 Eth100/1/13
Eth100/1/12 Eth100/1/11 Eth100/1/10 Eth100/1/9
Eth100/1/8 Eth100/1/7 Eth100/1/6 Eth100/1/5
Eth100/1/4 Eth100/1/3 Eth100/1/2 Eth100/1/1

TM1# sh interface ethernet 1/20 fex-intf
Fabric FEX
---------------------------------------------------
Eth1/20 Eth100/1/48 Eth100/1/47 Eth100/1/46 Eth100/1/45
Eth100/1/44 Eth100/1/43 Eth100/1/42 Eth100/1/41
Eth100/1/40 Eth100/1/39 Eth100/1/38 Eth100/1/37
Eth100/1/36 Eth100/1/35 Eth100/1/34 Eth100/1/33
Eth100/1/32 Eth100/1/31 Eth100/1/30 Eth100/1/29
Eth100/1/28 Eth100/1/27 Eth100/1/26 Eth100/1/25

!"#$ 5 verlfy server connecLlvlLy by plng Lhe Svl lnLerface of CaLalysL 6300 100.100.x.1".
!"#$ 6 8rlng down one fabrlc lnLerface and verlfy Lhe server lnLerfaces plnned Lo Lhls fabrlc lnLerfaces
are broughL down
TM1(config)# int e1/19
TM1(config-if)# shut
TM1(config-if)# sh int ethernet 100/1/1
Ethernet100/1/1 is down (fabricIfDown)
Hardware: 1000 Ethernet, address: 000d.ecc9.ec42 (bia
000d.ecc9.ec42)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is access
auto-duplex, 1000 Mb/s
<snip> .
TM1(config-if)#

Lab 4 Configuration of FCoE

Lab Objective

In this exercise you will configure basic connectivity to the switch, 802.1q trunking, LACP Port
Channels, Fibre Channel Port Channels, and FCoE connectivity.
Note: The exercises in the lab were performed on POD 1. Please use the correct switches,
VSANs, VLANs, IP addresses, and host associated with your assigned POD.

Lab Procedure

Continuing from the N5K/N2K lab, access the Nexus 5000 switch for your pod via the terminal server.

Step 1. In this step you will enable FCoE feature for the Nexus 5000 switch.

First turn on the FCoE feature. This will require a reboot of the switch to go into effect.
TM1# configure terminal
TM1(config)# feature fcoe
2008 Aug 13 22:56:01 switch %$ VDC-1 %$ %PFMA-2-FC_LICENSE_DESIRED:
FCoE/FC feature will be enabled after the configuration is saved
followed by a reboot

TM1(config)# copy running-config startup-config
[########################################] 100%
Packaging and storing to flash: \
TM1(config)# reload

Step 2. Creating a VSAN for the Fibre Channel Database
Once the switch comes back up, login into the switch and create the vsan needed for your POD.
Please note that the convention used would be VSAN 20X, where X is your Pod #. Example would
be for Pod1, would need to create VSAN 201, Pod 2 would be VSAN 202, etc.
TM1#configure terminal
TM1(config)# vsan database
TM1(config-vsan-db)# vsan 20X " Again POD specific
TM1(config-vsan-db)# end

Step 3. Creating a SAN Port-Channel

TM1(config)# interface san-port-channel 10X " Again POD specific
TM1(config-if)# interface fc2/3-4
TM1(config-if)# channel-group 10X force " Again POD specific
fc2/3 fc2/4 added to san-port-channel 10X and disabled
please do the same operation on the switch at the other end of the
channel, then do "no shutdown" at both ends to bring them up
Please note that the MDS9134 has already been initially configured to
be in switching mode and the Port-Channels has been created on it as
well. So the students nor instructor "DO NOT" need to do anything on
the MDS switch.
TM1(config)# interface san-port-channel 10X " Again POD specific
TM1(config-if)# end

Verify the SAN Port Channel is up and Trunking.
TM1# show interface san-port-channel 10X
san-port-channel 10X is trunking
Hardware is Fibre Channel
Port WWN is 24:0b:00:0d:ec:6d:90:40
Admin port mode is auto, trunk mode is on
snmp link state traps are enabled
Port mode is TE
Port vsan is 1
Speed is 8 Gbps
Trunk vsans (admin allowed and active) (1,20X)
Trunk vsans (up) (1,20X)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
[snip]

TM1# show san-port-channel summary
--------------------------------------------------------------
Group Port Type Protocol Status Member Ports
Channel
-----------------------------------------------------------------
101 san-port-channel 101 FC NONE U fc2/3 (P)
fc2/4 (P)
-----------------------------------------------------------------

Step 4. Configuring a VLAN to be FCoE capable. This also does the VLAN to VSAN mapping.

Note: The VLAN number will be following the convention of VLAN 20X,
where X is your Pod #. Example would be VLAN 201 for Pod 1, VLAN 202
for Pod 2, etc.
TM1# configure terminal
TM1(config)# vlan 20X " Again POD specific
TM1(config-vlan)# fcoe vsan 20X " Again POD specific
TM1(config-vlan)# show vlan fcoe
VLAN VSAN Status
-------- -------- --------
201 201 Operational

Step 5. Create the virtual Fibre Channel interface, and move it to the VSAN assigned to your pod.
TM1(config-vlan)# interface vfc5 " The "vfc" number is not relevant
and can be any number the student chooses. But for consistency,
please use #5 for all pods. All student pods should have interface
"vfc5" when this is completed.
TM1(config-if)# bind interface Ethernet1/5

Add Virtual Fibre Channel (vfc5) Interface to the VSAN database for
your Pod.
TM1 (config-if)#vsan database
TM1 (config-vsan-db)#vsan 20X interface vfc5 " Again POD specific

Step 6. Allow the FCoE VLAN to traverse interface ethernet 1/5
TM1(config-vsan-db)# interface ethernet 1/5
TM1(config-if)# switchport mode trunk
TM1(config-if)# switchport trunk allowed vlan 10X, 20X " Again POD
specific
TM1(config-if)# switchport trunk native vlan 10X " Again POD specific
Step 7. Verify vfc interface is up
Note: If the student does a "show interface vfc5", the vFC interface
will be down awaiting the FCoE VLAN assignment, please be patient and
the CNA port will come up.
vfc5 is down (STP not forwarding in FCoE Mapped VLAN)

Verify your CNA host is logged into the fabric and registered with the
name server
TM1# show interface vfc5
vfc5 is up
Bound interface is Ethernet1/5
Hardware is GigabitEthernet
Port WWN is 20:04:00:0d:ec:b2:51:7f
Admin port mode is F
snmp link state traps are enabled
Port mode is F, FCID is 0x3c0001
Port vsan is 201
Beacon is turned unknown
5 minutes input rate 344 bits/sec, 43 bytes/sec, 0 frames/sec
5 minutes output rate 472 bits/sec, 59 bytes/sec, 0 frames/sec
983 frames input, 99240 bytes
0 discards, 0 errors
1089 frames output, 126152 bytes
0 discards, 0 errors

Step 8. Verifying zoning of the pod is active
TM1# show zoneset active vsan 20X
zoneset name pod1 vsan 201
zone name pod1-server vsan 201
* fcid 0x73001e [pwwn 21:00:00:04:cf:26:96:89] " Disk
* fcid 0x3c0001 [pwwn 10:00:00:00:c9:85:05:48] " CNA

Note: Below is the list of all the active zonesets for all the Pods.
* fcid 0x73001e [pwwn 21:00:00:04:cf:26:96:89]
* fcid 0x3c0001 [pwwn 10:00:00:00:c9:85:05:48]

* fcid 0xdb006e [pwwn 21:00:00:04:cf:67:74:ff]
* fcid 0x470001 [pwwn 10:00:00:00:c9:85:03:c8]

* fcid 0xa10002 [pwwn 22:00:00:20:37:f7:da:31]
* fcid 0x2b0001 [pwwn 10:00:00:00:c9:85:08:b0]

* fcid 0x4d0000 [pwwn 10:00:00:00:c9:85:00:d8]
* fcid 0xe10002 [pwwn 21:00:00:20:37:f7:da:31]

* fcid 0x3300cd [pwwn 50:00:00:e0:10:5c:24:e1]
* fcid 0x910001 [pwwn 10:00:00:00:c9:85:08:98]

* fcid 0x7000e2 [pwwn 22:00:00:04:cf:fe:4e:a0]
* fcid 0x280000 [pwwn 10:00:00:00:c9:85:01:f8]

* fcid 0xa80073 [pwwn 22:00:00:04:cf:67:47:fe]
* fcid 0x820001 [pwwn 10:00:00:00:c9:85:02:6c]

* fcid 0xd400e8 [pwwn 21:00:00:04:cf:fb:20:a9]
* fcid 0xdf0001 [pwwn 10:00:00:00:c9:85:02:71]

Step 9. Checking on the Server Pod sees the new SAN Disk. Use Remote Desktop to the server for
your POD and verify the host can see storage

Step 10. Open Disk Management.

Step 11. Find the LUN allocated for your server based on the size.

N5k N2k Ethernet FCoE Lab Guide Student v2 1

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

N5k N2k Ethernet FCoE Lab Guide Student v2 1

Uploaded by

Copyright:

Available Formats

Nexus 5000/Nexus 2000 Lab Guide

Data Center Ethernet, and Fibre Channel over Ethernet

You might also like