You are on page 1of 107

BRKCRS-2041

WAN Architectures and Design Principles


2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
2
Cisco Live & Networkers Virtual
Special Offer Save $100
Cisco Live has a well deserved reputation as one the industrys
best educational values. With hundreds of sessions spanning four
educational programs Networkers, Developer Networker,
Service Provider, IT Management, you can build a custom
curriculum that can make you a more valuable asset to your
workplace and advance your career goals.
Cisco Live and Networkers Virtual immerses you in all facets of
Cisco Live, from participating in live keynotes and Super Sessions
events to accessing session content to networking with your peers.
Visit www.ciscolivevirtual.com and register for Cisco Live and
Networkers Virtual. To get $100 USD off the Premier pass, which
provides access to hundreds of technical sessions, enter
slideshareFY11.
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
3
Agenda
WAN Technologies & Solutions
WAN Transport Technologies
WAN Overlay Technologies
WAN Optimization
Wide Area Network Quality of Service
WAN Architecture Design Considerations
Secure WAN Communication with GETVPN
Internet Backup Connectivity with DMVPN
WCCP Implementation Consideration
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
4
WAN Transport Technologies
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
5
Hierarchical Network Design
Core
Distribution
Access
Data Center
/HQ
Regional
hub
Spoke
Site 1
Spoke
Site N
...
Regional
hub
Spoke
Site 1
Spoke
Site N
...
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
6
MPLS VPN Topology
MPLS WAN is provided by a service provider
As seen by the enterprise network, every site
is one IP hop away
Equivalent to a full mesh, or to a hubless
hub-and-spoke
Spoke
Site 1
Spoke
Site 2
Spoke
Site N
Spoke
Site Y
Spoke
Site X
Spoke
Site 1
Spoke
Site N
Spoke
Site 2
Spoke
Site X
Hub Site
(The Network)
Spoke
Site Y
Equivalent to
SP-Provided
MPLS IP WAN
Definition
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
7
MPLS VPN
Direct Layer 2 Adjacencies Only
Between CE and PE Routers
Layer 3 (L3) Service
CE CE PE PE
local loop
VRF
VRF
Global
VRFVirtual Routing and Forwarding
! PE Router Multiple VRFs
ip vrf blue
rd 65100:10
route-target import 65100:10
route-target export 65100:10
ip vrf yellow
rd 65100:20
route-target import 65100:20
route-target export 65100:20
!
interface GigabitEthernet0/1.10
ip vrf forwarding blue
interface GigabitEthernet0/1.20
ip vrf forwarding yellow
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
8
MPLS VPN Design Trends
Single Carrier Designs:
Enterprise will home all sites into a single carrier to provide L3 MPLS VPN
connectivity.
Pro: Simpler design with consistent features
Con: Bound to single carrier for feature velocity
Con: Does not protect against MPLS cloud failure with Single Provider
Dual Carrier Designs:
Enterprise will single or dual home sites into one or both carriers to provide
L3 MPLS VPN connectivity.
Pro: Protects against MPLS service failure with Single Provider
Pro: Potential business leverage for better competitive pricing
Con: Increased design complexity due to Service Implementation Differences
(e.g. QoS, BGP AS Topology)
Con: Feature differences between providers could force customer to use
least common denominator features.
Variants of these designs and site connectivity:
Encryption Overlay (e.g. IPSec, DMVPN, GET VPN, etc.)
Sites with On-demand / Permanent backup links
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
9
Single Carrier Site Types (Non-Transit)
Dual Homed Non Transit
Only advertise local prefixes (^$)
Typically with Dual CE routers
BGP design:
EBGP to carrier
IBGP between CEs
Redistribute cloud learned routes
into site IGP
Single Homed Non Transit
Advertise local prefixes and
optionally use default route.
CE1
C1
CE2
AS 64512
C2
CE5
Site IGP
CE3
CE4
AS 64517
AS 200
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
10
Dual Carrier: Transit vs. Non Transit
C1
CE2
Prefix Z
AS 64512
C2
CE5
Prefix X
Prefix Y
Site
IGP
CE3
CE4
AS 64517
Transit
AS 100 AS 200
AS 64545
CE1
To guarantee single homed site
reachability to a dual homed site
experiencing a failure, transit sites
had to be elected.
Transit sites would act as a BGP
bridge transiting routes between
the two provider clouds.
To minimize latency costs of
transits, transits need to be
selected with geographic diversity
(e.g. from the East, West and
Central US.)
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
11
Single Provider Dual Providers
Pro: Common QoS support
model
Pro: More fault domains
Pro: Only one vendor to tune
Pro: More product offerings to
business
Pro: Reduced head end circuits
Pro: Ability to leverage vendors
for better pricing
Pro: Overall simpler design
Pro: Nice to have a second
vendor option
Con: Carrier failure could be
catastrophic
Con: Increased Bandwidth
Paying for bandwidth twice
Con: Do not have another carrier
in your pocket
Con: Increased overall design
complexity
Con: May be reduced to common
denominator between carriers
Resiliency Drivers vs. Simplicity
Single vs. Dual Carriers
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
12
WAN Overlay Technologies
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
13
Tunneling Technologies
IPSecEncapsulating Security Payload (ESP)
Strong encryption
IP Unicast only
Generic Routing Encapsulation (GRE)
IP Unicast, Multicast, Broadcast
Multiprotocol support
Layer 2 Tunneling ProtocolVersion 3 (L2TPv3)
Layer 2 payloads (Ethernet, Serial,)
Pseudowire capable
Packet Encapsulation over IP
Tunnels
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
14
GRE Tunneling
Original IP header IP payload GRE header New IP header
20 bytes 20 bytes 4 bytes
GRE packet with new IP header: protocol 47 (forwarded using new IP dst)
Original IP header IP payload
20 bytes
Original IP datagram (before forwarding)
! Router A GRE Example
interface Loopback 0
ip address 192.168.1.1 255.255.255.255
interface Tunnel0
ip address 172.16.1.1 255.255.255.0
encapsulation gre
ip mtu 1476
tunnel source Loopback0
tunnel dest 192.168.2.2
! Router B GRE Example
interface Loopback 0
ip address 192.168.2.2 255.255.255.255
interface Tunnel0
ip address 172.16.1.2 255.255.255.0
encapsulation gre
ip mtu 1476
tunnel source Loopback0
tunnel dest 192.168.1.1
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
15
IP HDR
Encrypted
ESP HDR
IP HDR
IP Payload
Tunnel mode
Transport mode
ESP
Trailer
ESP
Auth
Authenticated
Encrypted
Authenticated
IPSec ESP
IP Payload
IP Payload
IP HDR ESP HDR IP HDR
ESP
Trailer
ESP
Auth
Transport and Tunnel Modes
20 bytes
30 bytes
54 bytes
2 bytes
2 bytes
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
16
VPN Technology
EzVPN
Spoke
GET GM
DMVPN
Spoke
DMVPN
Spoke
Data Center
Internet
Edge
WAN
Edge
GET GM GET GM
Positioning EzVPN, DMVPN, GETVPN
MPLS/Private
Network
KS KS
GM GM
IPsec IPsec
Internet/
Shared
Network*
This Topic Is Covered in
Detail in BRKSEC-2011
* Note: DMVPN Can Also Be Used on MPLS/Private Network
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
17
VPN Technology Comparison
EzVPN DMVPN GET VPN
Infrastructure
Network
Public Internet
Transport
Private & Public
Internet Transport
Private IP
Transport
Network Style
Hub-Spoke;
(Client to Site)
Hub-Spoke and
Spoke-to-Spoke;
(Site-to-Site)
Any-to-Any;
(Site-to-Site)
Routing
Reverse-route
Injection
Dynamic routing
on tunnels
Dynamic routing
on IP WAN
Failover
Redundancy
Stateful Hub
Crypto Failover
Route
Distribution
Model
Route
Distribution
Model + Stateful
Encryption Style
Peer-to-Peer
Protection
Peer-to-Peer
Protection
Group Protection
IP Multicast
Multicast
replication at hub
Multicast
replication at hub
Multicast
replication in IP
WAN network
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
18
Dynamic Multipoint VPN
Provides full meshed
connectivity with simple
configuration of hub
and spoke
Supports dynamically
addressed spokes
Facilitates zero-touch
configuration for addition
of new spokes
Features automatic IPsec
triggering for building an
IPsec tunnel
Spoke n
Traditional Static Tunnels
DMVPN Tunnels
Static Known IP Addresses
Dynamic Unknown IP Addresses
Hub
VPN
Spoke 1
Spoke 2
Secure On-Demand Meshed Tunnels
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
19
Network Designs
Hub and spoke Spoke-to-spoke
Server Load Balancing
Hierarchical
Spoke-to-hub tunnels
Spoke-to-spoke path
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
20
Dynamic Multipoint VPN (DMVPN)
Operational Example
Spoke A
192.168.1.1/24
192.168.2.1/24
Physical: 172.17.0.1
Tunnel0: 10.0.0.1
Spoke B
Physical: (dynamic)
Tunnel0: 10.0.0.11
Physical: (dynamic)
Tunnel0: 10.0.0.12
10.0.0.11 172.16.1.1
10.0.0.12 172.16.2.1
192.168.0.1/24
192.168.1.0/24 10.0.0.11
192.168.2.0/24 10.0.0.12
192.168.0.0/24 Conn.
CEF FIB Table
172.16.1.1
172.16.2.1
NHRP mapping
192.168.1.0/24 Conn.
10.0.0.1 172.17.0.1
192.168.2.0/24 Conn.
10.0.0.1 172.17.0.1
192.168.2.1 ???
192.168.0.0/16 10.0.0.1
192.168.0.0/16 10.0.0.1
CEF Adjacency
10.0.0.1 172.17.0.1
10.0.0.11 172.16.1.1
Data packet
NHRP Redirect
NHRP Resolution
10.0.0.1 172.17.0.1
10.0.0.12 172.16.2.1
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
21
Dynamic Multipoint VPN (DMVPN)
Operational Example (cont)
Spoke A
192.168.1.1/24
192.168.2.1/24
Physical: 172.17.0.1
Tunnel0: 10.0.0.1
Spoke B
Physical: (dynamic)
Tunnel0: 10.0.0.11
Physical: (dynamic)
Tunnel0: 10.0.0.12
10.0.0.11 172.16.1.1
10.0.0.12 172.16.2.1
192.168.0.1/24
192.168.1.0/24 10.0.0.11
192.168.2.0/24 10.0.0.12
192.168.0.0/24 Conn.
CEF FIB Table
172.16.1.1
172.16.2.1
NHRP mapping
192.168.1.0/24 Conn.
10.0.0.1 172.17.0.1
192.168.2.0/24 Conn.
10.0.0.1 172.17.0.1
192.168.2.1 ???
192.168.0.0/16 10.0.0.1
192.168.0.0/16 10.0.0.1
CEF Adjacency
10.0.0.1 172.17.0.1
10.0.0.11 172.16.1.1
10.0.0.11 172.16.1.1
10.0.0.11 172.16.1.1
Data packet
NHRP Redirect
NHRP Resolution
10.0.0.1 172.17.0.1
10.0.0.12 172.16.2.1
10.0.0.11 172.16.1.1
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
22
Any-to-Any Encryption
Before and After GET VPN
Scalabilityan issue (N^2 problem)
Overlay routing
Any-to-any instant connectivity cant
be done to scale
Limited QoS
Inefficient Multicast replication
WAN WAN
Multicast
Before: IPSec P2P Tunnels After: Tunnel-Less VPN
Scalable architecture for any-to-any
connectivity and encryption
No overlaysnative routing
Any-to-any instant connectivity
Enhanced QoS
Efficient Multicast replication
Public/Private WAN
Private WAN
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
23
Group Security Functions
Group
Member
Group
Member
Group
Member
Group
Member
Key Server
Routing
Members
Group Member
Encryption Devices
Route Between Secure/
Unsecure Regions
Multicast Participation
Key Server
Validate Group Members
Manage Security Policy
Create Group Keys
Distribute Policy/Keys
Routing Member
Forwarding
Replication
Routing
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
24
Group Security Elements
Group
Member
Group
Member
Group
Member
Group
Member
Key Servers
Routing
Members
Key Encryption Key
(KEK)
Traffic Encryption
Key (TEK)
Group Policy
RFC3547:
Group Domain of
Interpretation
(GDOI)
KS Cooperative
Protocol
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
25
GETVPN: Innovation
Group Key Technology
Step 1: Group Members (GM)
register via GDOI (IKE) with the
Key Server (KS)
KS authenticates and authorizes the GM
KS returns a set of IPsec SAs
for the GM to use
Step 2: Data Plane Encryption
GM exchange encrypted traffic using the
group keys
The traffic uses IPSec Tunnel Mode with
address preservation
Step 3: Periodic Rekey of Keys
KS pushes out replacement IPsec
keys before current IPsec keys expire;
This is called a rekey
GM1
GM2
GM3
GM4
GM5
GM6
GM7 GM8
GM9
KS
GM1
GM2
GM3
GM4
GM5
GM6
GM7 GM8
GM9
KS
GM1
GM2
GM3
GM4
GM5
GM6
GM7 GM8
GM9
KS
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
26
WAN Optimization
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
27
The WAN Is the Barrier to Branch
Application Performance
Applications are
designed to work
well on LANs
High bandwidth
Low latency
Reliability
WANs have opposite
characteristics
Low bandwidth
High latency
Packet loss
Round Trip Time (RTT) ~ 0mS
Client
LAN
Switch
Server
Round Trip Time (RTT) ~ usually measured in milliseconds
Server Client LAN
Switch
LAN
Switch
Routed Network
WAN Packet Loss and Latency =
Slow Application Performance =
Keep and manage servers in branch offices ($$$)
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
28
TCP Behavior
Time (RTT) Slow start Congestion avoidance
Packet loss Packet loss Packet loss
cwnd
Packet loss
TCP
Return to maximum
throughput could take a
very long time!
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
29
WAN
WAASTCP Performance Improvement
Transport Flow Optimization (TFO) overcomes TCP and WAN
bottlenecks
Shields nodes connections from WAN conditions
Clients experience fast acknowledgement
Minimize perceived packet loss
Eliminate need to use inefficient congestion handling
LAN TCP
Behavior
LAN TCP
Behavior
Window Scaling
Large Initial Windows
Congestion Mgmt
Improved Retransmit
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
30
Comparing TCP and Transport
Flow Optimization
Time (RTT) Slow start Congestion avoidance
cwnd
TCP
TFO
Cisco TFO provides significant throughput
improvements over standard TCP implementations
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
31
WAAS Overview
DRE and LZ Manage Bandwidth Utilization
Data Redundancy Elimination (DRE) provides
advanced compression to eliminate redundancy
from network flows regardless of application
LZ compression provides generic compression
for all traffic
FILE.DOC
DRE CACHE DRE CACHE
FILE.DOC
WAN
LZ LZ
Origin Connection Origin Connection
Optimized
Connection
Encode Decode
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
32
End-to-End Security
WAN Optimization for Application Performance
Route Optimization for Application Performance
Performance
Issues/Brown Out
WAN with PfR
Best Performing Path
Best Metric Path
ISP1
ISP2
Without Cisco WAAS
Without QoS
WAN
Email
ERP
Scavenger
VoIP
Email
ERP
Scavenger
VoIP
Branch
HQ
Additional
Capacity
With Cisco WAAS
With QoS
Email
ERP
Scavenger
VoIP
Integrated Branch-WAN Services
Example: Delivering Voice over the Network
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
33
Wide Area Network Quality of Service
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
34
Quality of Service Operations
How Does It Work and Essential Elements
Classification and
Marking
Queuing and
Dropping
Post-Queuing
Operations
Classification and Marking:
The first element to a QoS policy is to classify/identify the traffic that is to be treated differently. Following
classification, marking tools can set an attribute of a frame or packet to a specific value.
Policing:
Determine whether packets are conforming to administratively-defined traffic rates and take action
accordingly. Such action could include marking, remarking or dropping a packet.
Scheduling (including Queuing and Dropping):
Scheduling tools determine how a frame/packet exits a device. Queuing algorithms are activated only when
a device is experiencing congestion and are deactivated when the congestion clears.
Link Specific Mechanisms (shaping, fragmentation, compression, Tx Ring)
Offers network administrators tools to optimize link utilization
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
35
Enabling QoS in the WAN
Traffic Profiles and Requirements
Latency 150 ms
Jitter 30 ms
Loss 1%
One-Way Requirements
Smooth
Benign
Drop sensitive
Delay sensitive
UDP priority
Voice
Bandwidth per Call
Depends on Codec,
Sampling-Rate,
and Layer 2 Media
Bursty
Greedy
Drop sensitive
Delay sensitive
UDP priority
Telepresence
Latency 150 ms
Jitter 50 ms
Loss 0.05%
One-Way Requirements
IP/VC has the Same
Requirements as
VoIP, but Has
Radically Different
Traffic Patterns
(BW Varies Greatly)
Smooth/bursty
Benign/greedy
Drop insensitive
Delay insensitive
TCP retransmits
Data
Data Classes:
Mission-Critical Apps
Transactional/Interactive Apps
Bulk Data Apps
Best Effort Apps (Default)
Traffic patterns for
Data Vary Among
Applications
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
36
20 msec
Voice Packets
Bytes
200
600
1000
Audio
Samples
1400
Time
200
600
1000
1400
33 msec
Video Packets
Video
Frame
Video
Frame
Video
Frame
QoS Considerations
Voice vs. VideoAt the Packet Level
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
37
Police
Scheduling Tools
LLQ/CBWFQ Subsystems
CBWFQ
Fragment
Interleave
FQ
Link Fragmentation
and Interleave
Low Latency Queueing
Packets
Out
Packets
In
VoIP
IP/VC
PQ
Layer 3 Queueing Subsystem Layer 2 Queueing Subsystem
Signaling
Critical
Bulk
Mgmt
Default
TX
Ring
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
38
Traffic Shaping
Policers typically drop traffic
Shapers typically delay excess traffic, smoothing bursts
and preventing unnecessary drops
Very common with Ethernet WAN, as well as Non-
Broadcast Multiple-Access (NBMA) network topologies
such as Frame-Relay and ATM
With Traffic Shaping
Without Traffic Shaping
Line
Rate
Shaped
Rate
Traffic Shaping Limits the Transmit Rate to a Value Lower Than Line Rate
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
39
MPLS VPN
Branch 1
Branch 2
Outbound Policies: Inbound Policies:
HQoS Shaper (if required)
+ LLQ for VoIP (EF) Trust DSCP
+ LLQ or CBWFQ for RT-Interactive (CS4)
+ Remark RTI (if necessary) + Restore RT-Interactive to CS4 (if necessary)
+ CBWFQ for Signaling (CS3)
+ Remark Signaling (if necessary) + Restore Signaling to CS3 (if necessary)
33%
of BW
Enterprise Subscriber (Unmanaged CE Routers)
Service Provider:
Outbound Policies: Inbound Policies:
+ LLQ for Real-Time Trust DSCP
+ CBWFQ for Critical Data Police on a per-Class Basis
CE Routers CE Routers PE Routers
Campus VPN
Block
E
E
E
E
F
F
F
F
F
E
MPLS VPN QoS Design
MPLS VPN Port QoS Roles
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
40
TX
Ring
policy-map ACCESS-EDGE
class VOIP
priority 1000
class REALTIME
priority 15000
class CALL-SIGNALING
bandwidth x
class TRANSACTIONAL
bandwidth y
class BULK-DATA
bandwidth z
class class-default
fair-queue
Packets
in
Packets
out
policy-map HQoS-50MBPS
class class-default
shape average 50000000 1000000
service-policy ACCESS-EDGE
CBWFQ
Scheduler
FQ
Call-Signaling CBWFQ
Transactional CBWFQ
Bulk Data CBWFQ
Default Queue
1 Mbps
VOIP
Policer
15 Mbps
REALTIME
Policer
16 Mbps PQ (FIFO Between VOIP and VIDEO)
Class-
Based
Shaper
GE Interface
with a sub-line-rate
access service
(e.g. 50 Mbps)
Queuing policies will not engage unless the interface is congested
A shaper will guarantee that traffic will not exceed the contracted rate
A nested queuing policy will force queuing to engage at the contracted
sub-line-rate to prioritize packets prior to shaping
Ethernet WAN QoS Design
HQoS Shaping & Queuing Policy and Operation
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
41
WAN Architecture Design
Considerations
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
42
High Availability Design
- Multiple/diverse WAN connections
- PfR for intelligent path routing of applications
Latency and Bandwidth Optimization
- Upgrade aggregation points to OC3/OC12
- Upgrade branches to DS3 or higher
- Plan capacity and traffic engineering
- Implement IP multicast and/or stream splitting
services (e.g. WAAS)
Real-Time Application Delivery
-implement robust QoS service policies to
manage application service levels
- Insuring wanted/limiting unwanted bandwidth
consumers (tools like PISA)
Service Level Assurance
- SLAs from SPs
- Operationalize SLA tools (e.g. Netflow, IP SLA)
Confidentiality
- Comply to security policies with data protection
strategies, such as IPSec, DMVPN, GETVPN
WAN Transport
Branch
Edge
MAN Edge
Site 1
WAN
Aggregation
Edge
SONET
/ SDH
DWDM
MAN Edge
Site 2
Metro
Ethernet
MAN Transport
FR/ATM
MPLS
Internet

SLA
Enterprise WAN Design Best Practices
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
43
Borderless Network Architecture
Two Thousand to Ten Thousand User Organization
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
44
Data
Center/
Campus
WAN
Services/
Distribution
High Performance WAN Headend
Over 100Mbps Aggregate bandwidth, Up to 500 Branchs
MPLS A
MPLS B
Campus/
Data Center
WAAS Service
Key
Servers
VPN Termination
Internet
WAN Edge
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
45
Internet
Internet
Internet Internet
Remote Branch
Transport & Redundancy Options
MPLS
MPLS WAN
MPLS +
Internet WAN
Internet
Internet WAN
MPLS MPLS
MPLS MPLS
MPLS
MPLS
Non-Redundant
Redundant-Links
Redundant-Links
& Routers
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
46
Routing Topology at Hub Location
MPLS A
Campus/
Data Center
DMVPN/
Internet
MPLS B
iBGP
EIGRP AS200
EIGRP AS 100
eBGP
Summaries +
Default
10.5.0.0/16
0.0.0.0/0.0.0.0
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
47
WAN Edge
All:
No static routes
No FHRPs
WA
N
Connection Methods Compared
WA
N
WAN
Edge
Router
WA
N
Core/Distribution
Si Si
Core/Distribution
Core/Distribution
Single Logical
Control Plane
Port-Channel for H/A
Recommended
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
48
Link redundancy achieved through
redundant L3 paths
Flow based load-balancing through
CEF forwarding across
Routing protocol reconvergence when
uplink failed
Convergence time may depends on
routing protocol used and the size of
routing entries
Optimize Convergence with EtherChannel
Si Si Si Si
P-to-P Link
Layer 3
Provide Link Redundancy and
reduce peering complexity
Tune L3/L4 load-balancing
hash to achieve maximum utilization
No L3 reconvergence required when
member link failed
No individual flow can go faster than
the speed of an individual member
of the link
VSS/3750
Stacks
IGP recalc
Channel
Member
Removed
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
49
interface Port-channel1
description Interface to MPLS-A-CE
no switchport
ip address 10.4.128.1 255.255.255.252
ip pim sparse-mode
ip summary-address eigrp 100 10.5.0.0 255.255.0.0
Best Practice
Summarize at Service Distribution
It is important to force summarization
at the distribution towards WAN Edge
and towards campus & data center
Summarization limit the number of
peers an EIGRP router must query
(minimize SIA) or the number of
LSAs an OSPF peer must process
MPLS B
MPLS A
Campus/
Data Center
Summaries +
Default
10.5.0.0/16
0.0.0.0/0.0.0.0
Summary
10.5.0.0/16
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
50
Run iBGP between the CE routers
Prefixes from carrier-A will be
advertised to carrier-B and vice versa
Allows the preservation of AS Path
length so remote sites can choose the
best path to destination
Use IGP (OSPF/EIGRP) for prefix re-
advertisement will result in equal-cost
paths at remote-site
Dual MPLS Carrier Hub
Use iBGP to Retain AS Path Information
MPLS B
Campus
iBGP
MPLS A
iBGP
10.5.128.0/21
bn-br200-3945-1# sh ip bgp 10.5.128.0/21
BGP routing table entry for 10.5.128.0/21, version 71
Paths: (2 available, best #2, table default, RIB-failure(17))
Not advertised to any peer
65401 65401 65402 65402, (aggregated by 65511 10.5.128.254)
10.4.142.26 from 10.4.142.26 (192.168.100.3)
Origin IGP, localpref 100, valid, external, atomic-
aggregate
65402 65402, (aggregated by 65511 10.5.128.254)
10.4.143.26 (metric 51456) from 10.5.0.10 (10.5.0.253)
Origin IGP, metric 0, localpref 100, valid, internal,
atomic-aggregate, best
E
I
G
R
P
E
I
G
R
P
10.5.128.0/21
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
51
Best Practice - Implement AS-Path
Filter
Dual carrier sites can unintentionally
become transit network during network
failure event and causing network
congestion due to transit traffic
Design the network so that transit path
between two carriers only occurs at
sites with enough bandwidth
Implement AS-Path filter to allow only
locally originated routes to be
advertised on the outbound updates for
branches that should not be transit
router bgp 65511
neighbor 10.4.142.26 route-map NO-TRANSIT-AS out
!
ip as-path access-list 10 permit ^$
!
route-map NO-TRANSIT-AS permit 10
match as-path 10
MPLS B
Campus
iBGP
MPLS A
Prevent Branch Site Becoming Transit Network
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
52
EIGRP Metric Calculation - Review
EIGRP Composite Metric
EIGRP Metric = 256*([K
1
*Bw + K
2
*Bw/(256-Load) + K
3
*Delay]*[K
5
/(Reliability + K
4
)])
Bandwidth [Bw] (minimum along path)
Delay (aggregate)
Load (1-255)
Reliability (1-255)
MTU (minimum along path)
For default bahavior (K1=K3=1), the formula metric is following:
metric = bandwidth + delay
EIGRP uses the following formula to scale the bandwidth & delay
bandwidth = (10000000/bandwidth(i)) * 256
delay = delay(i) *256
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
53
Best Practice Use Delay Parameter to
Influence EIGRP Path Selection
EIGRP uses the minimum bandwidth along the path
and the total delay to compute routing metrics
Does anything else use these values?
EIGRP also uses interface Bandwidth parameter to avoid congestion
by pacing routing updates (default is 50% of bandwidth)
Interface Bandwidth parameter is also used for QoS policy
calculation
PfR leverages Bandwidth parameter
Delay parameter should always be used to
influence EIGRP routing decision
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
54
eBGP routes are redistributed into EIGRP 100
as external routes with default Admin Distance
170
Running same EIGRP AS for both campus and
DMVPN network would result in Internet path
preferred over MPLS path
Multiple EIGRP AS processes can be used to
provide control of the routing
EIGRP 100 is used in campus location
EIGRP 200 over DMVPN tunnels
Routes from EIGRP 200 redistributed into EIGRP 100
appear as external route (distance = 170)
Routes from both WAN sources are equal-cost
paths. To prefer MPLS path over DMVPN use
eigrp delay to modify path preference
MPLS + Internet WAN
Use EIGRP Autonomous System for Path
Differentiation
MPLS A
Campus
EIGRP
AS100
EIGRP
AS200
Internet
D EX 10.5.48.0/21 [170/28416] via 10.4.128.2,
10.4.128.2
e
B
G
P
10.5.48.0/21
MPLS CE router#
router eigrp 100
default-metric 1000000 10 255 1 1500
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
55
Best Practice Assign Unique Router-ID
for Routing Protocols
For EIGRP & OSPF highest IP address assigned to a loopback is
selected as Router-ID. If there are no loopback interface configured,
the highest IP address from the other interfaces is selected
Router-ID can be used as tie breaker for path selection in BGP.
Prefer route that come from neighbor with lowest Router-ID
Duplicate EIGRP Router-ID will not prevent neighbor adjacency from
establishing, but can cause redistributed EIGRP external routes with
the same RID to be rejected from routing table
For OSPF and BGP duplicate Router-ID will prevent neighbors from
establishing adjacency
Certain OSPF LSA are tied to RID. When router receive network
LSA with LSA ID conflicts with IP address of interface on the router, it
will flush the LSA out of the network
Modification to Router-ID will result in adjacency reset
I am John!
I am John!
You must be Imposter
X
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
56
BGP Weight Metric Issue
Router prefer IGP over eBGP
MPLS B
MPLS A
e
B
G
P
eBGP
IGP
E
I
G
R
P
Dual MPLS VPN Network providing
primary and secondary network
connectivity between locations
eBGP peering with MPLS VPN
providers
Preferred path are learned via BGP to
remote location with backup path
learned via IGP
With default configuration the failover
works to the backup IGP path, but
reconvergence back to primary path is a
problem
10.4.160.0/24
Campus
10.4.160.0/24
R1
R2
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
57
Path Selection
Admin Dist [170] is better than [20] ?
MPLS B
MPLS A
e
B
G
P
eBGP
IGP
10.4.160.0/24
Campus
D EX 10.4.160.0/24 [170/3584]....
B 10.4.160.0/24 [20/0]....
R1# show ip route
B 10.4.144.0/24 [20/0] via 10.4.142.2, 01:30:06
B 10.4.145.0/24 [20/0] via 10.4.142.2, 01:30:06
D EX 10.4.160.0/24 [170/3584] via 10.4.128.9, 00:30:06
E
I
G
R
P
10.4.160.0/24
R1
R2
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
58
ASR1004-1#show ip bgp 10.4.160.0 255.255.255.0
BGP routing table entry for 10.4.160.0/24, version 22
Paths: (3 available, best #3, table default)
Advertised to update-groups:
4 5
65401 65401
10.4.142.2 from 10.4.142.2 (192.168.100.3)
Origin IGP, localpref 200, valid, external
Local
10.4.128.1 from 0.0.0.0 (10.4.142.1)
Origin incomplete, metric 26883072, localpref 100, weight 32768, valid, sourced, best
BGP Route Selection Criteria
BGP Prefers Path with:
1.Highest Weight
2.Highest Local PREF
3.Locally originated via network or aggregate BGP
4.Shortest AS_PATH
5.Lowest Origin type
IGP>EGP>INCOMPLETE
6.Lowest MED
7.eBGP over iBGP paths
8.Lowest IGP metric to BGP next hop
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
59
ASR1004-1#show ip bgp 10.4.160.0 255.255.255.0
BGP routing table entry for 10.4.160.0/24, version 22
Paths: (3 available, best #3, table default)
Advertised to update-groups:
4 5
65401 65401
10.4.142.2 from 10.4.142.2 (192.168.100.3)
Origin IGP, localpref 200, valid, external
Local
10.4.128.1 from 0.0.0.0 (10.4.142.1)
Origin incomplete, metric 26883072, localpref 100, weight 32768, valid, sourced, best
Prefer the eBGP Path over IGP
Set the eBGP weight > 32768
IGP (EIGRP) route is redistributed into BGP
Routes redistributed into BGP are considered locally
originated and get a default weight of 32768
The eBGP learned prefix has default weight of 0
BGP prefers the path with highest weight and the prefix
learned via eBGP is not selected
To resolve this issue set the weights on route learned
via eBGP peer higher than 32768
neighbor 10.4.142.2 weight 35000
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
60
Securing WAN communication
with GET VPN
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
61
GETVPN Topology
COOP Key Server
WAN Agg Switches Key Servers
MPLS B MPLS A
GM
GM
GM GM GM
GM
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
62
Best Practice - High Availability with
Cooperative Key Servers
Two or more KSs known as COOP KSs manage a common set of
keys and security policies for GETVPN group members
Group members can register to any one of the available KSs
Cooperative KSs periodically exchange and synchronize groups
database, policy and keys
Primary KS is responsible to generate and distribute group keys
GM 1
GM 3
Subnet 1
Subnet 4
Subnet 2
Subnet 3
GM 4
GM 2
Cooperative KS1
IP Network
Cooperative KS2
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
63
Transition from Clear-text to GETVPN
Receive-Only Method
Goal
Incrementally deploy
infrastructure without encryption
Immediate transition to
encryption controlled by KS
Method
Deploy KS with Receive-only
SAs (dont encrypt, allow
decryption)
Deploy GM throughout
infrastructure and monitor rekey
processes
Transition KS to Normal SA
(encrypt, decrypt)
Assessment
Pro: Simple transition to network-
wide encryption
Con: Correct policies imperative
Con: Deferred encryption until all
CE are capable of GM functions
permit ip 10.1.4.0 0.0.3.255 10.1.4.0 0.0.3.255
GM
GM
GM
GM
KS
10.1.4.0/24
10.1.6.0/24
10.1.5.0/24 10.1.7.0/24
GM
GM
GM
GM
GET
KS
10.1.4.0/24
10.1.6.0/24
10.1.5.0/24 10.1.7.0/24
permit ip 10.1.4.0 0.0.1.255 10.1.4.0 0.0.1.255
GET
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
64
crypto isakmp key c1sco123 address 10.4.128.151
crypto isakmp key c1sco123 address 10.4.128.152
crypto isakmp policy 10
encr 3des
authentication pre-share
group 2
!
crypto gdoi group GETVPN
identity number 65511
server address ipv4 10.4.128.151
server address ipv4 10.4.128.152
!
crypto map dgvpn 10 gdoi
set group dgvpn
!
interface FastEthernet0/0
crypto map GETVPN
Group Member Configuration
MPLS A
Key
Server
Group
Member
Group
Member
GDOI Group
Primary KS Address
Secondary KS
Address
GDOI configuration
mapped to crypto map
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
65
crypto keyring gdoi1
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
!
crypto isakmp policy 10
encr aes 256
authentication pre-share
group 2
!
crypto ipsec transform-set AES256/SHA esp-aes 256
esp-sha-hmac
!
crypto ipsec profile GETVPN-GDOI-PROFILE
set security-association lifetime seconds 7200
set transform-set AES256/SHA
!
IPSec Profile
IPSec Transform
Key Server Configuration
MPLS A
Key
Server
Group
Member
Group
Member
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
66
crypto gdoi group GETVPN
identity number 65511
server local
rekey lifetime seconds 86400
rekey retransmit 40 number 3
rekey authentication mypubkey rsa GETVPN-Key
rekey transport unicast
sa ipsec 10
profile GETVPN-GDOI-PROFILE
match address ipv4 GETVPN-MATCH-ACL
no replay
address ipv4 10.4.128.151
redundancy
local priority 100
peer address ipv4 10.4.128.152
!
GDOI Group ID
RSA Key to authenticate
rekeys
Unicast Rekey
Lifetime for Key
Encryption Key
Coop Server Config
KS Configuration (Cont.)
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
67
ip access-list extended GETVPN-MATCH-ACL
!Dont double encrypt traffic thats encrypted
deny esp any any
! Allow telemetry traffic
deny ip 10.4.0.0 0.1.255.255 10.4.142.0 0.0.1.255
deny ip 10.4.142.0 0.0.1.255 10.4.0.0 0.1.255.255
deny tcp any any eq tacacs
deny tcp any eq tacacs any
deny tcp any any eq 22
deny tcp any eq 22 any
!Allow BGP between CE-PE router
deny tcp any any eq bgp
deny tcp any eq bgp any
!Dont encryption ISAKMP traffic
deny udp any eq isakmp any eq isakmp
!Dont encrypt GDOI messages
deny udp any eq 848 any eq 848
!Allow CE-PE to form PIM adjacency
deny pim any 224.0.0.0 0.0.0.255
permit ip any any
Access-list denying
encryption for ISAKMP,
GDOI, BGP, TACACS, SSH
packets and permitting
encryption for all IP traffic
GET VPN Encryption Policy
Access-List configuration on KS
Allow communication from internal nets to the
PE-CE subnets (summarized):
10.4.0.0/16 to/from 10.4.142.0/24, 10.4.143.0/24
10.5.0.0/16 to/from 10.4.142.0/24, 10.4.143.0/24
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
68
DMVPN over Internet Deployment
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
69
Running EIGRP inside
the DVMPN using a
different AS number than
the campus EIGRP
Capable of dynamic
spoke-to-spoke tunnel to
other Internet attached
spokes
DMVPN over Internet Design Consideration
. . .
Internet
tun10
tun10
tun10
tun10
vpn-7206-1 vpn-7206-2
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
70
VPN Headend has a default
route to ASA firewalls VPN-DMZ
interface to reach Internet
Remote site policy requires
centralized Internet access
Enable EIGRP between VPN
headend & Campus core to
propagate default to remote
Static default (admin dist=0)
remains active,
VPN-DMZ is wrong firewall
interface for user traffic
Adjust admin distance so EIGRP
route installed (to core)
VPN tunnel drops
DMVPN Deployment over Internet
VPN-DMZ
Internet Edge
Block
default
default
INSIDE
OUTSIDE
E
I
G
R
P
default
Internet
default
default
Internet
Multiple Default Routes for VPN Headend
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
71
DMVPN Deployment over Internet
VPN-DMZ
Internet Edge
Block
default
default
INSIDE
OUTSIDE
E
I
G
R
P
d
e
f
a
u
l
t
Internet
default
E
IG
R
P
(2
0
0
)
default
default
Enable FVRF with DMVPN to
separate out the two default routes
The RED-VRF contains the default
route to VPN-DMZ Interfae needed
for Tunnel Establishment
A 2
nd
default route exist on the
Global Routing Table used by the
user data traffic to reach Internet
To prevent split tunneling the
default route is advertised to
spokes via Tunnel
Spokes tunnel drops due to 2
nd
default route conflict with the one
learned from ISP
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
72
Internet
Enable FVRF DMVPN on the
Spokes
Allow the ISP learned Default
Route in the RED-VRF and used
for tunnel establishment
Global VRF contains Default
Route learned via tunnel. User
data traffic follow Tunnel to
INSIDE interface on firewall
Allow for consistency for
implementing corporate security
policy for all users
Best Practice VRF-aware DMVPN
Keeping the Default Routes in Separate VRFs
VPN-DMZ
Internet Edge
Block
default
default
INSIDE
OUTSIDE
E
I
G
R
P
default
default
E
IG
R
P
(2
0
0
)
default
default
No Split Tunneling at Branch location
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
73
Internet
DMVPN and FVRF
Dual Default Routes Packet Flow
Based on incoming interface, the IPsec packet is directly
associated with VRF
After decryption the GRE packet is assigned to GRE tunnel
in the VRF
GRE decapsulated clear-text packets forwarded using Global
Routing table
Two routing tables one global (default) routing table and a
separate routing table for VRF
Clear-text packets forward
using Global Routing Table
Interface
I
P
s
e
c
GRE+IPsec
mGRE
Interface
Global
Routing Table
I
n
t
e
r
f
a
c
e
Default
Default VRF-RED
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
74
Internet
DMVPN and FVRF
Dual Default Routes Show IP Route Outputs
Clear-text packets forward
using Global Routing Table
Interface
I
P
s
e
c
GRE+IPsec
mGRE
Interface
Global
Routing Table
I
n
t
e
r
f
a
c
e
Default
Default VRF-RED
bn-vpn-7206-1#sh ip route
Gateway of last resort is 10.4.128.17 to network 0.0.0.0
D*EX 0.0.0.0/0 [170/3328] via 10.4.128.17, 2d22h, Port-channel3
....
bn-vpn-7206-1#sh ip route vrf RED
Gateway of last resort is 10.4.128.35 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 10.4.128.35
....
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
75
Internet
DMVPN and FVRF
Configuration Example
Clear-text packets forward
using Global Routing Table
Interface
I
P
s
e
c
GRE+IPsec
mGRE
Interface
Global
Routing Table
I
n
t
e
r
f
a
c
e
Default
Default VRF-RED
ip vrf INET-PUBLIC
rd 65512:1
!
crypto keyring DMVPN-KEYRING vrf INET-PUBLIC
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
!
!
crypto isakmp policy 10
encr aes 256
authentication pre-share
group 2
!
crypto isakmp keepalive 30 5
!
crypto isakmp profile FVRF-ISAKMP-INET-PUBLIC
keyring DMVPN-KEYRING
match identity address 0.0.0.0 INET-PUBLIC
!
interface GigabitEthernet0/1
ip vrf forwarding INET-PUBLIC
ip address dhcp
!
interface Tunnel10
ip address 10.4.132.201 255.255.254.0
.
tunnel mode gre multipoint
tunnel vrf INET-PUBLIC
tunnel protection ipsec profile DMVPN-PROFILE
!
router eigrp 200
network 10.4.132.0 0.0.0.255
network 10.4.163.0 0.0.0.127
eigrp router-id 10.4.132.201
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
76
Dead Peer Detection (DPD) is a mechanism for
detecting unreachable IKE peers
Each peers DPD state is independent of the
others
Without DPD spoke routers will continue to
encrypt traffic using old SPI which would be
dropped at the hub. May take up to 60 minutes
for spokes to reconverge
Use ISAKMP keepalives on spokes
crypto isakmp keepalives <initial>
<retry>
ISAKMP invalid-SPI-recovery is not useful with
DMVPN
ISAKMP keepalive timeout should be greater than
routing protocol hellos
Not recommended for Hub routers may cause
an increase of CPU overhead with large
number of peers
Best Practices
Enable Dead Peer Detection (DPD)
Internet
br201-2911 br202-2911
tun0
tun0
tun10
vpn-7206-1
Informational RFC 3706
T
r
a
f
f
i
c

D
r
o
p
p
e
d

U
n
t
i
l

n
e
w

I
K
E

s
e
s
s
i
o
n
s
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
77
Spokes are receiving dynamic address assignment
from the ISP
Spoke reboots and receive a new IP address from
the ISP, VPN session is established but no traffic
passes
Following error message appears on the spoke
Hub router (NHS) reject registration attempts
for the same private address that uses a
different NBMA address
To resolve this issue, configure following command
on spoke routers - ip nhrp registration no-unique
DMVPN Internet Deployment
Dynamic IP Address Assignment on the Spokes
Internet
br201-2911 br202-2911
tun0
tun0
tun10
vpn-7206-1
"%NHRP-3-PAKREPLY: Receive Registration Reply
packet with error - unique address registered
already(14)"
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
78
IP fragmentation will cause CPU and memory overhead and
resulting in lowering throughput performance
When one fragment of a datagram is dropped, the entire
original IP datagram will have to be resent
Use mode transport on transform-set
NHRP needs for NAT support and saves 20 bytes
Avoid MTU issues with the following best practices
ip mtu 1400
ip tcp adjust-mss 1360
crypto ipsec fragmentation after-encryption (global)
Best Practices
Avoid Fragmentation with IPSec VPN
MTU 1500
MTU 1500
MTU 1400
Tunnel Setting Minimum MTU Recommended MTU
GRE/IPSec (Tunnel Mode) 1440 bytes 1400 bytes
GRE/IPSec (Transport Mode) 1420 bytes 1400 bytes
GRE+IPsec
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
79
By default router uses OIL to
correlate multicast group join to
interface
This causes problem when hub is
connected to multiple spokes over
NBMA network
Any spoke that leaves a multicast
group would case all the spokes to
be pruned off the multicast group
Enable PIM NBMA mode under
tunnel interface on hubs and spokes
ip pim nbma-mode
Allows the router to track multicast joins
based on IP address instead of interface
Applies only to PIM sparse-mode
Router treats NBMA network as a
collection of point-to-point circuits,
allowing remote sites to be pruned
off traffic flows
Best Practices Multicast over DMVPN
Internet
br201-2911 br202-2911
tun10
vpn-7206-1
Multicast
Receiver
Receiver
M
u
l
t
i
c
a
s
t
M
u
l
t
i
c
a
s
t
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
80
By default router uses OIL to
correlate multicast group join to
interface
This causes problem when hub is
connected to multiple spokes over
NBMA network
Any spoke that leaves a multicast
group would case all the spokes to
be pruned off the multicast group
Enable PIM NBMA mode under
tunnel interface on hubs and spokes
ip pim nbma-mode
Allows the router to track multicast joins
based on IP address instead of interface
Applies only to PIM sparse-mode
Router treats NBMA network as a
collection of point-to-point circuits,
allowing remote sites to be pruned
off traffic flows
Best Practices Multicast over DMVPN
Internet
br201-2911 br202-2911
tun10
vpn-7206-1
Receiver
Receiver
IGMP
Leave
PIM
Prune
M
u
l
t
i
c
a
s
t
M
u
l
t
i
c
a
s
t
PIM
Prune
towards
RP
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
81
WCCP Implementation Consideration
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
82
Design Considerations for WAAS
Interception and Redirection Mechanisms
Implementation and operational consequences?
Planned Outages? Inline cabling changes are
disruptive, WCCP graceful start
Unplanned failures? Inline simple, fail to wire,
WCCP involves configuration changes to the
existing infrastructure
Placement decisions?
WAN Edge, WAN Distribution, Core,
Server Distribution, Server Access
Redirecting device used depends on
placement decision
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
83
Design Considerations for WAAS
Interception and Redirection Mechanisms
Scalability
Clusters with Load Balancing
Interception Methods
Large Number of Branch Offices to Fan Out and cache
High Availability
Through Clusters
Loss of single Device absorbed
Convergence Times depending on Integration Technique
Not stateful WAE loss causes session restart
A
B
A B
C
Src Balance 61 62 Dst Balance
e1 e2
r1
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
84
WAAS Integration Options
Inline Deployment
Policy-Based Routing (PBR)
Web-Cache Communication Protocol V2 (WCCPv2)
Hardware Load Balancers Inline with C/S Traffic Flow
PBR with HW Load Balancers
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
85
WCCP Characteristics
WCCP Reconvergence for failed WAE
Three failed Hello packets for failover i.e. 30-40 sec
Traffic partially not forwarded during failure
Supports asymmetric traffic across WCCP-enabled
routers
Supports up to 32 routers
and 32 WAEs in a cluster
Redirect-Lists allow granular
selection of traffic by use
of Extended ACLs
VRF-aware WCCP in IOS
12.4(20)T/15.0M and NX-OS
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
86
WCCP Redirect and Return
Redirect Method
WCCP GRE - Entire packet WCCP GRE tunneled to the cache(common
cache default)
Layer 2 - Frame MAC address rewritten to cache MAC
Return Method
WCCP GRE Packet WCCP GRE returned router (may be returned to
same router that performed redirect as in WAAS)
WCCP Layer 2 Frame rewritten to router MAC (Not yet supported in
WAAS)
Two assignment methods available
Hash
Byte level XOR computation divided into 256 buckets (default)
Available on software IOS routers only
Mask
Bit level AND divided up to 128 buckets (7 bits)
Available on all ASIC based L3 switches
Available on software routers as of IOS 12.4(20)T
Only method supported for ASR1000 as of IOS 12.2(33)XNF
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
87
Single Carrier Branch
WCCP intercepted in from client AND in from server
Services balance on source from client and destination from server
to maintain flow symmetry
E1 spoofs C1 to S1
S1 replies to C1
E1 spoofs S1 to C1
E1 must use WCCP GRE return to avoid loops when placed on
client network
C1
S1
E1
R1 SG 61 In
SG 62 In
WAN
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
88
Dual Router Branch
Transparent Client Transit Network Loop
R1 is HSRP/VRRP primary for clients
and WAE
Routing across client subnet
R1 upstream WAN failure
Packets route across client subnet
R2 intercepts packet a 2
nd
time and
redirects to cache
E1 receives packet for a 2
nd
time
(WAE drops packet)
Device WCCP GRE router
Intercept In only
Assign Mask or Hash
Redirect WCCP GRE
Return WCCP GRE
Egress WCCP negotiated
C1
S1
E1
R1
R2
62 61
61
62
WAN
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
89
Best Practice - Avoid Loop with Transit Subnet
Dual Router Branch
R1 is HSRP/VRRP primary for clients
and WAE
Routing across client subnet
R1 upstream WAN failure
Packets route across transit subnet
R2 forwards traffic without intercepting
packet a 2
nd
time
Device WCCP GRE router
Intercept In only
Assign Mask or Hash
Redirect WCCP GRE
Return WCCP GRE
Egress WCCP negotiated
Routers
Passive interface client subnet
Route on transit subnet
Use GRE return
C1
S1
E1
R1
R2
62 61
62
61
WAN
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
90
Summary
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
91
Key Takeaways
Understand how WAN characteristics can affect your
applications
Bandwidth, latency, loss
Dual carrier designs can provide resiliency but have
unique design considerations
A QoS-enabled, highly-available network infrastructure
is the foundation layer of the WAN architecture
Encryption is a foundation component of all WAN
designs and can be deployed transparently
Understand the how to apply WCCPv2 in the branch
network to enable WAN optimization appliances.
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
93
WAN Characteristics
Bandwidth
Bandwidth constraints keep applications from performing well
Too much data and too small of a pipe causes congestion, packet
loss, and backpressure
Packet loss, congestion, and retransmission
Packet loss and congestion cause retransmission which hinders
application performance and throughput
Commonly caused by saturated device transmit queues in the
network path
Packet Loss
Congestion
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
94
Latency
Network latencythe amount of time necessary for a
message to traverse the network
Transport latencythe amount of time necessary for the
transport mechanism (TCP) to acknowledge and
retransmit data
Application latencychattiness of an application
protocol causing messages to be exchanged across the
network
Round Trip Time (RTT) ~ usually measured in milliseconds
Typically:
High Bandwidth Low Latency
Low Bandwidth High Latency
Latency Impairs Application Performance in Three Ways:
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
95
G.729A: 25 ms
CODEC
Variable
(Can Be Reduced
Using LLQ)
Queuing
Variable
(Can Be Reduced
Using LFI)
Serialization
6.3 s/Km +
Network Delay
(Variable)
Propagation
and Network
2050 ms
Jitter Buffer
Enabling QoS
Elements That Affect End-to-End Delay
IP WAN
Campus
Branch Office
Cisco
Unified
Communication
Manager
Cluster
SRST
Router
PSTN
End-to-End Delay (Should Be < 150 ms)
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
96
KS System Scalability (7200)
1000 4000 2000 3000
1 KS - Multicast
Number of GM per Group
5000 500 250
P
r
e
-
s
h
a
r
e
d

K
e
y
s
P
u
b
l
i
c

K
e
y
1 KS - Unicast
1 KS - Unicast
2 KS - Unicast
1 KS - Multicast
2 KS - Multicast
8 KS - Unicast
7200 Assumptions
Single Key Server
TEK Lifetime = 3600
5% Registration Window
or 30 sec Window
PSK Reg. Rate = 100/sec
PKI Reg. Rate = 12/sec
To Be Tested in Phase 1.4 (based on 12.4(22)T
To Be Tested in Phase 1.4 (based on 12.4(22)T
To Be Tested in Phase 1.4 (based on 12.4(22)T
To Be Tested in Phase 1.4 (based on 12.4(22)T
To Be Tested in Phase 1.4
To Be Tested in Phase 1.4
8 KS - Multicast
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
97
KS System Scalability (Other Platforms)
100 400 200 300
unicast
Number of GM per Group
500 50 25
P
r
e
-
s
h
a
r
e
d

K
e
y
s
1841
Platform Assumptions
Single Key Server
Image < 12.4(22)T
30 sec Registration Window
multicast 2821
unicast multicast 2851
unicast multicast 3825
1000
unicast multicast 3845
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
98
Scheduling Tools
Queuing Algorithms
Congestion can occur at any point in the network where
there are speed mismatches
Routers use Cisco IOS-based software queuing
Low-Latency Queuing (LLQ) used for highest-priority traffic
(voice/video)
Class-Based Weighted-Fair Queuing (CBWFQ) used for
guaranteeing bandwidth to data applications
Cisco Catalyst switches use hardware queuing
Voice
Video
Data
3 3
2 2
1 1
This Topic Is Covered in Detail
in TECRST-2500
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
99
CBWFQ
Scheduler
FQ
Call-Signaling CBWFQ
Transactional CBWFQ
Bulk Data CBWFQ
Default Queue
TX
Ring
100 kbps
VOIP
Policer
WAN/Branch QoS Design
Dual-LLQ Design and Operation
The implicit LLQ policers allow for the configuration of
multiple LLQs, even though (under-the-hood) all LLQ
traffic is serviced by a single strict-priority queue
This PQ is serviced on a First-In-First-Out basis between
the VOIP and VIDEO classes, until the implicit LLQ
policing limits of each class have been reached
In this manner both VOIP and VIDEO receive an EF PHB,
but VIDEO cannot interfere with VOIP
400 kbps
VIDEO
Policer
100 kbps PQ
policy-map WAN-EDGE
class VOIP
priority 100
class VIDEO
priority 400
class CALL-SIGNALING
bandwidth x
class TRANSACTIONAL
bandwidth y
class BULK-DATA
bandwidth z
class class-default
fair-queue
500 kbps PQ (FIFO Between VOIP and VIDEO)
Packets
in
Packets
out
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
100
Enterprise-to-Service Provider WAN QoS
WAN QoS Matters!
Aggregation Site
Branch 1
Branch 2
Branch 3
Branch 4
1.544 Mbps
6 Mbps
100 Mbps
50 Mbps
7600-SIP-(200,400,600)
10 Mbps
3750 Metro ES Ports
7200
3750 Metro
ISR G2
ASR 1000
Si Si
ASR 1000
7600
Hierarchical QoS Required:
per vlan/subinterface shaping
CBWFQ/LLQ destination
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
101
Internet
DMVPN Topology
Hub Configuration
172.17.130.1
172.16.130.1
. . .
(.1),
10.4.163.0/24
(.1),
10.4.164.0/24
(.1),
10.4.132.0/24
bn-br201-2811 bn-br202-2811
ip vrf INET-PUBLIC
rd 65512:1
!
crypto keyring DMVPN-KEYRING vrf INET-PUBLIC
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
!
crypto isakmp policy 10
encr aes 256
authentication pre-share
group 2
crypto isakmp keepalive 30 5
crypto isakmp profile FVRF-ISAKMP-INET-PUBLIC
keyring DMVPN-KEYRING
match identity address 0.0.0.0 INET-PUBLIC
!
crypto ipsec transform-set AES256/SHA/TRANSPORT esp-aes 256 esp-
sha-hmac
mode transport
!
crypto ipsec profile DMVPN-PROFILE
set security-association lifetime seconds 7200
set transform-set AES256/SHA/TRANSPORT
set isakmp-profile FVRF-ISAKMP-INET-PUBLIC
!
interface GigabitEthernetx/x
ip vrf forwarding INET-PUBLIC
ip address 172.16.130.1 255.255.255.248
!
interface Tunnel10
ip address 10.4.132.1 255.255.255.0
no ip redirects
ip mtu 1400
ip hold-time eigrp 200 35
ip pim nbma-mode
ip pim sparse-mode
ip nhrp authentication cisco123
ip nhrp map multicast dynamic
ip nhrp network-id 101
ip nhrp holdtime 600
ip nhrp registration no-unique
ip nhrp redirect
no ip split-horizon eigrp 200
tunnel source GigabitEthernetx/x/
tunnel mode gre multipoint
tunnel vrf INET-PUBLIC
tunnel protection ipsec profile DMVPN-PROFILE
!
router eigrp 200
network 10.4.132.0 0.0.0.255
redistribute eigrp 100
eigrp router-id 10.4.132.1
10.4.132.201
10.4.132.202
vpn-7206-1
tun10
tun0
tun0
(.1),
10.4.133.0/24
tun10
dist-3750-stack
vpn-7206-1 vpn-7206-2
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
102
Internet
DMVPN Topology
Spoke Configuration
172.17.130.1
172.16.130.1
. . .
(.1),
10.4.163.0/24
(.1),
10.4.164.0/24
(.1),
10.4.132.0/24
br201-2911 br202-2911
ip vrf INET-PUBLIC
rd 65512:1
!
crypto keyring DMVPN-KEYRING vrf INET-PUBLIC
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
!
crypto isakmp policy 10
encr aes 256
authentication pre-share
group 2
crypto isakmp keepalive 30 5
crypto isakmp profile FVRF-ISAKMP-INET-PUBLIC
keyring DMVPN-KEYRING
match identity address 0.0.0.0 INET-PUBLIC
!
crypto ipsec transform-set AES256/SHA/TRANSPORT esp-aes 256 esp-
sha-hmac
mode transport
!
crypto ipsec profile DMVPN-PROFILE
set security-association lifetime seconds 7200
set transform-set AES256/SHA/TRANSPORT
set isakmp-profile FVRF-ISAKMP-INET-PUBLIC
!
interface GigabitEthernet0/1
ip vrf forwarding INET-PUBLIC
ip address dhcp
!
interface Tunnel10
ip address 10.4.132.201 255.255.255.0
no ip redirects
ip mtu 1400
ip pim nbma-mode
ip pim sparse-mode
ip nhrp authentication cisco123
ip nhrp map multicast 172.16.130.1
ip nhrp map 10.4.132.1 172.16.130.1
ip nhrp network-id 101
ip nhrp holdtime 600
ip nhrp nhs 10.4.132.1
ip nhrp shortcut
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel vrf INET-PUBLIC
tunnel protection ipsec profile DMVPN-PROFILE
!
router eigrp 200
network 10.4.132.0 0.0.0.255
network 10.4.163.0 0.0.0.127
eigrp router-id 10.4.132.201
10.4.132.201
10.4.132.202
br201-2911
tun10
tun0
tun0
(.1),
10.4.133.0/24
tun10
vpn-7206-2 vpn-7206-1
dist-3750-stack
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
103
Ethernet WAN QoS Design
Medium-Speed Ethernet WAN Access Configuration
<a 5 to 11 Class Model can be used>
!
policy-map WAN-EDGE-QUEUING
class VOICE
priority percent 33 ! LLQ Voice
class CALL-SIGNALING
bandwidth percent 5 ! BW guarantee for Call-Signaling
class CRITICAL-DATA
bandwidth percent 36 ! Critical Data class gets min 36% BW
random-detect dscp-based ! Enables DSCP-WRED for Critical-Data class
class SCAVENGER
bandwidth percent 1 ! Scavenger class is throttled
class class-default
bandwidth percent 25 ! Default class gets a 25% BW guarantee
random-detect ! Enables WRED for class-default
!
!
policy-map MQC-SHAPING-5MBPS
class class-default
shape average 4750000 47500 0 ! CIR=95% rate, Bc=CIR/100, Be=0
service-policy WAN-EDGE-QUEUING ! Queues packets before shaping
!
interface FastEthernet1/0
service-policy output MQC-SHAPING-5MBPS ! Attaches the hierarchical MQC policy
!
WAG
WAN Service with
1-99 Mbps Ethernet Access
BR
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
104
policy-map WAN-EDGE-QUEUING
class VOICE
priority percent 33 ! LLQ Voice
class CALL-SIGNALING
bandwidth percent 5 ! BW guarantee for Call-Signaling
class CRITICAL-DATA
bandwidth percent 36 ! Critical Data class gets min 36% BW
random-detect dscp-based ! Enables DSCP-WRED
class SCAVENGER
bandwidth percent 1 ! Scavenger class is throttled
class class-default
bandwidth percent 25 ! Default class gets a 25% BW guarantee
random-detect
policy-map shape-600M
class class-default
shape average 600000000 6000000 0 ! CIR = 600M
service-policy WAN-EDGE-QUEUING ! Hardware based platforms may
! auto-configure the Bc
interface GigabitEthernet0/0
bandwidth 600000 ! Interface Bandwidth = Shape Rate
service-policy output shape-600M ! All GE traffic shaped to 600Mbps!
GE 600 Mbps Subrate Service
High-Speed Ethernet WAN Access Configuration
CE
PE
Subrate GE
~600 Mbps
WAN
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
105
WAN Edge QoS Design Considerations
Link-Speed Considerations
Slow speed links ( 768 kbps)
Voice or video (not both)three
to five class model
LFI mechanism required
cRTP recommended
Medium speed links ( T1/E1)
Voice or video (not both)five
class model
cRTP optional
High speed links (> T1/E1)
Voice and/or videofive to 11
class model
Multiple links require L2 bundling
(best) or L3 load-balancing
Use newest CPUs for complex
QoS policies on DS3/OC3/OC12
links
WAN Agg
WAN Service with
768 kbps
BR
WAN Agg
WAN Service with
199 Mbps
BR
WAN Agg
WAN Service with
100 Mbps
BR
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
106
Phase 1
Hub and spoke
functionality 12.2(13)T
Simplified and smaller
config for hub & spoke
Support dynamically
address CPE
Support for multicast
traffic from hub to
spoke
Summarize routing at
hub
Spoke to spoke
functionality 12.3(4)T
Single mGRE interface
in spokes
Direct spoke to spoke
data traffic reduced
load on hub
Cannot summarize
spoke routes on hub
Route on spoke must
have IP next hop of
remote spoke
Phase 2 Phase 3
Architecture and
scaling 12.4(6)T
Increase number of
hub with same hub
and spoke ratio
No hub daisy-chain
Spokes dont need full
routing table
OSPF routing protocol
not limited to 2 hubs
Cannot mix phase 2
and phase 3 in same
DMVPN cloud
DMVPN Phases
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID
107
Branch WAAS/WAE Dual Router Branch
ip wccp 61 redirect-list WAAS-REDIRECT-
LIST group-list BN-WAE password c1sco123
ip wccp 62 redirect-list WAAS-REDIRECT-
LIST group-list BN-WAE password c1sco123
interface GigabitEthernet0/0
description WAN Interface
ip address 10.4.142.25 255.255.255.248
ip wccp 62 redirect in
!
interface Port-channel1.50
encapsulation dot1Q 50
ip address 10.5.0.1 255.255.255.252
ip wccp 61 redirect in
!
ip access-list standard BN-WAE
permit 10.5.1.8
permit 10.5.1.9
WCCP 61
WCCP 62
MPLS A
MPLS B
WCCP 62
WCCP 61
ip access-list extended WAAS-
REDIRECT-LIST
remark WAAS WCCP Mgmt
Redirect List
deny tcp any any eq telnet
deny tcp any any eq 22
deny tcp any any eq 161
deny tcp any any eq 162
deny tcp any any eq 123
deny tcp any any eq bgp
deny tcp any any eq tacacs
deny tcp any eq telnet any
deny tcp any eq 22 any
deny tcp any eq 161 any
deny tcp any eq 162 any
deny tcp any eq 123 any
deny tcp any eq bgp any
deny tcp any eq tacacs any
permit tcp any any

You might also like