The ACTION Project

Preliminary Results and Project Plan

Eiji Oki, Naoaki Yamanaka,
Andrea Fumagalli and Malathi Veeraraghavan

JUNO PI Meeting
June 25, 2014

PI cover area
Application

Design

MV

Experiment
Data-Center
+
Network

NY
Energy efficient

EO
Platform

Model
+
Optimized Method

Optical Transport

AF

Outline

Background
Objectives and work program organization
Short presentations from the individual PIs
Research plan for 36 months with milestones
Collaboration plan
Dissemination plan
3

Background
New enabling technologies
1.

Elastic Optical Networks (Flex Grid) [a]

WSS, Bandwidth Variable Transceivers (BVT), Energy Efficient Ethernet (EEE)

2.
3.

Sub-wavelength circuits (OTN, DSON)

Dynamic circuit/virtual circuit (VC) technologies

4.

MPLS, VLAN, Ctrl. plane solutions (RSVP-TE, PCEP)

Software Defined Network (SDN)/OpenFlow

Measurements show that

Internet links are underutilized [b]

Networks are not operated in an energy-efficient manner [c]

[a] Gerstel, O.; Jinno, M.; Lord, A.; Yoo, S.J.B., "Elastic optical networking: a new dawn for the optical layer?," IEEE
Comm. Mag., February 2012
[b] Sushant Jain, et al., B4: experience with a globally-deployed software defined WAN, SIGCOMM '13.
[c] Dennis Abts, et al. Energy proportional datacenter networks. SIGARCH Comput. Archit. News June 2010
4

Good reasons for operating todays Internet links at low utilization (25-35%)
Challenges of high-utilization operation of a network
1.
2.

Failure handling: Additional network load placed on links

Long-term growth: A provider upgrading the network in say 2012 designs
network to handle loads upto say 2017
http://es.net/overview-of-the-network/network-maps/historical-network-maps/

3.

High-speed file transfers: dedicated data-transfer nodes are designed to

move data at high speeds relative to link capacity
e.g., ESnet 100G testbed has hosts that can each push data at 40 Gbps
with striped transfers across three hosts, can fill 100G links

4.
5.

Avoid packet losses: TCP throughput sensitive to losses

Allow sufficient headroom for poor planning (imperfect traffic forecasts) or
poor routing (imbalanced load)
5

Objectives
Develop an Applications Coordinating with Transport, IP,
and Optical Networks (ACTION) architecture
by integrating four enabling technologies
by operating links at higher utilization while meeting the five
challenges of high-utilization operation

Why operate at high utilization: the network will need

fewer powered-on links, and hence the network will
consume smaller levels of energy (power) to move the same
number of information bits (bits/sec)
analogy: flying a half-empty large airplane!
6

Track 0: ACTION PCE

algorithms and design

ACTION Project Overview

Track 3: Introduce
4 new technologies
into datacenter networks

ACTION
Management System

Datacenters

ACTION SDN controller

ACTION SDN
controller

Hosts

IP/Eth/VLAN

Access
link
Campus
networks

su

ADC

O/E

s1 s2
w

su

O/E

ADC

FlexGrid/OTN/DSON

Track 4: Introduce 4 new

technologies into
campus networks

Receiver

Tracks 1 and 2:
Introduce 4 new
technologies into core
(also metro) networks

Campus
networks

c12

c1u

Receiver

OF

s1 s2

c11

cw1
cw2

cwu

DSP

s1 s2

FlexWDM
DSON

ADC

su
1

E/O

Transmitter

s1 s2
DSP

IP/MPLS/VLAN

Metro/access
provider networks

Cross-bar circuit switch

OF

DSP

Parallel links

DSP

Metro/access
provider
networks

Core provider networks

ADC

su

E/O

Transmitter

Long-term Vision (with Current Focus)

Broaden scope of dynamic circuit/VC services
Trigger to setup/release
circuit comes from
user applications

Circuit/VC endpoints
IP routers

Intradomain

Trigger to setup/release
circuit comes from NMS/admins

Computers (endpoints may be

closeby switches/routers)
Campuses

Inter-domain

Track 1: semi-static
Track 2: dynamic

Datacenters

Intra-domain

Inter-domain
Track 4:dynamic

Intra-datacenter

Track 3: dynamic

Inter-datacenter
(intra- or inter-domain)

distributed
hadoop job scheduling8

Work Program Organization

Track 0: ACTION SDN controller
Controls both OpenFlow Ethernet switches/IP routers PLUS optical crossconnects

Four application tracks (applications for dynamic circuit services)

Routers are circuit endpoints (aggregate traffic: ACTION Management System)
Track 1: Virtual Topology Management: Leverage long-timescale variations (such as night/day
traffic patterns) to power off or reduce link rates for energy savings while planning for failures
Track 2: Link Self-Sizing : Analyze short-timescale variations by observing IP-network link-level
traffic (via SNMP MIB reads) and then ask ACTION SDN controller to adjust rate of elastic
optical paths (used to realize IP-layer links) whenever possible for energy savings

Computers are circuit/VC endpoints (individual flows: application triggers from

endpoints)
Track 3: Hybrid Data-Center Networks: EON + applications (Hadoop scheduling)
Track 4: Campus Networks: Router access links adjustment
9

Track 0: ACTION SDN Controller

New path computation algorithms

Take into account Quality of Transmission (QOT) metrics

Consider energy consumption
Account for failures
Handle traffic fluctuations
Intra-domain path selection
Inter-domain path selection (East-West API)
Multi-layer path selection

e.g., by coordinating with per-layer SDN controllers

Architecture, design and prototyping

Reduction of circuit setup delay

Domain definition: a network owned and operated by one organization

10

ACTION SDN
Controller

Quality Confidential
of Transmission (QoT)

OSNR and BER

Traffic Independent PLIs
Traffic Dependent PLIs

Circuit setup delay factors

Control plane
Optical devices (OA, WSS)
Tx/Rx re-synchronization

(Structure and function of ACTION SDN)

Core, metro, campus,
1GE increments

Quality
of
Transmission
(QoT)
datacenter networks
Circuit setup performance
Power consumption of NW
ACTION SDN
controller

Router or switch

Parallel links

OXC
FlexWDM
DSON

WSS

OA

Power consumption factors

Router I/O BVT
FFT for DSC multiplexing
Optical amplifiers

11

Track 1 (lead: Eiji)

Issued to be addressed
Difficult to estimate exact traffic demand each edge nodes
Avoid frequent dynamic route changes according to traffic fluctuation

Objective
Virtual Topology Management: Leverage long-timescale variations (such as
night/day traffic patterns) to power off or reduce link rates for energy savings while
planning for failures
Account for failures
Handle traffic fluctuations
Utilize flexibility of elastic optical networks

Application
Robust and stable energy-efficient network
Changeable traffic demand of IP and layer 2 networks
12

Virtual topology management for

Confidential
aggregate traffic
Determine robust and stable
IP/Layer 2 routing
ACTION
PCE
Bandwidth allocation
Link reinforcement
(protection, bandwidth)
in cooperated with
elastic optical networks

(Aggregated trafficMonitor
model)
aggregated traffic, but not exact traffic
allowing traffic fluctuation
Path ComputationAction
Engine matrix,
- Daily traffic (day, night, )
Management System
- Monitoring point
topology
ACTION SDN (virtual
- Incoming/outgoing traffic volume at edge node
management)
- Traffic volume passing through each link
ACTION SDN
Monitor
traffic
controller
ACTION Management system

Incoming/outgoing
aggregated traffic
Aggregated
link traffic

13

Work plan (Track 1)

Virtual topology optimization
framework developed with

Energy
Efficiency
High

Suitable traffic demand model

Failure model
Multi-layer orchestration

Simulator developed
Simulate energy saving effect in cooperation
with elastic optical network simulator
Provide simulated for integrated
demonstration

Pipe
model

Suitable
traffic
model

Hose
model
Low

Narrow

Wide

Range of
traffic matrix

IP networks
API

Optical networks
14

Track 2 (lead: Naoaki)

Aggregate flows (aggregate link traffic: all flows): Next, consider aggregate traffic. In
todays network (commercial and REN), utilization is around 30-35%. Say we decide to
operate at 70% utilization by lowering the link rates (using Bandwidth Variable
Transceivers). We start using a network management system (NMS) read SNMP MIBs
at IP routers of all links and monitor these levels. We can set some thresholds to
adjust link rates, e.g. (65, 75)%. In 30 sec or min.

Objective:
Bring 2 new technologies to improve IP network performance using elastic optical technique

Applications:
Improve link efficiency by leveraging SNMP MIB data
Change virtual link bandwidth to maintain a predefined link utilization, e.g. 70%

Study issue 1: Speed of SNMP is enough? Or new NFV is needed?

Study issue 2: BVT and elastic network adjustment speed: P2P link vs. network path
15

Track 2 : Self-sized
link
by
SNMP
data
2
Confidential

ACTION
PCE

ACTION SDN
controller

Action
Management
System
(link self-sizing)

Simple (starting pt) heuristic

1. read past 30 sec byte count
2. forecast that next 30sec byte count is going to be same
3. create a SDN-northbound request (endpoints: router1 and router2, rate)
to send to SDN controller asking for BW modification of link; wait for response

1Gb/s Fixedtechnique)
(Link
Self-sized
Router 1
Router 2
SNMP MIB
Path Computation
Engine
Self-sized Link
ACTION4 Manager
Today : link= fixed bandwidth fiber
Bottle neck link
ACTION: self-sized
virtual link
Low utilization
ACTION SDN Controller
Elastic Optical Network
100%

4 OpenFlow message
to increase/decrease
spectrum allocation

Elastic Optical

16

Method and Advantages

Confidential
(Energy saving
and higher throughput)
Step1 Traffic Monitor (SNMP-Mib)
Step2 Forecast 30sec

Step3
SDN North band IF
Step4 OpenFlow control
G

Scalability
Elastic opticalPower
Bandwidth flexibility

Advantages
Higher Throughput or high-link utilization

Self-sized link Self-sized MiDORi Network

(Method and Advantages)

Monitor-Control-Openflow switch
Power consumption results
IP throughput

17

Work Plan (Track 2)

Try to obtain real SNMP traces
Measure Energy Efficient Ethernet (EEE) power consumption and response
time for making rate changes (also make projections for future BVTs)
Simulate Internet2 or ESnet or SINET using ns2 or P2A simulator to
determine the optimal timescales and characterize energy savings
tradeoffs (with and without elastic optical path modifications).
Network structure by multiple parallel links or VON( Virtual optical
network)
Prototyping in Keio Univ. NW with test with VLAN switches on testbeds
emulation and JGN using deeply programmable node
Test assumptions
Collect measurements

Extended ideas to virtual optical network slice and/or inter datacenter

18

Track 3 (lead: MV)

Objective:
Bring 4 new technologies to datacenter networks

Applications:
Hadoop scheduler extended to co-schedule CPU
and network resources (trigger dynamic circuits)
Filesystem writes of large files (host-application
triggered dynamic circuits)
19

ACTION Hadoop scheduler

(integrated CPU + network)

ACTION
Track 3
Confidential
SDN controller/OSCARS

FlexiGrid/
DSON
optical
switch

Dynamic circuit
triggered by
scheduler
or host apps

(Data center model)

ACTION Hadoop Scheduler
SDN Controller
Optical Dynamic Circuit by apps.
Pod 0

Pod 1

Power-on second NIC and Ethernet

switch port only when required

Pod 2

Core

Aggregation

Edge

Pod 4

Dyn. circuit oriented

apps on servers

Hybrid OpenFlow packet/optical circuit networks

20
OSCARS: On-Demand Secure Circuits and Advance Reservation System

Work Plan (Track 3)

Prototype applications and test with VLAN switches on
testbeds (e.g., ExoGENI, PRObE, Keio test-bed, CPqD
test-bed)
Test assumptions
Collect measurements

If possible, obtain real traces from datacenters

Run simulations or create analytical models of largescale datacenter network to estimate energy savings
21

Track 4 (lead: Andrea)

Objective:
Bring 4 new technologies to campus networks

Applications:
Network administrator requests temporary
augmentation of access link capacity for special events
on campus (e.g., football game)
Host-application triggers dynamic circuits (e.g., moving
human genome sequence rough or processed data)
22

Current Campus

ACTION Track 4
Metro/access
Confidential
provider network

ACTION SDN
controller

Metro/access
provider
network

Save energy
Improve utilization
Genome app.

Port(s)inter-campus)
that
(Campus network and
is (are) shared
dynamically
Metro/accessFootball
elastic
network
game
Inter-Campus dynamic network

ACTION SDN
controller

FlexWDM
DSON

Fixed Ethernet rate,

e.g. 1GE, 10GE

Access
link

ACTION SDN
controller

Hosts

Campus networks

IP/Eth/VLAN

Hosts

Access
links

Access
link

Port that is
dynamically
powered on
and off

Hosts

Campus networks

ACTION SDN
controller

23

Work Plan (Track 4)

Prototype campus applications and test with VLAN switches on testbeds (e.g., Keio test-bed)
Test assumptions
Collect measurements

Define and implement simulation modules based on experimental

data (e.g., CPqD test-bed)
PLI models for EON and DSON
Circuit setup delay models for optical components, physical control
plane and signaling
Energy consumption models

Obtain real traffic traces from campuses

Run simulations and/or create analytical models of campus network
to estimate energy savings
24

Outline
Background
Objectives and work program organization
Short presentations from the individual PIs
Research plan for 36 months with milestones
Collaboration plan
Dissemination plan
25

Track 0

Track 0 - QoT

ACTION Architecture&Design (NY,

0.5 EO, MV, AF)

Non-coherent TI-PLIs

0.5

Estimate of power
consumption for key already
existing devices

Intra-domain PCE (EO,AF)

Track 2

Track3

Analyze Internet2 SNMP MIB

traces: heuristic for rateadjustment duration (MV,NY)

Coherent TI-PLIs and TD-PLIs

and WSS equalization

Modeling and formulation for

IP+optical networks (EO,AF)

Obtain energy and response time

measurements for EEE and switch
port (NY)

Designing & solving PCE for

IP/layer 2 and optical PCE
(EO,AF,NY)

Modeling and formulation (EO)

ACTION Hadoop scheduler +

characterize comm. traffic (MV,NY)

paper #1

Prototype Link Self Sizing

Emulation Module of ACTION
Mgmt System(NY,MV)

Energy and response time data for

Ethernet and optical switch
configuration (NY)

Circuit setup latency models

for control plane and key
optical devises

Track 4

Modeling and formulation for

IP/layer 2 switch networks
(EO,MV)

OA: EDFA non-flat gain and

Test hypothesis of day-night
noise level from experimental traffic patterns being different
work
(MV)

1.5 Cross-layer PCE (EO,AF)

PCE prototype for IP/layer 2
networks w/ EON interfaces
1.5 (EO,AF,NY)

Track 1

Prototype Integrated system demo: Analyze campus access NetFlow

Prototype system demo (NY and
apps+scheduler+SDN controller
records for alpha flows and SNMP
MV); Performance evaluation (EO) (MV and NY)
data (MV)

Prototype SDN Controller (IP and

layer 2) (NV,NY)

DSCM Architecture and spectrum

allocation algorithms (AF,NY)

Applications redesigned to make

full use of circuits oriented
services (MV)

Inter-domain PCE RSA strategies

2.5 (MV,NY,EO, AF)

Contain circuit setup delays and

their impact on applications
(AF,MV)

Definition of APIs required

between application and SDN
controller (MV, NY, AF)

SDN controller East-West

2.5 interfaces

paper #3

Define SDN controller East-West

SDN interfaces (AF, NY, MV)

DSCM transmission and power

consumption models

Campus application-SDN
controller APIs

paper #2

Prototype system demo (NY and

MV)

paper #4

26

Collaboration Plan

Have been meeting via WebEx (almost) every week since the start of the project
IP Agreement is finalized
Face-to-face meetings done: NOC14, ICC14 and PI meeting (June 2014)
Plan by coordinating conference attendance
Also, hold dedicated meetings

Dropbox folder: already under heavy use!

Joint platforms for simulations, software/shell scripts for running experiments,
software for data analysis
Support graduate student/PI visits to other labs for more immersive learning
Joint workshop

27

Dissemination Plan
http://venividiwiki.ee.virginia.edu/mediawiki/index.php/ACTION
Papers: Journals and conferences
COIN paper

Software: post on wiki site

Data: store measurements from testbed, SNMP from Internet2, SINET in
university library data stores
Organize workshop or special session
IEEE/IEICE HPSR 2016 (TBD)

28