RuggedCom

Industrial Strength Networks

Ethernet in Substation Automation Applications

Issues and Requirements

Marzio Pozzuoli
RuggedCom Inc. - Industrial Strength Networks
Concord, Ontario, Canada

Presented at the:
Western Power Delivery Automation Conference (WPDAC) April 2nd 2003
Sponsored by Washington State University
 RuggedCom
Industrial Strength Networks
Key Topics Covered

The Emergence of Ethernet in the Substation Ethernet everywhere!

UCA 2.0 (Real-Time Architecture)
IEC 61850-9-x (Process Bus)
Ethernet ready IEDs
The LAN based Substation

The Substation Environment No place for the faint of heart!

Key issues and requirements for harsh industrial / substation environments
EPRI and Rockwell studies confirm what most engineers already knew.
IEC 61000-6-2, IEC 61850-3, IEEE P1613 Standards to the rescue

Real-time, Deterministic Performance Look Ma, No Collisions!

Technological improvements in modern Ethernet technology - facts and myths.
IEEE 802.3x, 802.1p, 802.1Q, 802.1w,
Layer 3 - IGMP Snooping / Multicast Filtering

A Comparison of Network Architectures Different Strokes..

Basic Bus and Star architectures.
IEEE 802.1w Rapid Spanning Tree allows fault tolerant loops!
Fault tolerant network architectures.
RuggedCom
Industrial Strength Networks

Ethernet in the
Substation!
Even electric utilities do it
 RuggedCom
Industrial Strength Networks
Ethernet in the SubstationWhy?
!Back in the 90s North American Utilities were seeking a common
communications architecture for substation and utility automation.
Every IED vendor supported their own, often proprietary, protocols.
Protocol converters were required in order to integrate multi-vendor IEDs.
Costs were high, performance was low!

!Utilities and vendors under the auspices of EPRI developed the Utility
Communications Architecture (UCA2.0 / IEC 61850)
A collection of standards to allow for a Utility communications architecture
Supporting: multi-vendor IED interoperability, real-time control over a
substation LAN and a seamless flow of information across the entire Utility
enterprise.

!Ethernet chosen as the underlying technology for UCA / IEC 61850

Ethernet is the most prevalent local area network technology in the world.
Large installed base: over 95% of all back office or enterprise layers.
Non-proprietary technology with multiple vendors.
No single IED vendor would have an advantage over the other.

4
 RuggedCom
Industrial Strength Networks
Traditional Substation

Inter
Inter IED
IED control
control signaling
signaling done
done via
via wiring
wiring or
or low
low speed
speed serial
serial communications.
communications.
Limited
Limited to
to simple
simple schemes
schemes due
due to
to cost
cost and
and complexity
complexity of
of wiring
wiring
and
and limited
limited performance
performance capabilities
capabilities of
of communications.
communications. 5
 RuggedCom
Industrial Strength Networks
Connecting IEDs via the LAN

UCA / IEC 61850 Substation LAN

Substation LAN
10/100Mbps Ethernet

Ethernet Switch

Digital Relay Metering IED RTU PLC LTC

CT/VT and Breaker Wiring

CT/VTs BREAKERs

ALL
ALL IEDs
IEDs are
are connected
connected via
via aa single
single connection
connection (or
(or dual
dual redundancy)
redundancy) to
to the
the
LAN.
LAN.

Simple
Simple or
or complex
complex control
control schemes
schemes are
are possible
possible with no increased wiring
costs
costs or
or complexity.
complexity.
Other
Other real-time
real-time data
data (e.g.
(e.g. analog,
analog, status)
status) can
can be
be shared
shared across
across ALL
ALL IEDs.
IEDs.
6
 RuggedCom
Industrial Strength Networks
The LAN Based Substation

UCA / IEC 61850

Substation LAN PC
10/100Mbps Ethernet
E th ern et S w itch

Digital Relay Metering IED RTU PLC LTC

IEC 61850-9-2
E th ern et S w itch
Process Bus LAN
100Mbps Ethernet

Digital Modern
CT/VTs Switchgear

Sampled
Sampled real-time
real-time data
data can
can be
be broadcast
broadcast to
to ALL
ALL IEDs
IEDs simultaneously.
simultaneously.
Similar
Similar to
to aa Producer-Consumer
Producer-Consumer or or Publisher-Subscriber
Publisher-Subscriber model
model used
used in
in
many
many Industrial
Industrial Automation
Automation protocols:
protocols: e.g.
e.g. EtherNet/IP,
EtherNet/IP, Fieldbus
Fieldbus HSE
HSE7
 RuggedCom
Industrial Strength Networks
Different strokes for different folks

The Utility Automation Market The Industrial Automation Market

IEC61850(UCA2) ProfiNet Modbus/TCP

DNP3/TCP Modbus/TCP Application Fieldbus HSE EtherNet/IP
Presentation

Session

TCP/UDP TCP/IP Transport TCP/UDP TCP/IP

IP Network IP

IEEE 802.1 Data Link IEEE 802.1

Ethernet Ethernet
IEEE 802.3 Physical IEEE 802.3

Bottom half is pretty much consistent. Bottom half is pretty much consistent.

Application Layer is dominated by a Application Layer is still being hotly

few players. contested by major OEMs.
8
RuggedCom
Industrial Strength Networks
Convergence in the utility automation sector

C
EPRI omprehe
-proj n
ect U sive
CA. 2

The international agreed goal IEC 61850

03 , -104
0 1
1, - rience
0- 5- 1 pe
60 8 7 ex
IEC

IEC 61850 represents convergence (i.e. One protocol) for the

utility industry and the Holy Grail of multi-vendor device
interoperability! 9
 RuggedCom
Industrial Strength Networks
Cant we all just get along
Q. How do you get multiple devices which communicate using
different protocols to coexist on a common network?
Ans. You use Ethernet!

S W I T C H

RTU
RTU Relay
Relay Meter
Meter PLC
PLC
(DNP3)
(DNP3) (UCA2/IEC61850)
(UCA2/IEC61850) (Modbus)
(Modbus) (Fieldbus)
(Fieldbus)

Ethernet allows multiple protocols to coexist on the same network!

10
RuggedCom
Industrial Strength Networks

The Substation Environment

No place for the faint of heart!
 RuggedCom
Industrial Strength Networks
The Substation Environment
Power Station MV/HV Substation

Electric Fields Phenomena

Phenomena Encountered
Encountered
Magnetic Fields
Electrostatic Discharge
Conducted High Frequency Electrical Transients
High Energy Power Surges
Ground Potential Rise during ground faults
Climactic Variation: Temperature & Humidity
Seismic / Vibration
Pollution: Dust, Metallic Particles, Condensation, Solar Radiation
12
 RuggedCom
Industrial Strength Networks EMI Phenomena - Frequency of Occurrence

Continuous
Continuous Transient
Transient Phenomena
Phenomena Transient
Transient Phenomena
Phenomena
Phenomena
Phenomena (High
(High Occurrence)
Occurrence) (Low
(Low Occurrence)
Occurrence)
Radiated
Radiated RFI
RFI Electrostatic
Electrostatic Discharge
Discharge Power
Power Frequency
Frequency Variation
Variation
Induced
Induced RFI
RFI Voltage
Voltage Dips
Dips Power
Power System
System Faults
Faults
Power
Power freq.
freq. Magnetic
Magnetic Field
Field Lightning
Lightning Short
Short Duration
Duration Power
Power Freq.
Freq.
Slow
Slow Voltage
Voltage Variations
Variations HV
HV Switching
Switching by
by Isolators
Isolators Magnetic
Magnetic Fields
Fields
Harmonics,
Harmonics, Interharmonics
Interharmonics Reactive
Reactive Load
Load Switching
Switching
Ripple
Ripple on
on d.c.
d.c. power
power supply
supply
Power
Power Frequency
Frequency Voltage
Voltage

Devices in substation environments must deal

with a combination of EMI phenomena which are
continuous and transient.

13
RuggedCom
Industrial Strength Networks Devices are Hardened for the Substation
Substation LAN

E th e rn e t S w itch

Digital Relay Metering IED RTU PLC LTC

Type Test Standards

For Substation Equipment:
IEC 60255-x
IEEE C37.90.x

Devices connected to the substation LAN are specifically Hardened

for the substation environment.
14
RuggedCom
Industrial Strength Networks A Substation Hardened LAN

Substation LAN

E th e rn e t S w itch

Digital Relay Metering IED RTU PLC LTC

What about the substation LAN?

It will also be exposed to the same environment as the IEDs. Most
often being mounted in the same rack side-by-side! 15
RuggedCom
Industrial Strength Networks IEC 61850-3 Communications networks and systems in substations

IEC 61850-3
New IEC Standard (Jan/2002)
Communications networks and systems
in substations

IEC 870-4 TS 6100-6-5 IEC 870-2-2

Reliability EMC Environmental 16
RuggedCom
Industrial Strength Networks IEC 61850-3 (61000-6-5): Apparatus Port Definitions

Power
Power Ports
Ports Enclosure
Enclosure Port
Port Signal
Signal Ports
Ports

(P/S In) APPARATUS (Local)

(Field)
E th ern et S w itch
(To HV)
(P/S Out)
(Telecom)
Functional
Functional Earth
Earth
Port
Port

(Other than Safety)

A port is defined as a particular interface of the specified apparatus

with the external electromagnetic environment
Type tests are defined and assigned for each specific port type.
17
 RuggedCom
Industrial Strength Networks IEC 61850-3 (61000-6-5): Substation Definitions

CONTROL BUILDING (Switch Yard)

High Voltage
Equipment
SHIELDED AREA
(if any) PROTECTION
TELECOM IED KIOSK
Power ROOM (L) (P) (L)
Line
Carrier
(F) (H)
(T) (F) (F)
Remote Earth Earth Network

Signal Port Connection Ethernet Sw itch Ethernet Sw itch

Types
Local (L)
Field (F) Typical Locations of
HV Equipment (H) Substation Ethernet Equipment
Telecom Power Line Carrier (T)
Protected (P)
18
 RuggedCom
Industrial Strength Networks
IEC 61850-3EMI Immunity Requirements
UTILITY IEC 61850-3 (61000-6-5) Communications Networks and Systems In Substations (Jan 2002)

TEST Description Test Levels Severity Levels

Enclosure Contact +/- 6kV 3
IEC 61000-4-2 ESD
Enclosure Air +/- 8kV 3
IEC 61000-4-3 Radiated RFI Enclosure ports 10 V/m 3
Signal ports +/- 4kV @ 2.5kHz x
D.C. Power ports +/- 4kV 4
IEC 61000-4-4 Burst (Fast Transient)
A.C. Power ports +/- 4kV 4
Earth ground ports3 +/- 4kV 4
Signal ports +/- 4kV line-to-earth, +/- 2kV line-to-line 4
IEC 61000-4-5 Surge D.C. Power ports +/- 2kV line-to-earth, +/- 1kV line-to-line 3
A.C. Power ports +/- 4kV line-to-earth, +/- 2kV line-to-line 4
Signal ports 10V 3
D.C Power ports 10V 3
IEC 61000-4-6 Induced (Conducted) RFI
A.C. Power ports 10V 3
Earth ground ports3 10V 3
IEC 61000-4-8 Magnetic Field Enclosure ports 40 A/m continuous, 1000 A/m for 1 s N/A
D.C. Power ports 30% for 0.1s, 60% for 0.1s, 100% for 0.05s N/A
IEC 61000-4-29
Voltage Dips & Interrupts 30% for 1 period, 60% for 50 periods N/A
A.C. Power ports
IEC 61000-4-11 100% for 5 periods, 100% for 50 periods2 N/A
Signal ports 2.5kV common, 1kV differential mode @ 1MHz 3
IEC 61000-4-12 Damped Oscillatory D.C. Power ports 2.5kV common, 1kV differential mode @ 1MHz 3
A.C. Power ports 2.5kV common, 1kV differential mode @ 1MHz 3
Signal ports 30V Continous, 300V for 1s 4
IEC 61000-4-16 Mains Frequency Voltage
D.C. Power ports 30V Continous, 300V for 1s 4
IEC 61000-4-17 Ripple on D.C. Power Supply D.C. Power ports 10% 3

EMI
EMI Immunity
Immunity type
type tests
tests reflect
reflect substation
substation environment
environment and
and phenomena.
phenomena.

More
More type
type tests
tests and
and much
much higher
higher test
test levels
levels than
than IEC
IEC 61000-6-2
61000-6-2 EMI
EMI
immunity
immunity standard
standard often
often used
used for
for industrial
industrial equipment.
equipment. 19
RuggedCom
Industrial Strength Networks IEC 61850-3 (61000-6-5): Performance Criteria

IEC 61000-6-5 Table 7 - Performance Criteria for the most relevant functions

(in descending order of criticality)

Functional requirements versus electromagnetic phenomena

Functions Transient phenomena Transient phenomena
Continous phenomena
with high occurance with low occurance
Protection and teleprotection
On-line processing and
regulation
Metering
Command and Control Short delay

Supervision
** No Delays or Data Loss ** Temporary loss, self
recovered
Man-machine interface Stop and reset
Alarm Short delay, temporary wrong indication
Data transmission and No loss, possible bit
Temporary loss
telecommunication error rate degradation
Data acquisition and storage Temporary degradation
Measurement Temporary degradation, self recovered

Off-line processing Temporary degradation Temporary loss and reset

Passive monitoring Temporary degradation Temporary loss

Self-diagnosis Temporary loss, self recovered
20
 RuggedCom
Industrial Strength Networks
IEEE P1613

P1613 Draft
S tandard Enviro nme ntal and Te s ting
Re quire me nts fo r Co mmunic atio ns Ne two rking
De vic e s in Ele c tric Powe r S ubs tatio ns

Sponsored by the
Substations Committee
of the
IEEE Power Engineering Society

IEEE working on North American equivalent of IEC 61850-3.

Newly approved as of March 20, 2003!

21
 RuggedCom
Industrial Strength Networks
IEEE P1613EMI Immunity Requirements
IEEE P1613 Draft Standard Environmental Requirements for Communications Devices Installed in Electric
Power Substations
TEST Description Test Levels Severity Levels
Enclosure Contact +/- 8kV N/A
IEEE C37.90.3 ESD
Enclosure Air +/- 15kV N/A
IEEE C37.90.2 Radiated RFI Enclosure ports 35 V/m N/A
Signal ports +/- 4kV @ 2.5kHz N/A
D.C. Power ports +/- 4kV N/A
IEEE C37.90.1 Fast Transient
A.C. Power ports +/- 4kV N/A
Earth ground ports3 +/- 4kV N/A
Signal ports 2.5kV common mode @ 1MHz N/A
IEEE C37.90.1 Oscillatory D.C. Power ports 2.5kV common & differential mode @ 1MHz N/A
A.C. Power ports 2.5kV common & differential mode @ 1MHz N/A
Signal ports 2kVac N/A
IEEE C37.90 Dielectric Strength D.C. Power ports 2kVac N/A
A.C. Power ports 2kVac N/A

Borrows
Borrows heavily
heavily from
from existing
existing IEEE
IEEE C37.90.x
C37.90.x standards
standards for
for Protective
Protective
Relaying
Relaying devices.
devices.
Fewer
Fewer tests
tests than
than IEC
IEC 61850-3
61850-3 but
but with
with test
test levels
levels just
just as
as high
high and
and higher
higher in
in
the
the case
case of
of Radiated
Radiated RFI:
RFI: 35V/m
35V/m v.s.v.s. 10V/m
10V/m !!
Defines
Defines two
two classes
classes of
of devices:
devices:
Class
Class 11 devices
devices allow
allow communications
communications errors
errors or
or loss
loss during
during EMI
EMI type
type tests.
tests.
Class
Class 22 devices
devices allow
allow zero
zero communications
communications errors
errors or
or loss
loss during
during EMI
EMI type
type tests.
tests.
22
 RuggedCom
Industrial Strength Networks
Climatic Requirements
IEC 61850-3Climatic Requirements

Four Classes of Locations: Class C Operating Temperature Ranges:

1. Class A: air-conditioned locations (indoor) 1. Class C1: -5 to +45C
2. Class B: heated or cooled enclosed conditions 2. Class C2: -25 to +55C
3. Class C: sheltered locations 3. Class C3: -40 to +70C
4. Class D: outdoor locations 4. Class Cx: Special (defined by mfg)
Conformance based on IEC 60068-2-1 &
IEC 60068-2-2!

IEEE P1613Climatic Requirements

Device Operating Temperature Ranges

a) -40 to +70C
b) -30 to +65C
c) -20 to +55C
d) Range defined by the manufacturer

IEEE P1613 also stipulatesNo cooling fans allowed!

Both IEC and IEEE allow for a variety of temperature ranges.

23
 RuggedCom
Industrial Strength Networks Fiber or Copper?

EPRI (AEP) 1997 EMI Immunity Testing

Transient IEC 1000-4-4

Generator E.F.T.
Copper
CAT5 Cable
Coupling
Coupling

Data Frames

24
 RuggedCom
Industrial Strength Networks Fiber or Copper: The EPRI Tests repeated

Transient
Generator IEC 61000-4-4
Electrical Fast
Copper CAT5 Cable Transients
Coupling
Coupling Medium
Medium

Data Frames
Media Media
Converter Converter
Fiber Optical
Cable
Network Analyzer
Simulator
(SmartBits)

Electrical Fast Transients (IEC 61000-4-4) Applied to CAT5 cable

Resultant Frame Loss:
32% @ 1kV Unacceptable Performance For
66%@ 2kV Real-Time Control!
75% @ -2kV 25
 RuggedCom
Industrial Strength Networks
Industry Studiesconfirm what most of us already knew!

Rockwell Automation - Study CMR of CAT-5 Cable

IEC 61000-4-6 (Induced RFI) SmarBits 600
Communications Tester
AC Mains
EUT
AC Signal
Port
Fiber Optical
DC Connections
Metallic DC Power Source
Plane
(grounded)
CDN-AF2 Power
Schaffner NSG2070-1 EUT Grounding
150KHz 80MHz Connection

Non-Metallic / Non-Conductive Table

Ground
Strap

80 cm

Metallic Ground Plane

Induced RFI (IEC 61000-4-6) Applied to CAT5 cable to test CMR.

Common mode noise coupling will occur via adjacent cabling
Resultant Bit Error Rate:
22% @ 10Vrms (noise coupled)!
Error rates of this magnitude render the network useless! 26
 RuggedCom Real-time Performance - An Illustrative Example During
a Power System Fault...
Industrial Strength Networks

Power System Fault

Current Waveform

= 16.7ms = 16.7ms

Fault period will be period of:

High levels of transient EMI phenomena!
Sub-cycle (i.e. 16.67ms) processing by IEDs and critical
communications performance by the LAN in ( < 4ms)!
Cant afford frame errors, delays or loss of
communications!
27
RuggedCom
Industrial Strength Networks

Real-time, Deterministic Performance

Look Ma, No Collisions!
 RuggedCom
Industrial Strength Networks
Key advances in Ethernet technology for real-time control!

! IEEE 802.3x Full Duplex Operation

! IEEE 802.1p Priority Queuing
! IEEE 802.1Q VLAN
! IEEE 802.1w Rapid Spanning Tree
! IGMP Snooping (Multicast Filtering)
The above features are essential for real-time deterministic
performance!

The above features are typically found only on Managed

Ethernet Switches! 29
 RuggedCom
Industrial Strength Networks
Benefits of Managed Switches

Additional Benefits of Managed Ethernet

Switches
! SNMP Simple Network Management Protocol
! RMON Remote Monitoring
! Port Mirroring for diagnostics
! SNTP Simple Network Timing Protocol
! Critical Alarm Annunciation
! Remote Configuration - via Telnet or Browser
30
 RuggedCom
Industrial Strength Networks
Look ma no collisions
Q. How do you make Ethernet deterministic or how do you
prevent collisions from occurring on an Ethernet network?
Ans. You use an Ethernet switching hub (i.e. Switch) with full-
duplex ports IEEE 802.3x!

S W I T C H

Tx Rx Tx Rx Tx Rx
Full-Duplex
Ports

IED
IED IED
IED IED
IED
#1
#1 #2
#2 #N
#N

Ethernet Switches with full-duplex ports dont have collisions!

31
RuggedCom
Industrial Strength Networks Minding your ps and Qs

Standard Ethernet Frame IEEE 802.1

SYNCH MAC MAC LENGTH

DESTINATION SOURCE OR DATA FCS
PREAMBLE ADRESSS ADRESSS TYPE

Ethernet Frame with new VLAN IEEE 802.1Q & Priority IEEE 802.1p Tag.

SYNCH MAC MAC 802.1p 802.1Q LENGTH

DESTINATION SOURCE VLAN OR DATA FCS
PREAMBLE ADRESSS ADRESSS PRIORITY ID TYPE

IEEE 802.1Q VLAN Tag

Allows
Allows priority
priority tagging
tagging of
of mission
mission critical
critical frames
frames 802.1p
802.1p

Allows
Allows isolation
isolation &
& grouping
grouping of
of IEDs
IEDs into
into virtual
virtual LANs
LANs 802.1Q
802.1Q VLAN
VLAN

32
 RuggedCom
Industrial Strength Networks Minding your ps and Qs

Q. How do I make sure during periods of heavy traffic that

critical messages make to it through without delay?
Ans. You tag them with a higher priority IEEE 802.1p.

High-Priority Queue

In-coming Frames
Priority
Port Transmit Queue
Queue
Mapping

Low-Priority Queue Inside the Switch

(per port)

IEEE 802.1p allows prioritization for critical messages! 33

 RuggedCom
Industrial Strength Networks
Minding your ps and Qs

Q. How do you segregate network traffic between devices used

for data collection and those utilized for real-time control?
Ans. You assign them to different VLANs IEEE 802.1Q.
Substation
Substation
Computer
Computer

S W I T C H

VLAN
VLAN VLAN
VLAN
11 22
IED
IED IED
IED IED
IED IED
IED IED
IED IED
IED IED
IED IED
IED
11 22 33 44 55 66 77 88

Protective
Protective Relaying
Relaying IEDs
IEDs Data
Data collection
collection IEDs
IEDs

IEEE 802.1Q allows segregation of devices into VLANs !34

 RuggedCom
Industrial Strength Networks
IGMP Management for Multicast Messaging..

Q. How do you listen to only the streaming data (multicast

messages) youre interested in?
Ans. You register to listen only to the streams youre
interested in via IGMP (Internet Group Multicast Protocol).

S W I T C H
Multicast Traffic

IED
IED IED
IED IED
IED IED
IED IED
IED IED
IED IED
IED IED
IED
11 22 33 44 55 66 77 88

Producer
Producer IEDs
IEDs e.g
e.g Sensors
Sensors Consumer
Consumer IEDs
IEDs e.g.
e.g. RTU
RTU

Multicast
Multicast Traffic
Traffic from
from Sensor
Sensor IEDs
IEDs can
can be
be assigned
assigned by
by the
the Switch
Switch to
to specific
specific
Consumer
Consumer IEDs.
IEDs.
35
RuggedCom
Industrial Strength Networks

Comparing Network Architectures

Flexibility, Redundancy, Performance
 RuggedCom
Industrial Strength Networks
CASCADING ARCHITECTURE

Cost-effective
Cost-effective bus
bus architecture
architecture messages
messages cascade
cascade from
from switch
switch to
to switch.
switch.

Maximum
Maximum number
number of
of hops
hops (N)
(N) is
is determined
determined by
by worst
worst case
case latency
latency
requirements.
requirements.

1 2 3 4 N
SWITCH SWITCH SWITCH SWITCH SWITCH

IED IED IED IED IED IED IED IED IED IED
IED IED IED IED IED
IED IED IED IED IED

Typical
Typical Switch
Switch Latency
Latency =5us
=5us ++ Frame
Frame Time
Time (100Mbps
(100Mbps Port)
Port)
Therefore,
Therefore, for
for aa 64
64 byte
byte frame
frame (512
(512 bits):
bits): the
the frame
frame time
time == 5.12us
5.12us (100Mbps)
(100Mbps)
Total
Total Latency
Latency perper switch
switch hop
hop == 5us
5us ++ 5.12us
5.12us == 10.12us
10.12us
For
For N
N == 10
10 the
the worst
worst case
case latency
latency would
would be:
be: 10
10 xx 10.12us
10.12us == 101.2us.
101.2us.
37
 RuggedCom
Industrial Strength Networks
STAR ARCHITECTURE

N
SWITCH

1 2 3 4 N-1
SWITCH SWITCH SWITCH SWITCH SWITCH

IED IED IED IED IED IED IED IED IED IED
IED IED IED IED IED
IED IED IED IED IED

Low-Latency
Low-Latency Architecture
Architecture Any
Any IED
IED to
to IED
IED communications
communications requires
requires
only
only two
two hops.
hops.

38
 RuggedCom
Industrial Strength Networks
RING ARCHITECTURE

N
SWITCH

PATH
PATH
2
1

IED IED
IED
IED
Fault
Fault

1 2 3 4 N-1
SWITCH SWITCH SWITCH SWITCH SWITCH

IED IED IED IED Fault

Fault Tolerant
IED IED Ring Architecture!
Tolerant Ring Architecture!
IED IED IED IED
IED IED IED IED IED
IED IED IED IED IED
Rapid
Rapid (i.e.
(i.e. ms)
ms) Reconfiguration
Reconfiguration via
via IEEE
IEEE 802.1w
802.1w Rapid
Rapid Spanning
Spanning Tree!
Tree!
Typical
Typical reconfiguration
reconfiguration times
times << 50ms
50ms for
for ring
ring with
with 10
10 switches.
switches. 39
 RuggedCom
Industrial Strength Networks Fault Tolerant Hybrid (Star/Ring) ARCHITECTURE

Fault
Fault
Fault
Fault
N N-1
SWITCH SWITCH
Fault
Fault

1 2 3 4 N-2
SWITCH SWITCH SWITCH SWITCH SWITCH

IED IED IED IED IED IED IED IED IED IED
IED IED IED IED IED
IED IED IED IED IED

Low-Latency,
Low-Latency, Fault
Fault Tolerant
Tolerant Architecture.
Architecture.
Able
Able to
to tolerate
tolerate failure
failure of
of uplink
uplink or
or backbone
backbone switches.
switches. 40
 RuggedCom
Industrial Strength Networks
High Redundancy Architecture via IEDs with Dual Ethernet Ports
Fault
Fault
Fault
Fault

SWITCH SWITCH SWITCH SWITCH

Fault
Fault

IED IED
IED
SWITCH

SWITCH
IED
SWITCH

SWITCH
IED
Fault
Fault

IED
IED
IED IED
IED

SWITCH
IED
SWITCH
SWITCH

SWITCH
IED
IED
IED
Fault
Fault

IED IED
IED

SWITCH
IED
SWITCH

SWITCH
IED
IED
IED

Able
Able to
to tolerate
tolerate multiple
multiple fault types!
Rapid
Rapid (i.e.
(i.e. ms)
ms) Reconfiguration
Reconfiguration via
via IEEE
IEEE 802.1w
802.1w Rapid
Rapid Spanning
Spanning Tree!
Tree!
IED IED
IED

SWITCH
IED
SWITCH
SWITCH

SWITCH
IED

41
IED
IED
 RuggedCom
Industrial Strength Networks
Key Takeaways
!Ethernet for substation applications should be substation hardened
to the same level as substation IEDs e.g. protection relays.
IEEE P1613 Class 2 error free networking device for real-time control
IEC 61850-3 Communications networks and systems in substations
Extended operating temperature range no fans!
Fiber should be used in high-noise environments for control applications.

! Real-time control applications require Managed Ethernet Switches:

IEEE 802.3x Full-Duplex operation for collision free operation
IEEE 802.1p Prioritization to allow real-time critical messages to get through.
IEEE 802.1Q VLAN to allow isolation of critical IEDs from non-critical IEDs .
IEEE 802.1w Rapid Spanning Tree to allow fault-tolerant ring architectures
with rapid (i.e. <50ms) reconfiguration times.
IGMP Snooping / Multicast filtering to prevent multicast intensive protocols
(e.g. EtherNet/IP) from inundating non participating IEDs on the LAN.

!Flexible networking architectures are possible with Ethernet:

Simple cost-effective Cascading or Star architectures
Fault-Tolerant Loop Architectures via IEEE 802.1w Rapid Spanning Tree
Hybrid Architectures with multiple-fault tolerance .
42
RuggedCom
Industrial Strength Networks

Thank you.