You are on page 1of 42

Processor Ethernet (PE) for Duplicated Main Servers

Issue 1.2
Compas ID 134539
Work Item: 8609

PROPRIETARY
-i-

Table of Contents

1. Introduction - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -1
1.1 Testability - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2
1.2 PRD Reference - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2
1.3 Scope - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2
1.4 External Documentation Impact - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2
1.5 Related Documents - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2
1.6 Document History - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -2
1.6.1 MR’s Incorporated in Current Issue - - - - - - - - - - - - - - - - - - - - - - - - - - -2
1.7 Terminology - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -3
2. Earlier Work - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -4
3. Architecture - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -5
3.1 PE State of Health (SOH) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -6
3.1.1 PE SOH Timing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -7
3.1.2 Failure Cases - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -7
3.1.2.1 Active Server Failure - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -7
3.1.2.2 PE Path Failure: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -8
3.1.2.3 Dual Active - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -8
3.2 TTS IP Telephone Behavior - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -9
3.3 H.248 Gateway Behavior - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 10
3.3.1 Existing Behavior - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 10
3.3.2 New Interchange Behavior - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 11
3.4 A Summary of CM Changes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 12
3.4.1 Socket Initialization - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 12
3.4.2 Mixed PE and IPSI Configurations - - - - - - - - - - - - - - - - - - - - - - - - - - 13
3.5 Server Interchange Link Bounce Behavior for Other Far Ends - - - - - - - - - - - - - - - - 13
3.5.1 H.323 Trunks - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 13
3.5.2 SIP Trunks - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 14
3.5.3 Adjuncts - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 14
3.6 CLAN vs. PE - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 15
3.7 LSP Operation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 15
3.8 Why no ESS; Why no Port Networks - - - - - - - - - - - - - - - - - - - - - - - - - - - - 15
3.9 PE NIC and Address Assignment - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 17
3.10 System Availability - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 19
3.11 Call Center Availability - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 20
3.12 Scalability - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 20
3.13 File Sync - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 20
3.14 Backup/Restore - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 20
3.15 Upgrades and Restore from an Earlier Release - - - - - - - - - - - - - - - - - - - - - - - 20
3.15.1 Duplicated Main Servers - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 20
3.15.2 Simplex Main/ESS/LSP Servers and Duplicated ESS Servers - - - - - - - - - - - - - 20
3.16 Operation on ESS/LSP with Main Server on an Older Release - - - - - - - - - - - - - - - 21
3.17 System Impacts - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 21
4. Hardware Requirements - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 21
5. H.248 Gateway Requirements - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 21
6. CM Software/Firmware Requirements - - - - - - - - - - - - - - - - - - - - - - - - - - - - 22
6.1 SAT Form Changes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 28
7. Integrated WEB Tools Impact - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 28

February 4, 2009 Avaya Inc. - PROPRIETARY


Use pursuant to Company Instructions
- ii -

8. Firewall - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 33
9. Other Systems Impacted - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 35
10. Special Requirements - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 35
11. Acknowledgements - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 37

February 4, 2009 Avaya Inc. - PROPRIETARY


Use pursuant to Company Instructions
PE for Duplicated Main Servers -1-

1 1. Introduction
2
3 Support for Processor Ethernet (PE) on duplicated servers was examined in an earlier work1, but not implemented
4
because the solution did not support recovery in the spirit of five 9’s availability. New ideas have surfaced which
5
6 are causing a re-look at the problem.
7 The new "idea" is based on the discovery that certain IP telephones will accept an incoming connection request from
8
9
a server on their gatekeeper list, use this new connection to replace an existing connection, and continue operation
10 without the need to re-register. This mechanism allows Communication Manager (CM) to quickly originate a new
11 connection to each of these telephones during a server interchange, causing the telephones to move quickly to the
12 server transitioning from the standby to active state.
13
14 Stimulated by the discovery of telephone behavior, a modification to H.248 gateways is specified to cause gateways
15 to accept an incoming connection as a signal that an interchange has occurred. This causes the gateway to re-connect
16 to the new server more quickly. Additional modifications to the "audit" behavior in CM immediately following an
17 interchange are also made to reduce processor workload immediately after the interchange.
18
19 These requirements are focused on support of Processor Ethernet (PE) on duplicated MAIN servers and server
20 interchange of duplicated MAIN servers when IP devices are connected to the PE interface of the MAIN servers.
21
Improvements are made to IP telephone and H.248 gateway recovery following a server interchange in the MAIN
22
23 CM servers. Behavior of other devices is not improved or modified by these changes.
24 The following limitations apply:
25
26 1. Support for Processor Ethernet (PE) on duplicated servers is limited to duplicated MAIN servers
27 only.
28
29 2. Port networks (G650) and ESS servers are NOT supported. Certain enhancements needed to state
30 of health calculations are not implemented in this release for a mixed configuration. See also the
31 discussion in on page 15.
32
33
3. Improvements are made for Time to Service (TTS) aware IP telephones only. These telephones
34 will require a new firmware release.
35 4. H.248 gateways will require a new release of software for this configuration. Improved availability
36 is only achieved if ALL H.248 gateways are upgraded to the new release of software.2
37
38 5. The S8710 server will NOT be supported (MR 68051).
39
40
A key goal of the work specified herein is to shorten the time for recovery following a server interchange in order
41 to improve availability. This work will make significant improvements to recovery times for TTS aware IP
42 telephones and H.248 gateways in the event of server interchange and hence will improve availability. The ability
43 to achieve five 9’s availability depends heavily on a significant improvement in gateway recovery time, which will
44 not be totally understood until modification of the gateway interface is implemented and measured. On paper, this
45
looks very favorable. A full discourse on this topic is provided by Bahareh Momken in "PE Duplication Availability
46
47 Analysis, CM 5.2" CID 135123.
48 Note that the work specified herein does not provide any improvement in the behavior of adjuncts. In some cases,
49
50
behavior could be worsened because links to the PE interface will drop whereas this does not happen when CLANs
51 are used.
52
53
54 1. SRAD: Processor Ethernet (PE) on Duplicated Servers, CID 122596.
55 2. Strictly speaking, this is not exactly true. The software will not prevent a mix of "old" gateways (or phones) and upgraded gateways
56 (phones) and the availability will depend on the mix. If one out of 250 gateways or one out of 12,000 phones is not upgraded, the impact
57 should be minimal, unless, of course, its your phone or the gateway you need. However, if only one gateway out of 250 and 1 phone out of
58 12,000 is upgraded, availability goals will not be met. It is possible that performance may be worse if only part of the system is upgraded,
59 but this needs to be understood via testing when the code is available.

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -2-

1 1.1 Testability
2
3 All requirements in this document are considered "testable" unless otherwise marked. Testable has a broad meaning
4 which includes inspection of code as well as lab based operation.
5
6 1.2 PRD Reference
7
8 This effort corresponds to work item 8609 to be MR’d into the PRD for Communication Manager 5.2, CID 132632.
9
10 1.3 Scope
11
12
This document does not affect current operation of PE on LSPs or ESSs. Operation of PE on these servers is not
13 changed by these requirements.
14
15 1.4 External Documentation Impact
16
See the Learning Development Plan for CM, Gateways, and Servers for R5.2, CID 134883.
17
18
1.5 Related Documents
19
20 • Processor Ethernet and Enhanced LSP, CID 104793
21
22
• SRAD: Processor Ethernet (PE) on Duplicated Servers, CID 122596
23 • Performance Analysis of Processor Ethernet on an S8500B in CM 3.1, CID 114055, and
24
• PE Performance Analysis, a power point presentation CID 122664.
25
26 • Flexible Ethernet Usage on CM Servers, CID 126798, work item 9419
27 • CM SRAD for Simplified Licensing via WebLM, CID 126467, work item 8472
28
29 • Auto-Migration of Media Gateway to Primary Media Server Controller ACM 2.1, CID 98465
30 • G700 Media Gateway: H.248 Link Bounce Recovery, CID 94767
31
32 • Maintenance Application Design: H.248 Link Establishment, CID 102216
33
34 1.6 Document History
35
Each official revision of the document is assigned a revision number. This section provides a historical record of the
36
37 revisions of the document.
38 Issue 0.n Initial pre-baseline drafts
39
40 Issue 1.0 First Baselined version
41
42 1.6.1 MR’s Incorporated in Current Issue
43 This section contains the MR number and abstract for any MR that has been incorporated into this document after
44
45
this document has been baselined.
46
47
48
49 MR # Included in Issue # Sections Affected Abstract
50
51 67263 1.1 section 7, Require- Inconsistency in where doc says PE address is stored
52 ments 42, 48, and 53.
53 67356 1.1 Requirement 20 Add a requirement for a feature bit to identify phone FW as
54 supporting the new recovery strategy.
55
56 67593 1.1 Section 3.15, Table 3, Correct usage of the terms server number and server ID
57 Requirements 30 and
58 44
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -3-

1
2 MR # Included in Issue # Sections Affected Abstract
3
4 67705 1.1 section 3.2 update the list of supported phones
5 67920 1.2 Table 5 Added a comment to arbiter ports 1332 and 1333 to say
6 they need to be opened on PE for PE-only systems.
7
8 67922 1.2 Requirement 37 Requirement 37 re-written to reflect enhanced alarm strat-
9 egy
10 68051 1.2 section 1.0 Item 5 added to introduction to note non-support of S8710.
11
12 68159 1.2 Table 5, Requirement Ports 81 and 411 modified per MR.
13 56
14
68345 1.2 -- This MR was no-changed.
15
16
17 1.7 Terminology
18
19 Glossaries of terminology can be found in the following references:
20 • Glossary (S75/S85/DEFINITY Terminology), J. H. Coyne, Compas ID 26837
21
22 • OR&M GLossary and Acronyms (OR&M 202), OR&M PMT, Compas ID 38023
23 • Mongoose Project Glossary, R. Windhausen, Compas ID 54154
24
25 • Glossary for DEFINITY R6 Technical Prospectus, J. C. Raphel, Compas ID 56340
26 • Stingray Project Glossary, Compas ID 70610
27
28 ACL Access Control List
29 AGL Alternate Gatekeeper List
30
31
ARP Address Resolution Protocol
32 ARQ Admission Request
33
CET Concept Evaluation Team
34
35 C-LAN or CLAN or clan. The IP interface board in a port network
36 CM Communication Manager
37
38 CNA/B Control Network A/B
39 DHCP Dynamic Host Configuration Protocol
40
41 ESS Enterprise Survivable Server
42 GCF Gatekeeper Confirm
43
44 GIP or gip: Gaz Interface Process within CM/TA
45 GK Gate Keeper
46
47
GRQ Gatekeeper Request
48 IP Alias IP aliasing is the process of adding more than one IP address to a network interface. With
49 this, one node on a network can have multiple connections to a network, each serving a
50 different purpose.
51
52 IPSI IP Server Interface. The board in a port network that talks to CM.
53 LSP Local Survivable Processor
54
55 MAC Address Media Access Control Address
56 MCIPADD Manually Configured IP Address
57
58 NIC Network Interface
59 OS Operating System

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -4-

1 PCD Packet Control Driver: The interface process within CM/TA that talks to IPSIs.
2
3 PE Processor Ethernet; a LAN interface inside of the TA.
4 PST Primary Search Timer
5
6
pTLS Pseudo TLS -- an variant on TLS used to connect to gateways
7 RCF Registration Confirm
8
RRQ Registration Request
9
10 Server Interchange The process in which a pair of servers change roles, the active server becomes the standby
11 server and the standby server becomes the active server.
12
13
SOH State of Health
14 SRAD System Requirements and Architecture Document
15
SUSER or suser: the user manager process within CM/TA
16
17 TA Telephony Application: the application running on a Communication Manager server that
18 provides telephony services as opposed to other applications such as the arbiter, watchdog
19 or global maintenance manager also running on the platform.
20
21 TTS Time to Service
22 UDP User Datagram Protocol
23
24 UDT Unsuccessful Discovery Time
25 In addition, abbreviations and acronyms are generally defined where first used within this document.
26
27
28
2. Earlier Work
29 An earlier SRAD was written on this same subject and can be found in compas 122596, SRAD: Processor Ethernet
30
(PE) on Duplicated Servers. This earlier work contains a large amount of background material that generally will not
31
32 be repeated here.
33 It is possible to assign multiple IP addresses to a given NIC. This is known as IP aliasing. See figure 1. The PE
34
35
36
37 TA
38 GIP
39
40 PE PCD
41 procr=ipC
42
43
44 CM Server
45
nic
46
47
48
ipA ipC
49
50 Figure 1. IP Aliasing
51
52
53
function can be assigned to one of these "alias" addresses as illustrated in the figure.
54
55 When CM main servers are duplicated, IP aliasing can already be used for the control networks and the customer
56 LAN interfaces in a special way. For each NIC assigned to a given function (CNA, CNB, Customer LAN), one
57
58
address is assigned uniquely to each server and the second address is assigned the same (shared) address on both
59 servers. However, only one server (the active server) responds to the "shared" IP address. During a server

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -5-

1 interchange, the active server becomes the standby server and the standby server becomes the active server. The
2 server becoming the active server sends an "ARP broadcast" for the shared address on the interface where the
3
4
aliasing is administered. This causes all connected devices to update their "ARP cache" to point to the new active
5 server. Figure 2 illustrates this with focus on the NIC assigned to the PE interface. Note that PE interface is not active
6 ipA = Ip address of server A
7 TA ipB = ip address of server B TA
8 ipC = shared ip address
9 GIP GIP
10 PCD PCD
PE
11
12 procr=ipC
13
14 CM Server B
15 CM Server A
16 nic nic
17
18
19 ipA ipB
20
21
22 ipC
23 Sign
24 ath aling
gP Path
25 Si g nalin
26
27 Talk Path
28
29
30
Figure 2. IP Aliasing on the PE interface
31
32
33
on the standby server because the GIP is loaded but not executing.
34 Note the following:
35
36 • Using an alias IP address means that all devices connected to this interface see the same IP address regard-
37 less of which server is active.
38
39 • The active server attempts to capture control of the IP address assigned to the alias
40 • Only signaling links go through CM servers. The bearer channel (talk path) does not; it might connect to a
41 media processor but not to the server itself.
42
43 • In the above figure, the telephone is a generic stand-in for any device connected to the TA via PE.
44
45 3. Architecture
46
47 Processor Ethernet on duplicated MAIN servers will "work" if we simply remove the blocks in the code that prevents
48 its use today. However, removing such blocks is only of interest if doing so results in a system with sufficient
49 availability. For a discussion of what is sufficient, see PE Duplication Availability Analysis, CM 5.2, CID 135123.
50 Suffice it to say here, that the following areas must be addressed in order to support a notion of availability for the
51 core servers and equipment:
52
53 • Each server must be able to determine the state of health (SOH) of its Processor Ethernet (PE) interface .
54
55
• The SOH of the PE interface must be included in server interchange decisions.
56 • H.248 gateways must switch allegiance to the server becoming the active server in a timely fashion and
57 without undue burden on that server.
58
59 Additionally, IP telephones must recover from a server interchange in a timely fashion and without undue burden

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -6-

1 on the CM server becoming the active server if the customer’s experience is to be one of high availability.
2
3 3.1 PE State of Health (SOH)
4
5 Determining the health of the PE interface is not as easy as it might seem at first glance. Approaches to making this
6 determination include the following:
7
8 • The gip is the termination point for the PE interface. So perhaps the gip could report loss of message traffic. But
9 this isn’t reliable by itself because there may be legitimate periods when there is no message traffic on this
10 interface and a failure would go undetected for an unknown length of time. The gip is not running on the standby
11 server and so PE SOH could not be determined on the standby.
12
13 • Perhaps a ping message could periodically be sent out the NIC associated with the PE interface to the NIC
14 associated with the PE interface on the other server. If the message receives a proper response, this confirms that
15 the NIC assigned to the PE interface is working. However, if the message receives no response the result is
16 inconclusive. If either of the NICs fail, both servers will think their PE interface is down.
17
18 • Perhaps a ping message could periodically be sent out the NIC associated with the PE interface on the local server
19 to each of the NICs on the other server. This would require that there be more than one NIC on each server and
20 that all the LANs to which these NICs connect are interconnected. If we ever implement the feature called
21 flexible NIC assignment, it is possible there would be only one NIC (plus the laptop NIC) on each server. Further,
22
23
it is not a good assumption that the LANs are always interconnected.
24 • To solve the above dilemma another device must be available on the local subnet. It must be on the local subnet
25 so as not to confuse failure of PE’s NIC/path with downstream network failures. The possibilities include the
26 following:
27
28 1. The data network default gateway. In some sense, this is the ideal choice because the default gate-
29 way must be present on the local subnet anyway. However, a Communication Manager server
30 could have multiple NICs and it is possible that the customer has assigned the default gateway to a
31 LAN not connected to the NIC associated with the PE interface. For example, the "customer LAN"
32
33
might be assigned to a NIC different from the one used by PE and the default gateway could be on
34 the "customer LAN".
35 2. An H.248 gateway on the local subnet. An H.248 gateway on the local subnet has the advantage of
36 being part of the communication system. However, the data network default gateway is still a bet-
37
38
ter choice, because reaching the default gateway is generally necessary to reach equipment in other
39 parts of the network.
40 3. Another device that will respond to queries used to determine the SOH of the PE interface.
41
42 Another issue in determining PE SOH is use of the ping message. The easiest message to use is ping since it doesn’t
43 require some special new code in the router. However, ping can be disabled in the router or in firewalls. An
44
45
alternative is ARP ping1, which is a layer 2 message that asks, "who has this IP address". This message won’t be
46 blocked because it is necessary to make layer 2 function properly. Thus the ARP ping will be used instead of ping.
47
To conclude, the SOH of the PE interface must be determined independent of the health of the other server via
48
49 another device on the same subnet as the NIC associated with the PE interface. Choices for this device, in order of
50 preference, are:
51
52
a. The data network default gateway
53 b. An H.248 gateway on the local subnet
54
c. Another device on the local subnet
55
56 The following table shows the SOH contribution provided by pinging both the other server and some other device
57
58
59 1. Thanks to Phil Whelan for this great solution.

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -7-

1 on the subnet using the NIC assigned to the PE interface.


2
3 Table 1. PE SOH Contribution
4
5 Reply
Reply
6 Received
Received
7 From the PE SOH
from Default
8 Other
Gateway
9 Server
10 yes yes healthy
11
yes no healthy
12
no yes healthy
13
14 no no PE SOH lowered
15
16 3.1.1 PE SOH Timing
17
18 In order to avoid lowering SOH of the PE interface due to short transient conditions, failure is not declared on a
19 single lost ping message response. Three lost responses are normally used. As figure 3 illustrates, if the time between
20 ping messages is Tp and if the failure occurs immediately before a ping response is to be sent, the time to detect the
21 failure is essentially only 2 ping intervals (2 x Tp). If the failure occurs immediately after a ping response is sent,
22
23 then failure isn’t detected for 3 ping intervals (3 x Tp), which is the worse case.
24
25
26 Tp ping/response
27
28 Failure immediately before ping response
29 1 2 3 declare PE
30 Failure
Time
31
32
33 Failure immediately after ping response
34
1 2 3
35
Failure declare PE
36 Failure
37
38
39 Figure 3. Gateway Failure Detection
40
41
42 3.1.2 Failure Cases
43
44 There are three failure cases of interest. Note that in all cases it is assumed that the standby server is refreshed.
45
46 1. The active server fails. In this case the active server does not close open sockets and does not respond to
47 any messages on any interface beyond the time of failure.
48 2. The network path being used by PE on the active server fails. This might be due to a cable being
49
50
unplugged, an ethernet switch failure, or a failure of the network interface (NIC) on the Communication
51 Manager (CM) server. In this case, the active server responds on other interfaces, e.g. duplication link, but
52 connections are not closed on the PE path and no messages are sent or received on this path following the
53 failure.
54
55
3. An operator initiates an interchange. In this case the active server will close connections, which means
56 all connected devices will become aware of the interchange as it happens. This produces the shortest recov-
57 ery time and is of least interest here as it is not really a failure.
58
59 3.1.2.1 Active Server Failure

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -8-

1 If the active server fails, an interchange is made up of the following time segments:
2
3 Arbiter Detect: "Inter-arbiter heartbeats are generated every 875 milliseconds, with SOH changes
4 being transmitted asynchronously and within 1 millisecond. The timeout for
5 having a Standby server start to go active is 4.1 seconds. So with those those two
6
bits of information, we know that if the active server fails, the Standby will start to
7
8 go active within 3.225 - 4.100 seconds." (E.-mail from Luke Young)
9 PE Verification1: The standby server needs to know it has a good PE interface. If the ping rate is one
10 per second, the standby server will receive confirmation that its PE interface is
11
working within 1 second, so this time does not add to the interchange time. That is,
12
13 the standby server will receive multiple responses on a good PE interface in less
14 time than it takes the arbiter to decide the active server is not responding.
15 Go Active: The arbiter on the standby server causes processes on the standby server to start up.
16
4-7 seconds.
17
18 Interchange Time: The interchange time is the sum of the above times or 7.2 to 11.1 seconds. Note that
19 this is the first point at which the newly active server could enter "normal op".
20 Changes specified herein will lengthen the time to "normal op" in order to speed
21
recovery.
22
23 3.1.2.2 PE Path Failure:
24
25 When an interchange is done as a result of a failure of the PE interface/path on the active server, the time to perform
26 the interchange is made up of the following time segments.
27
28 PE Detect If the ping interval is once per second, it will take the active server 2-3 seconds to
29 verify that its PE path has failed.
30
Arbiter Detect It will take the arbiter in the active server 1 millisecond to decide to do an
31
32 interchange in the face of a critical loss of call control (no PE interface). So this
33 time can be ignored.
34 PE Verify The standby server will be sending ping messages once per second and will receive
35
multiple responses on a good PE interface in less time than it takes the active server
36
37 to lower its PE SOH, so this time can be ignored.
38 Go Active: The arbiter on the standby server causes processes on the standby server to start up.
39 4-7 seconds.
40
41 Interchange Time: The interchange time is the sum of the above times or 6 to 10 seconds. Note that
42 this is the first point at which the newly active server could enter "normal op".
43 Changes specified herein will lengthen the time to "normal op" in order to speed
44 recovery.
45
46 3.1.2.3 Dual Active
47
48 It is possible that both servers could go active at the same time and remain so for an undetermined duration. This is
49 not a supported configuration and any service provided in this state is provided courtesy of the gods and may be
50 interrupted randomly via the same courtesy. These requirements do not necessarily increase nor decrease the
51 likelihood of a dual active condition. If both servers are active, each will try to take over control of the alias IP
52
address associated with each LAN connected to the servers. This will occur on the NIC associated with PE as well
53
54 as the NICs associated with the control LANs or Customer LANs.
55
56
57
58 1. If a standby takes over as active because it cannot hear from the active, it will still try to go active, REGARDLESS of what the standby's
59 SOH is. ( E-mail from Luke Young. )

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers -9-

1
3.2 TTS IP Telephone Behavior1
2
3 The behavior described in this section is for Time to Service (TTS) aware IP telephones that have an as
4 yet-to-be-developed version of firmware in accord with IPT MR# IPT00036834, written on our behalf by Cherian
5
6 Abraham. As of July 2, 2008, the plan for TTS aware telephones that will support the new firmware2 include the
7 following:
8
9 • 46xx Broadcom sets if firmware R2.8 or later release.
10 - 4601+, 4602SW+, 4610SW, 4620SW, 4621SW, 4622SW, 4625SW
11
12 • 96xx all phone models if firmware S1.2.1 or later
13 - 9610, 9620, 9630, 9640, and 9650
14
15 The following telephones do not support the new firmware:
16 • 16xx telephones
17
18 • 4601/02/06/12/24/30/90
19 • 46xx Agere sets
20
21
- 4620, 4606, 4612 and 4624
22 The phone firmware modification is designed specifically to address server interchange when telephones connect to
23
the server PE interface. In this case the telephone has a connection/socket up to the active server at IP address IPa
24
25 with MAC address M1; a server interchange occurs.
26
27 1. There is a time period on the order of 10 seconds during which neither CM server is providing service.
28 During this time, messages from telephones will receive no response (messages may be "lost"), even
29 though the connection may "appear" to be up.
30
31
2. The existing connection/socket from the phone to the CM server may or may not be torn down (FIN)
32 3. When the standby server transitions to the active state, it will open a new socket to each telephone that had
33 a connection to the other server. The connection will originate from IP address IPa and MAC address M2;
34 that is, the same IP address but a different MAC address.
35
36 4. The telephone will respond as follows:
37 a. If the incoming connection is from the same IP address (IPa) as the current connection or the last
38
connection, in case there is no current connection, and the connect arrives prior to the primary
39
40 search timer expiry: See figure 4.
41 • The phone shall accept the incoming connection. (TCP syn, syn/ack, ack)
42
43
• The phone shall close the "old" connection if it has one (FIN)
44 • The phone shall simply replace the old connection with the new one without attempting to
45 register or establish a new session (setup olc). The newly active server will have shadowed
46 status/state data from the active server.
47
48 • Should this process fail for any reason, the phone shall do normal recovery using its exist-
49 ing timers.
50
b. If the incoming connection is not from the same IP address, the phone will behave as it does today.
51
52 The following description is provided courtesy of Rob Mitchell.
53
54 1. Note that these requirements are concerned with server interchange, not server initialization or boot-up. SV notes that we are limited to
55 about 5000 encrypted phone connections on initialization; more than this and the server never boots. These requirements do not address
56 this problem. Also note that encrypted phone links provide better security from a number of perspectives at the cost of increased boot time.
57 2. The list of telephones is believed accurate as of the date shown. Which phones will or will not support the new firmware is neither con-
58 trolled by, nor baselined in this SRAD. This list will not be updated if the supported set models change in the future. Consult telephone
59 development for this information.

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 10 -

1 My answer is grounded in the assumptions implicit in your Section 3.2 -- we're talking TTS environments, and the
2 telephone has had a normal registration in progress. The source RFS and Requirement for the 96xx sets is
3 COMPAS ID 111782, Requirement 96xxACP.2.1.500, and for the 46xx sets it is COMPAS ID 76339, Requirement
4 46xxDefIP.3.1.607.
5 Case 1: The telephone received a FIN, or the TCP keep-alives time out, before getting the incoming TCP
6 connection invite -- this case is generically termed "TCP Connection failure". In this case, the phone starts the
7 PST (Primary Server Search Timer), sets the "current server address" to the first primary server address in the
8 Gatekeeper List, and sends Keep-alive RRQs to the current server address until the KA-RRQs time out (or we get
a response). In the case of a timeout, the telephone sends a KA-RRQ to the next server address, etc., until the
9
PST expires, in which case the phone goes back to the beginning of RAS.. In the case of a positive response
10 (KA-RCF with matching module ID), the telephone assumes it has found the server, and uses the (potentially new)
11 server address.
12
While all the above is going on, if a TCP connection request comes in, the phone checks to see if the request
13 came from an address in its Gatekeeper List. If no, the telephone ignores the request; otherwise, the telephone
14 accepts the TCP connection, cancels the PST, and returns to normal registration.
15
Case 2. The telephone has not received a FIN, or the TCP keep-alives time out, before getting the incoming TCP
16 connection invite. The key difference between this Case and Case 1 is that in this Case, the telephone does not
17 know it has lost its connection to the original server. Therefore, the phone assumes it still has a TCP connection
18 with the original Gatekeeper (assume it is GK A). In this Case, the telephone sends a KA-RRQ to GK A, and waits
19 for a response. If the telephone gets a KA-RCF (presumably not, in your scenario), the phone ignores the TCP
20 Connection request from the new server (GK B). More likely, the telephone's KA-RCF and retries all timeout, then
21 and only then does the request from GK B get accepted.
22
23
Note that the interchange process works for both H.235 Annex H and non-annex H phones although recovery of
24 Annex H phones may take longer.
25
26
27 Standby
Active
28
29 Server A Phone Server B
30
31 Existing Connection
32
Server interchange from A to B
33
34 Standby Active
35 Server A Phone Server B
36
37 Syn - TCP establish
38 Syn/ack - TCP establish
Connection status unknown;
39 might have closed; might not.
40 ACK - TCP establish
41
42 phone drops old Connection if it exists Continue processing as if no failure
43
44
45
46
47 Figure 4. Phone Behavior During Interchange
48
49
50 3.3 H.248 Gateway Behavior
51
52 3.3.1 Existing Behavior
53
54 The existing H.248 gateways reconnect to a CM server in the following high level sequence:
55
1. The gateway establishes the TCP connection
56
57 2. The gateway initiates a PTLS session (if the session is to be encrypted)
58 a. CM starts key exchange
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 11 -

1 b. The gateway acknowledges key exchange


2
3 c. CM acknowledges the gateway's acknowledgment.
4 3. The gateway sends registration
5
6
a. CM either accepts registration or denies it. If the latter, the gateway tries again about 5 sec-
7 onds later.
8
Following registration:
9
10 4. User Manager handles gateway registration, sends a registration stim to the HMM and stims itself to run
11 the provisioning process. This function downloads some 1500-2600 bytes of parameters.
12
13 5. User Manager tells HMM of gateway registration
14 20 (0x0014) = MO_ANALYZE, really runs the alarm resolution function.
15
16 6. Gateway sends the board information, i.e.. what boards are present
17 1522 (0x05f2) = MG_FIRST_BD_CHG 1522 - handle first gateway bd chg msg
18
19 7. Gateway starts the context rebuild process
20 1617 (0x0651) = MG_CTXT_RBLD 1617 - start user manager context rebuild
21
22
8. Bring up D channels
23 1641 (0x0669) = CK_MG_DCHAN 1641 - Check for D chan ports on gateway
24
9. Audit the physical port connections
25
26 1523 (0x05f3) = MG_ADT_CONN 1523 - audit connections
27 10. HMM tells the user manager to let in the next gateway
28
29 11. Get the tone detector count
30 1637 (0x0665) = MG_TTR_AUDIT 1637 - get available TDs on a GW
31
32 12. Three minutes later
33 1586 (0x0632) = MMM_BDM_AUD 1586 - MMM BDM audit
34
35 13. If connection preservation timed out;
36 1524 (0x05f4) = MG_TERM_ADT 1524 - audit MG terminations
37
38 3.3.2 New Interchange Behavior
39
40 The new behavior on server interchange is as follows:
41 1. As was described for IP telephones in the previous section, there is a time period on the order of 10 seconds
42
43
during which neither CM server is responding to messages from any devices, even though the gateways are
44 not yet aware there is a problem. Messages sent during this time are most likely to be lost.
45 2. The existing connection/socket from the gateway to the CM server may or may not be torn down (FIN)
46
47 3. When the standby server transitions to the active state, it will open a new socket to each gateway that had a
48 connection to the other server. The connection will originate from IP address IPa and MAC address M2;
49 that is, the same IP address but a different MAC address.
50
4. The H.248 gateway will respond as follows:
51
52 a. If the incoming connection is from the same IP address (IPa) as the current connection or the last
53 connection, in case there is no current connection:
54
55
• The gateway shall accept the incoming connection. (TCP syn, syn/ack, ack)
56 • The gateway shall close the "old" connection if it has one (FIN)
57
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 12 -

1 • If the gateway is using a secure connection, the gateway will initiate re-establishment of
2 the secure connection. Note that re-establishing a secure connection will take longer than
3
4
the time for a non-secure connection.
5 • No other recovery messages shall be required.
6
• The gateway shall not reset in response to just the connection change. Both connections
7
8 and calls shall be preserved.
9 • Should this process fail for any reason, the gateway shall do normal recovery using its
10 existing timers.
11
12 b. If the incoming connection is not from the same IP address, the gateway will respond to the incom-
13 ing connection with a FIN.
14 5. CM shall not artificially rate limit the set up of secure connections to the gateways.
15
16 6. CM shall bring the gateway into service immediately on negotiation of the new connection without waiting
17 for ANY audits, in particular context audits. Audits shall be run at normal background priority.
18
7. CM shall respond to a context error by auditing the single context on which the error was received.
19
20
3.4 A Summary of CM Changes
21
22 There are 4 main areas of change being made to support the PE interface on duplicated MAIN servers:
23
24 1. Modify the suser and gip processes to perform special socket initialization on interchange
25 2. Add PE to the State of Health (SOH) calculation
26
27 3. Remove blocks on use of the PE for duplicated MAIN servers and add necessary administration
28 and initialization.
29
4. Increase capacity of gip/suser processes if/as necessary e.g. file descriptors, message queues, simul-
30
31 taneous requests
32 The next sections provide additional discussion on the first two items.
33
34 3.4.1 Socket Initialization
35
36 As a duplicated pair of MAIN servers operate, status information is transmitted from the active server’s memory
37 tables to the standby server’s memory tables in a process known as memory shadowing. Of particular importance to
38 the present discussion is the connection information from the suser and gip processes to external devices as
39
illustrated in figure-5.
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 13 -

1
2
Active Server Standby Server
3
4
5 suser suser
6 Oryx/Pecos Definity Socket Memory Shadowing Oryx/Pecos Definity Socket
(D-Socket) (D-Socket)
7
8 gip gip
9
Linux Socket Linux Socket
10
11 Linux OS Linux OS
12
13
14
15 TCP connection
to phone
16
17 Figure 5. Data Shadowing
18
19
20
21
22
The suser process has a "D-socket" (Oryx/Pecos connection) to the gip for each connected device; the gip in turn
23
24 has an associated Linux socket to the OS to maintain a TCP connection to the device. Tables containing data for
25 these connections are shadowed from the active server to the standby server as illustrated in the figure. Immediately
26 after a server interchange, the newly active server has a set of tables in the suser and gip processes containing data
27 for all the device connections, but the Linux socket does not exist; there is no real connection to the device. In the
28 current system, the D-sockets are closed, the gip socket tables are cleaned up and an entirely new connection is
29
30
established. How the connections get re-established depends on the specific device; see CID 122596.
31 These requirements will have the suser process mark the sockets connected to H.248 gateways and TTS aware IP
32
telephones as the sockets are created. Then, on interchange, the suser will not tear down these sockets; the gip will
33
34 automatically create new Linux sockets for all sockets marked for H.248 gateways and TTS aware IP telephones.
35 Tests on a Dell 1950 class machine indicate that Linux can open 12,000 sockets in under 3 seconds. Because this
36
37
can happen very quickly, the suser/gip can be modified to re-establish all the TTS sockets prior to entering suser
38 "normal op" and a priority among these sockets is not necessary. For example, it was originally thought that busy
39 phones would be brought back first or maybe call center phones first; but if they all come back in 3 seconds, order
40 doesn’t matter.
41
42 3.4.2 Mixed PE and IPSI Configurations
43
44 Determination of SOH for the PE has already been described in section 3.1 on page 6. Although port networks are
45 not supported in this release when PE is enabled on duplicated main servers, administration is not blocked. In this
46 release, an interchange will occur to the server with the best SOH for PE regardless of SOH of the IPSI connections.
47 If the SOH of the PE interfaces is the same on both servers, then the existing algorithms for IPSI connectivity will
48 govern.
49
50 3.5 Server Interchange Link Bounce Behavior for Other Far Ends
51
52 3.5.1 H.323 Trunks
53
54 H.323 trunks currently connect to two places -- other CM servers or the AT&T network. Calls to other CM servers
55 generally recover well from a link bounce; calls to the AT&T network generally drop. Remember that the talk path
56 does not involve the PE interface and so the talk path will remain up until either end takes action on the signalling
57
link to disconnect it.
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 14 -

1 Following a server interchange initiated link bounce on the PE interface, the newly active server will attempt to
2 re-establish the signalling socket for all known H.323 trunks after entering "normal op". CM retains the call records
3
4
for calls on these trunks for up to two hours. If the signaling link for the call can be re-established for the call within
5 this time, operation is resumed. If it cannot, CM will disconnect the call. The far end (e.g. AT&T) may choose to
6 disconnect the call sooner.
7
8 3.5.2 SIP Trunks
9
SIP trunks are currently used in CM to support the following types of connections:
10
11 • To connect to another CM
12
13 • To connect to a service provider such as AT&T
14 • To connect to SES
15
16
• To connect to a G860 high density trunk gateway
17 In all cases, the behavior is the same. The end that needs to send a message (CM or the "other end") will bring up
18
the trunk if it needs to do so to send a message. The trunk will be established as needed and not necessarily at the
19
20 time of server interchange. This means that link bounce for SIP is pretty innocuous.
21 If there are signaling messages being exchanged on a SIP trunk precisely at the time of failure and if a message is
22
23
lost, the associated call (or action) may fail. Activity during an interchange is not guaranteed regardless of interface
24 (PE or CLAN) or type of application (SIP, IP, DCP, etc.).
25
Specifically regarding SES:
26
27 Per Marsh Gosnell: "SES doesn't really care about link loss. SES is largely oblivious to the state of the connection
28 (actually connections since there is one in each direction). If a connection doesn't currently exist when SES wants
29 to send CM a SIP message, it will create a new one. If SES hasn't yet discovered that the connection is broken,
30 it is possible for one SIP message destined for that connection to fail and get returned with an appropriate error.
31 Likewise, SES will gladly accept a new connection from CM and receive messages on it and SES will eventually
tear down the dead connection
32
33 SES expects CM to send us an unsolicited NOTIFY when it starts up so that SES can re-establish its bulk
34 subscription. In the case of link bounce, we don't expect to get a NOTIFY. I don't know if enough stuff is mirrored
on CM to keep track of the SES subscriptions so CM may have no choice but to send that NOTIFY to SES after
35
an interchange."
36
37 Specifically regarding G860, Xiaoyan(Sharon) Xiong ran the following test:
38
39 "I made a call from PSTN stimulating switch (DS1 to T3) through G860 then to CM via sip trunk, left call up,
unplugged CLAN cable for 30 seconds. After reconnecting network cable to clan, checked talk path (it remained
40
up), made a call transfer from CM to another CM extension. It worked fine. Repeated once, result is the same."
41
42 Although this is not exhaustive testing by any means, it does tend to confirm the G860 should have reasonably good
43 behavior during a server interchange link bounce.
44
45 3.5.3 Adjuncts
46
47 CID 122495 provides additional detail regarding adjunct behavior. Table 2 shows a summary of anticipated
48 behavior of adjuncts as a result of a server interchange initiated link bounce on the PE interface when these devices
49 are connected to the PE interface.
50
51 Table 2. Adjunct Response to Link Bounce
52
53 Coded
54 Adjunct for Link Comments
55 Bounce
56 AES, IR, IMS, softphone, yes These devices understand link bounce and attempt to recover with minimal user impact. Specific
57 IP agent softphone, SIP times, affects of operator behavior etc vary by product.
58 softphone, SES
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 15 -

1 Table 2. Adjunct Response to Link Bounce


2
3 Coded
4 Adjunct for Link Comments
5 Bounce
6 MultiTech Gateway,G150, No but OK These devices are not coded for link bounce specifically but have other behaviors which make
7 operation through a link bounce acceptable.
8 CMS No May cause a re-pump of the CMS data base. CMS detects link failure within 15 seconds.
9 This product is not being enhanced. Note that SV observes that the re-pump "always" occurs
10 and may take 30 minutes to complete.
11 IQ/CCR no Expected behavior is similar to CMS.
12 Voice Portal, IALX, MM, No See CID 122596.
13 Legacy Octel
14 Apollo No all calls will drop
15 emmc, meeting exchange, No All calls will drop
16 crystal
17
18
3.6 CLAN vs. PE
19
20 There are significant differences between connecting through CLANs and connecting directly to the PE interface.
21
22 • When connecting through a CLAN, sockets are not lost during an interchange. Except for a few second
23 loss of service during the actual interchange, an interchange with CLANs can be transparent to telephones.
24 An interchange will never be transparent if connectivity is through PE.
25
26 • When connecting through CLANs, multiple CLANs are generally used. Failures do not necessarily affect
27 all the CLANs at the same time. Some kinds of failures (e.g. network outages) may affect only a subset of
28 users. However, connectivity through PE is like having a single CLAN for the entire system. If the PE is
29 affected in any way, failure is total; all users are affected. This tends to result is a larger disruption.
30
31 3.7 LSP Operation
32
33 Operation of LSPs is not altered by the changes specified herein.
34
35 3.8 Why no ESS; Why no Port Networks
36
37 The primary motivation for supporting Processor Ethernet on duplicated servers is precisely to provide a
38 configuration that has no port networks. This results in a system with a smaller hardware footprint, with lower
39 hardware cost, and with a better competitive configuration in the marketplace.
40
41 It is recognized that existing customers would want to use Processor Ethernet in place of adding additional CLANs,
42 especially if they would need to add another port network to accommodate the additional CLANs. Technically, this
43 could be supported in some future release. However, the present effort is focused on delivery to release 5.2 which
44 is currently scheduled for February of 2009 (and these requirements are being written in May 2008). In order to fit
45
46
the development in this compressed schedule, the work is being minimized. The additional work to support port
47 networks in both development and testing is beyond what can be done in the short term. The next paragraphs provide
48 an overview of the problem/work.
49
50 • Processor Ethernet on duplicated main servers requires that the PE interface be assigned to the shared, alias
51 address of the servers; because of the way the system is currently designed, PE on duplicated ESS servers
52 requires the PE interface to be assigned to the server unique IP address.
53
• Processor Ethernet cannot be used on simplex ESS either, as will be discussed shortly.
54
55
56
57
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 16 -

1 • If port networks are used on the main servers, then an ESS must be present in the configuration to provide
2 a survivability story. In the event that an ESS takes control, all the devices connected to PE on the main
3
4
server would have to connect to CLAN on the ESS. This means that the port networks/CLANs must have
5 sufficient capacity to handle all devices without using PE on the main. This defeats the purpose of trying to
6 use PE for cost reduction in the first place.
7
8 The obvious question is, why can’t we use PE on the ESS servers like we do on the main servers. The answer is
9 related to the technical implementation of LSPs and ESSs and what makes an LSP different from an ESS. The
10 primary distinguishing feature that makes the two different is that a port network can connect to an ESS but not an
11 LSP and IP devices can connect to the PE on an LSP but not to the PE on an ESS. If IP devices connect to PE on an
12 ESS, what is it? An ESS or an LSP? Lets look at LSPs and ESSs more closely.
13
14 • ESS: Enterprise Survivable Servers (ESS) are designed specifically to support port networks. ESS servers
15 provide no service until one or more port networks choose the ESS server for service via the Packet Con-
16
trol Driver (PCD) interface in CM. Telephones and gateways connect through a CLAN to the ESS server
17
18 and never to the PE interface on the ESS itself. In order to return service to the main servers, the port net-
19 works must move back to the main, which they can do on command or at a specified time. The return to
20 main policy is administered on the system-parameters ess form as shown in figure 6. The ESS server per-
21
22
23
24
25
26
27 Figure 6. Change
28
29
system-parameters ess
30
31
32
33
34
35
36
37
38
39 forms a reset system 4 when it no longer controls any port networks. The reset system 4 is used to clear
40 alarms, busy-outs and allow any pending translations to be loaded.
41
42
• LSP: Local Survivable Processors (LSP) are designed to support H.248 gateways. Port networks cannot be
43 hosted on an LSP. LSPs provide no service until (1) at least one H.248 gateway registers with the LSP and
44 at least one IP telephone registers with the LSP. Because there are no port networks there are no CLAN
45 interfaces and all IP devices must connect to the LSP’s PE interface. In order to return service to the main
46 server(s), the H.248 gateways poll the main every 30 seconds when connected to the LSP. If the main can
47
be contacted, the gateway tries to register with the main. The main server looks at gateway recovery rules
48
49 and decides when to allow the gateway to register back at the main. When all the gateways have finally left
50 the LSP, a maintenance process in the LSP sends an unregister message to groups of IP telephones, which
51 causes them to re-process their gatekeeper list and find the main server again. The return policy for gate-
52 ways connected to LSPs is administered on two forms, the system-parameters mg-recovery-rule form,
53
54
55
56
57
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 17 -

1 shown in figure , and the media-gateway form illustrated in figure 8. The LSP could also be reset manually
2
3
4 Figure 7. Change
5
system-parameters
6
7 mg-recovery-rule
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23 Figure 8. media-gateway
24
form
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39 to cause the return to main, but this would cause everything to return to main at once rather than metering
40 out the load.
41
42 In addition to the "return to main" differences, ESSs and LSPs are administered on different SAT forms and register
43 differently when registering with CM.
44
45 In order to support PE on ESS servers, something would need to be done about "return to main" differences between
46 ESS and LSP to prevent fragmentation and performance problems. The "right" solution would be to merge the
47 concepts of ESS and LSP into a single concept of survivable server which could support both port networks and
48 H.248 gateways and could fully utilize the PE interface. Doing this merge requires a number of changes in
49
50
administration, ESS/LSP registration, and recovery behavior. It would also require complete retesting of server
51 survivability. It is not something that could be done as a rush late adder to any release.
52
53 3.9 PE NIC and Address Assignment
54
Processor Ethernet (PE) is an interface within the Telephony Application (TA) of a Communication Manager (CM)
55
56 server. See figure 9. PE is assigned to a hardware NIC via the CM server web pages and to an IP address by the TA.
57 Within the TA, PE is known by the node name, procr. As the figure illustrates, PE is not the same interface as the
58 Packet Control Driver (PCD) interface which is used to talk to IPSIs, even if they might share use of the same
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 18 -

1 hardware NIC and have the same IP address. (They will use different port numbers.) The PE interface is also not the
2 same interface as is used by file synchronization, even though both might use the same NIC and have the same IP
3
4
address.
5
6
7
CM Server TA
8
9 PCD
10 PE
File
11 Sync
12
13
non-alias alias
14 Logical Interfaces
VLAN-x non-alias alias
15
Physical NIC Eth-x Eth-y
16
17
18
19 Figure 9. Relationship of PE to NICs, and IP Addresses
20
21 In current systems, the web interface allows the customer to assign the PE interface to either control network or to
22
the customer LAN. This assignment results in two variables being set in ecs.conf.
23
24 • PClanIntf0 is set to the identity of the NIC to be used. e.g. eth4
25
26 • PEInterface is set to a string to specify which logical interface (customer lan or control network) is assigned.
27 PClanIntf0 is read by the routine long_ipinfo and a license check for PE made in the routine allow_procr within CM.
28
29
This information is read as CM starts up and the node name procr assigned the appropriate IP address automatically.
30 PEInterface is used only by the web in order to be able to display the correct selection back to the user.
31
Currently, when procr is assigned an IP address within the TA on duplicated servers, it is assigned the server unique
32
33 IP address, not the alias address of the NIC. (This is done on duplicated ESS servers so they can register with the
34 main server.) On duplicated MAIN servers, procr needs to be assigned to the alias address of the NIC assigned to PE.
35
36
Both PClanIntf0 and PEInterface variables will be deleted from ecs.conf and PE information stored in servers.conf.
37 The arbiter needs to know the address not only of the near end for PE but for the other server as well.
38
Note that there is only one default gateway in the routing tables in a server. Usually the default gateway is assigned
39
40 to the customer LAN. If devices are connected to a LAN other than the LAN where the default gateway is located,
41 static routes are often needed to achieve the needed routing. (This is not new to these requirements -- just an
42 interesting but important configuration issue.)
43
44 Table 3 is an example of servers.conf content from an S8720 modified to show PE information added. Comments
45 in the actual1 file are not shown. The first column (use) and the title line are added here for readability, otherwise
46 the table shows the actual file content. Other Columns contain data as follows:
47
48 Name This column is the record identifier or function to which the other data applies
49
50
Eth This is the Ethernet interface number. i.e. 2 means Ethernet 2 as seen by the Linux
51 Operating system.
52 Host This is the host name on that interface
53
54 IP Address IP Address, Mask, and Default Gateway are the standard Ethernet parameters for the
55 interface. Note that the Default Gateway is set only on the "enterprise LAN".
56
57
58 1. This data was taken from a real switch and then PE entries added to illustrate how the data would appear. This example does not represent
59 a commitment to support any particular configuration, such as PE and port networks at the same time.

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 19 -

1 SNMP The SNMP GET and SET columns contain the SNMP community strings for GET and SET
2 operations on the Cajun switches that make up the control network. The original Stingray
3
4
web interface has/had a page where unused ports on the Cajun switches could be disabled
5 for better security. This was a convenience to the customer so that they didn’t have to access
6 the Cajun’s independently. Since Avaya no longer ships these switches and since customers
7 do not necessarily keep the control networks independent, this is a useless feature in newer
8 systems. Its not clear that SNMP messages sent to the Cajuns would work on non-Cajun
9
switches. The data in the SNMP column in the table would only appear in the line(s) for the
10
11 Ethernet switches. There is no data for the Ethernet switches in this example servers.conf
12 file.
13 Server ID/number Each server in "the system" is assigned a unique number in the range 1-256 which appears
14
in the last column as "server ID". Each duplicated server also has a number, either 1 or 2
15
16 appended to its name in the "name" column of the table. e.g. cna1 or cna2. If the state of
17 health of the duplicated servers is equal, the server whose names end in "1" will generally
18 be preferred to be the active server.
19
20
21
22
Table 3. Servers.conf on S8720
23 Default SNMP SNMP
24 Use Name Eth Host IP Address Mask Server ID
Gateway GET SET
25 CN-A active-cna 2 sv-st10-cna 198.152.254.200 255.255.255.0 NA NA NA NA
26 CN-A cna1 2 sv-st10-1-cna 198.152.254.201 255.255.255.0 NA NA NA NA
27 CN-A cna2 2 sv-st10-2-cna 198.152.254.202 255.255.255.0 NA NA NA NA
28 laptop laptop 1 NA 192.11.13.6 255.255.255.252 NA NA NA NA
29 dup dup1 0 sv-st10-1-dup 192.11.13.13 255.255.255.252 NA NA NA NA
30 dup dup2 0 sv-st10-2-dup 192.11.13.14 255.255.255.252 NA NA NA NA
31 CN-B active-cnb 3 sv-st10-cnb 198.152.255.200 255.255.255.0 NA NA NA NA
32 CN-B cnb1 3 sv-st10-1-cnb 198.152.255.201 255.255.255.0 NA NA NA NA
33 CN-B cnb2 3 sv-st10-2-cnb 198.152.255.202 255.255.255.0 NA NA NA NA
34 cust active 4 sv-st10 172.21.22.3 255.255.254.0 172.21.23.254 NA NA NA
35 cust server1 4 sv-st10-1 172.21.22.1 255.255.254.0 172.21.23.254 NA NA 96
36 cust server2 4 sv-st10-2 172.21.22.2 255.255.254.0 172.21.23.254 NA NA 97
37 PE active-pe 2 sv-st10-pe 198.152.254.200 255.255.255.0 NA NA NA NA
38 PE pe1 2 sv-st10-1-pe 198.152.254.201 255.255.255.0 NA NA NA 96
39 PE pe2 2 sv-st10-2-pe 198.152.254.202 255.255.255.0 NA NA NA 97
40 #SAMP/MPC
41
42 #Ethernet
43 Switches
44
#UPS
45
46
47 Note that there are entries in the file for both servers in the duplicated pair. The "active" entry refers to the alias
48 address shared by the two servers on that LAN. Note that the default gateway does not necessarily appear on the
49 same LAN as the NIC assigned to PE.
50
51 3.10 System Availability
52
53 The enhancements specified in these requirements are targeted specifically at TTS aware telephones and H.248
54 gateways connected to the PE interface. Recovery time for these devices is significantly improved. A configuration
55 with only H.248 gateways and no port networks is expected to support five 9’s availability in specific conditions. A
56
57
formal availability analysis is being done by Bahareh Momken and will be available in "PE Duplication Availability
58 Analysis, CM 5.2" CID 135123.
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 20 -

1 3.11 Call Center Availability


2
3 TTS aware telephones and H.248 gateways connected to the PE interface, are expected to recover quickly from a
4 server interchange. However, CMS/CCR/IQ declare link failure within 15 seconds and will re-pump its data base
5 (re-pump time can be 30 minutes). In the most optimistic case, interchange in less than 15 seconds is marginal at
6
best. Notice that this time is independent of whether the adjunct is connected to the PE or to a CLAN.
7
8 Ideally, CMS/CCR/IQ would be enhanced to either tolerate a longer link outage or have a mechanism to accurately
9 determine whether it needs to do a pump-up after the outage. The call center organization believes the pump-up is
10
11
not a problem for their customers and has declined modification of these adjuncts.
12
13
3.12 Scalability
14 These changes to do not affect scalability. Numbers of supported IP telephones or H.248 gateways is not changed.
15
16 3.13 File Sync
17
18 Assignment of the PE interface is done on a server by server basis and is not part of the data that is file synchronized.
19
20 3.14 Backup/Restore
21
22 There is no direct impact on backup and restore from the same release.
23
24 3.15 Upgrades and Restore from an Earlier Release
25
The upgrade processing described here is to keep the data files consistent in the new release. There is no real upgrade
26
27 of the feature "PE on duplicated main servers" because the feature doesn’t exist prior to this release. If a duplicated
28 main server is upgraded, port networks will be present; following the upgrade, the port networks will work as before,
29 and PE will be disabled.
30
31 The following description applies to upgrade or restore from a release prior to these changes. On an upgrade, files
32 are copied from the "other" partition and on a restore, files are coped from a backup. Other than the source of data,
33 the following operational behavior is common:
34
35 Upgrade processing for PE information depends on the role of server begin upgraded.
36
37 3.15.1 Duplicated Main Servers
38 Processor Ethernet was not supported on duplicated main servers in prior releases. The upgrade/restore process will
39
40
need to take the following actions with respect to PE data. These steps get ecs.conf and servers.conf in the proper
41 format for the new release and leave PE unassigned.
42
1. restore/copy servers.conf from the backup/other-partition
43
44 2. restore/copy ecs.conf
45 3. create a new variable, PEsohDeviceIP, in ecs.conf and assign it the value 0.0.0.0
46
47 4. delete the variables PClanIntf0 and PEInterface from the restored/copied ecs.conf
48
49 3.15.2 Simplex Main/ESS/LSP Servers and Duplicated ESS Servers
50 PE will always be assigned.
51
52 1. restore/copy servers.conf from the backup/other partition
53
54
2. restore/copy ecs.conf
55 3. create a new variable, PEsohDeviceIP, in ecs.conf and set its value to 0.0.0.0
56
4. add lines to the restored/copied servers.conf based on the content of ecs.conf/PClanIntf0 in the
57
58 backup/other-partition data.
59 • use PClanintf0 to identify the NIC

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 21 -

1 • Use this NIC value to replicate entries in servers.conf to create the PE entries, except do not create
2 an alias entry for PE even if the NIC already has one.
3
4 • Copy the server ID entries from the Cust entry in servers.conf to the PE entries
5 5. delete the variables PClanIntf0 and PEInterface from the restored/copied ecs.conf
6
7 3.16 Operation on ESS/LSP with Main Server on an Older Release
8
9 Operation and administration of the PE interface is server specific. There is no impact to having the main server
10 running a different release from the LSP server. If an ESS is in the configuration (not recommended) if a fail-over
11 to an ESS occurs, devices connected to the PE interface on the main will need to connect to a CLAN on the ESS.
12
13 3.17 System Impacts
14
15 • memory:
16
• real time:
17
18 • bugfix:
19
20 4. Hardware Requirements
21
22 None.
23
24 5. H.248 Gateway Requirements
25
26 <134539-1> Which Gateways
27 The following H.248 gateways shall be modified as specified in these requirements
28 • G450
29
• G350
30
31 • G250
32 • G700
33 • IG550
34
<134539-2> Accept Incoming Socket
35
36 H.248 gateways specified in requirement 1 shall listen on TCP port 1167 for an incoming
37 connection from CM, and shall accept this connection only under the following
38 circumstances:
39 • The incoming connection originates from an IP address that
40 • the gateway thinks it already has a connection to, or
41
• the gateway is currently not connect to a server and the IP address was where
42
the gateway was last connected.
43
44 If the incoming connection does not satisfy the above criteria, the gateway shall respond with
45 a TCP FIN message.
46 Note that only one listening port is used for either encrypted or non-encrypted sessions. Both ends already know
47 whether the link will or will not be encrypted because there had to be a prior connection.
48
49 <134539-3> Accept Response
50 If the gateway accepts an incoming connection per requirement 2, the gateway shall respond
51 as follows:
52 1. If the gateway thinks it has an existing connection to this IP address, it shall close
53 that connection.
54
55 2. If the gateway’s prior connection used pTLS, the gateway shall initiate a new pTLS
56 connection on the new socket.
57 If pTLS was not in use, then the gateway will begin using the TCP connection immediately. However, it may
58 take CM a couple of seconds to be able to begin to respond to messages. If the gateway has lots of activity,
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 22 -

1 messages may be lost.


2
3. The gateway shall use the new connection as a replacement for the old one.
3
4 4. The gateway shall not re-register.
5 5. The gateway shall not reset just due to the new connection arriving; both calls and
6
connections shall be preserved from the gateway’s perspective.
7
8 Note that establishment of a new connection will reset packet sequence numbers and packets may be lost in
9 either direction.
10
See related CM behavior beginning requirement 26 on page 25.
11
12
13
6. CM Software/Firmware Requirements
14 Note: the requirements define changes to provide faster recovery from a subset of all the devices that might be
15
connected to a CM server. Devices which do no receive this new treatment will still recover as they do now, but not
16
17 as quickly as devices receiving the new treatment.
18 <134539-4> SA9009
19
20 SA9009 shall be merged into the base as a normal system feature (ungreen); requirements in
21 this document shall take precedence if differences are found in SA9009 implementation and
22 these requirements.
23 It is important to not reference FEAT_DUP_PE_SIP_CFF in the code because this license bit could be removed
24 or re-used for something else in a future release.
25 <134539-5> FEAT_PRETH License Bit
26
27 Software on all CM servers in all configurations shall operate as if the FEAT_PRETH bit is set
28 to "on" regardless of its setting in the license file. Ideally, checks on the FEAT_PRETH bit
29 would be removed from the code entirely; however it is acceptable for the code to force the
30 bit to appear enabled internally regardless of its setting in the license.
31 <134539-6> H.248 Gateway License Support
32
CM shall be able to use both IPSIs and H.248 gateways connected to PE on duplicated servers
33
as a license host. i.e. serial number, etc.
34
35 See routine "ls_gw_addr".
36 <134539-7> PClanIntf0 and PEInterface
37
The /etc/opt/ecs/ecs.conf variables PClanIntf0 and PEInterface shall be deleted.
38
39 <134539-8> PE on Duplicated Main Servers Requires An Alias Address
40 PE on duplicated main servers shall be assigned the alias address of the NIC to which PE is
41 assigned. If an alias address is not administered, PE shall be disabled.
42
43 This is a requirement on how procr is assigned based on content of servers.conf.
44 <134539-9> PE on Duplicated ESS Servers
45 PE on duplicated ESS servers shall be assigned the server unique address of the NIC
46 assigned to PE and never an alias address. See also requirement 53 on page 32.
47
48 This is basically a duplicate of the referenced requirement but is here to emphasize what is new in this release.
49 <134539-10> VoIp Resource
50
CM servers shall not accept new registrations from endpoints unless the server has a VoIP
51
resource already registered. This requirement shall not inhibit recovery of sockets on a server
52
becoming active as a result of an interchange. i.e. such sockets are recovered regardless of
53
whether a VoIP resource is present or not.
54
55 An H.248 gateway must register before it is available to provide a VoIP resource. This "should" be boiler plate.
56 i.e. this should be how it already operates, but in case there is some missing check when there is no port
57 network....
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 23 -

1 <134539-11> IP Takeover
2
Only the active server shall listen on or otherwise use the PE interface on duplicated main
3
servers. Control of the PE interface shall transition to the new active server during a processor
4
interchange.
5
6 If the two servers are operational and if the two servers can communicate, the arbiter already has an algorithm
7 to drop one server to standby in case both are active.
8 There are failure conditions (e.g. dual active) in which the PE interface may become non-functional due to two
9 servers attempting to use the same IP address with different MAC addresses. The system will be out of service
10 as long as this condition persists.
11
<134539-12> Flooding Prevention
12
13 Communication Manager already uses socket level flow control mechanisms. The PE
14 interface shall be tested to verify that these flow control mechanisms operate properly.
15 See also Enhanced Firewall Administration, CID 122694 for use of the sklimit module.
16
17 <134539-13> PE Capacity
18 The Processor Ethernet interface shall be capable of supporting the full capacity of the server
19 (numbers of active devices) for all permitted IP devices connected to the PE interface
20 simultaneously. The BHCC of the server may be reduced; The call capacity shall be measured
21 at 65% processor occupancy using comparable call mixes used to determine BHCC for other
22 servers. The measured value shall be documented as the BHCC of the server.
23 <134539-14> Server Capacity
24
25 The total server capacity to support IP devices (numbers of active devices) shall not be
26 reduced by use of the Processor Ethernet interface alone or in combination with CLANs.
27 <134539-15> PE and CLAN Mix (testable=n)
28
Implementation shall not preclude both the PE and the CLAN interfaces being used at the
29
same time for any combination of devices provided that the published overall quantity and mix
30
of devices permitted for the server is not exceeded. Note that five 9’s availability may not be
31
supported with port networks in use.
32
33 i.e. you can use PE and CLANs in any mix but you don’t get any increased capacity.
34 A mixed configuration of PE and port networks will not be tested in this release.
35
<134539-16> Supported Adjuncts
36
37 The following types of adjuncts supporting IP connectivity shall be supported on the PE
38 interface. Note that this requirement says these devices can operate on the PE interface; it
39 does not prescribe interchange link bounce behavior which varies greatly among these
40 devices.
41 • CDR
42
• Messaging (all flavors that support IP connectivity)
43
44 • AES
45 • CMS/CCR/IQ
46 • SES
47 Note that BCMS uses a SAT interface, not the PE interface. The SAT interface does not use PE even when
48 connecting to port 5022.
49
50 <134539-17> System Initialization
51 Use of the Processor Ethernet shall not increase the time to service by more than 10% as
52 compared to a system with the same configuration of devices connected via CLAN. This
53 requirement shall apply in three cases:
54
• The time for server boot to full service
55
56 • The time for a Communication Manager application-only full restart to full service
57 • The time for an individual IP device to connect and achieve full service.
58 Full service is defined to mean the ability of all administered stations to receive an incoming
59 call or originate an outgoing call.

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 24 -

1 Note that this requirement does NOT cover processor interchange.


2
<134539-18> PE and Processor Interchange
3
4 Use of the processor ethernet on duplicated main servers shall be possible with either
5 hardware or software duplication. The advertised capacity of the servers shall not be reduced
6 due to use of the PE interface as compared to use of CLAN only. (This is a capacity of phones,
7 trunks, BHCC, etc. This is not necessarily internal capacity such as processor occupancy
8 which could be higher at the same supported capacity.)
9 Design of specific data being shadowed for PE needs to be examined to ensure this works properly.
10
<134539-19> Delay of Normal Operation
11
12 On a server interchange with the standby refreshed, "Normal Operation" shall be delayed until
13 after the gip has attempted to open all sockets per requirement 24 on page 25.
14 <134539-20> SUSER Connection Types
15
16 The suser process shall associate a type with each D-socket connection it establishes to the
17 gip process for devices connected to the PE interface and shall share this type with the gip.
18 e.g. setsockopt. Types shall include the following:
19 • Type 0: This connection is to a device, not otherwise identified by a unique non-zero
20 type, that will not receive special recovery processing on server interchange.
21 • Type 1: This connection is to an H.248 gateway that will accept an incoming connection
22 from the server to replace any existing connection from a gateway known to the device.
23 • Type 2: This connection is to a TTS aware IP telephone that identifies itself (protocol bit
24 in the GRQ message TTSFeatureSet) as being able to respond to the new recovery strat-
25 egy specified herein.
26
• Type 3: This connection is to a SIP trunk
27
28 • Type 4: This connection is to an H.323 trunk
29 The nomenclature, Type 0, Type 1, etc, is used in this document to enable discussion of the
30 concept; these terms do not imply an implementation, numeric or string value set.
31
<134539-21> Socket Port Numbers and Firewall Values
32
33 When the suser process informs the gip of socket type (type 1 and 2), the suser process shall
34 also inform the gip of the port number and firewall rate limit values to use if/when these
35 sockets are re-created as part of server interchange. Rate limits are only required for H.248
36 sockets. See also ecs.conf MGRateLimit.
37 The same port number can be used for all phones. The gateways normally originate the connection and listen
38 on a specific port (which may be hard coded in the gip).
39
A rule like the following is established on the H.248 listen port:
40
41 ACCEPT tcp -- anywhere anywhere tcp dpt:h248message sklimit: avg 50/sec burst 100 port 2945
42 When CM "connects" the incoming socket to port 2945, the connection is moved to a different port, N. The
43 sklimit associated with port 2945 is inherited by the connection moved to port N. When CM connects out to a
44 gateway, the same inheritance does not occur. Thus CM needs to establish the rate limit "manually".
45 A similar processes is currently used for the H.323 port:
46
ACCEPT tcp -- anywhere anywhere tcp dpt:h323hostcall flags:FIN,SYN,RST,PSH,ACK,URG/SYN acl port: 1720listener
47 LOG tcp -- anywhere anywhere tcp dpt:h323hostcall flags:FIN,SYN,RST,PSH,ACK,URG/SYN limit: avg 5/min burst 5 LOG
48 level warning
49 In the current CM implementation, when CM connects out to TTS phones, CM does not establish either the ACL
50 nor the limit values for these sockets. But since this is only for logging, it only really affects debugging at some
51 level.
52
53 <134539-22> SUSER Interchange Behavior
54 Suser in the server transitioning from standby to active on a server interchange shall not close
55 D-sockets for type 1 and 2 connections. Processing of connections for other types of sockets
56 shall not be modified.
57 <134539-23> GIP Connection Types
58
59 The gip process shall retain sufficient connection/socket type information received from the

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 25 -

1 suser to be able to satisfy requirement 24. The gip shall consider socket type to be type 0
2 unless told otherwise.
3
The gip will need to distinguish between type 1 and type 2 connections because they use different ports when
4
creating the new socket. The gip does not presently need to remember the details of the other types. i.e it needs
5
to know type 1, type 2 and everybody else.
6
7 <134539-24> GIP Interchange Behavior
8 In the server transitioning from standby to active, if the server interchange is
9
• in a server that is fully refreshed and
10
11 • the startup is a warm start,
12 before the system enters normal op, the gip process shall:
13
• Automatically create a new Linux socket for each type 1 and 2 connection and replace
14
any old socket information it may have with the new socket information (e.g. file/socket
15
descriptor).
16
17 • Automatically set socket option values, per information from the suser process, for sock-
18 ets it opens
19 • If creation of the new Linux socket fails, the GIP remember this failure and shall
20 close/drop the D-socket after entering normal op.
21 • The gip shall retain control during its initialization { init() } until one attempt is made to
22 open all type 1 and type 2 sockets.
23 The last bullet is the mechanism to prevent the system from entering normal op until the gip has done its work.
24
25 Initialization processing in the gip for sockets other than types 1 and 2 is not altered by this
26 requirement.
27 <134539-25> Socket Shadowing
28
Data tables in the suser and gip processes related to connections to external IP devices
29
connected to the PE interface shall be shadowed. Sufficient data shall be shadowed to
30
support the recovery action defined herein; the connection type shall be included in the
31
information being shadowed.
32
33 <134539-26> H.248 Gateway Session Restore
34 Sockets to H.248 gateways connected to PE will be established by the gip before normal op.
35
After entering normal op, if the gateway is using an encrypted link, the gateway shall
36
re-establish the pTLS session.
37
38 If the gateway is using an unencrypted link, the suser process shall resume operation without
39 additional session resumption messages to the gateway. i.e. there will likely already be
40 messages in queue from the gateway.
41 Ideally, if the session is encrypted, suser would send a special test message to the gateway using the current
42 encryption. If the gateway understands this message it responds to the suser and the session continues without
43 a key change. This would only happen if no packets were lost during the interchange. If the gateway does not
44 understand the special message then some packets were lost and the gateway signals to create new keys.
45
46 <134539-27> H.248 Registration Preserved
47 H.248 gateway registration shall be preserved across a warm-start server-interchange.
48 <134539-28> H.248 Gateway Throttling
49
50 CM shall not artificially limit the speed at which gateways return to service following a
51 warm-start server-interchange. Specifically, CM shall assume that state information is
52 preserved across an interchange and shall not require audit of this data prior to allowing the
53 gateway to return to service.
54 <134539-29> Upgrade/Restore of Duplicated Main Servers
55
This requirement applies to upgrade or restore from a release prior to these changes. On an
56
upgrade, files are copied from the "other" partition and on a restore, files are coped from a
57
backup. Other than the source of data, the following operational requirements are common:
58
59 • restore/copy servers.conf from the backup/other partition

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 26 -

1 • restore/copy ecs.conf from backup/other partition


2 • create a new variable, PEsohDeviceIP, in ecs.conf and assign it the value 0.0.0.0
3
• delete the variables PClanIntf0 and PEInterface from the restored/copied ecs.conf
4
5 Note that immediately following an upgrade from a prior release, PE is not usable because it was not usable in
6 the earlier release.
7 <134539-30> Upgrade/Restore of Simplex Main/ESS/LSP Servers and Duplicated ESS Servers
8
This requirement applies to upgrade or restore from a release prior to these changes. On an
9
upgrade, files are copied from the "other" partition and on a restore, files are coped from a
10
backup. Other than the source of data, the following operational requirements are common:
11
12 • restore/copy servers.conf from the backup/other partition
13 • restore/copy ecs.conf from backup/other partition
14 • create a new variable, PEsohDeviceIP, in ecs.conf and set its value to 0.0.0.0
15
• add lines to the restored/copied servers.conf based on the content of
16
ecs.conf/PClanIntf0 in the backup/other-partition data.
17
18 • use PClanintf0 to identify the NIC
19 • Use this NIC value to replicate entries in servers.conf to create the PE entries, except
20 do not create an alias entry for PE even if the NIC already has one.
21 • Copy the server ID entries from the Cust entry in servers.conf to the PE entries
22 • delete the variables PClanIntf0 and PEInterface from the restored/copied ecs.conf
23
<134539-31> SOH with No IPSIs
24
25 The arbiter shall ignore IPSIs in the state of health calculation when no IPSIs are administered.
26 <134539-32> PE SOH Effect on Interchange
27
28 If the SOH of the NIC assigned to the PE interface on duplicated servers is not the same, an
29 interchange shall be done to the server with the better PE SOH. Note that this is without
30 consideration of IPSI connectivity.
31 <134539-33> PE SOH Determination
32 The state of health of the PE interface shall be monitored on duplicated main servers only, and
33 shall be determined by analysis of the following:
34
35 • Responses to ARP ping messages sent out the NIC associated with the PE interface to
36 an external device (PING device) whose IP address is administered on the server’s web
37 pages. The recommended device is the subnet default gateway router.
38 • Responses to ARP ping messages sent out the NIC assigned to the PE interface to the
39 PE interface on the other server.
40 • The SOH for the PE shall be lowered if both ping targets fail to respond.
41
<134539-34> Exchange of PE SOH
42
43 Servers shall include PE SOH status in SOH information exchanged.
44 <134539-35> PING Device Unreachable
45
The network device being used to assist in PE state of health determination (PING device)
46
shall be ARP ping’d at a rate of once every 3 seconds.
47
48 If availability analysis later indicates that this rate must be faster than 3 seconds, the rate shall
49 be increased as needed within the range from three times every 2 seconds to once every 3
50 seconds. The PING code shall be modified/incorporated into the application if necessary to
51 satisfy the rate requirement.
52 The device shall be considered unreachable if 2 consecutive responses are
53 missed/not-received. The response to a PING shall be considered missed/absent if not
54 received prior to the time to send the next ARP ping.
55
56 The interval is not administrable; however it is desirable that the value be stored in ecs.conf
57 such that it could be modified in the field if needed or made administrable in the future if
58 desired.
59 The original version of this requirement required a 1 second rate, but the PING code does not operate reliably

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 27 -

1 at that rate. A custom version of the PING code is required to achieve a 1 second PING rate.
2
<134539-36> PING Priority
3
4 Tasks which monitor the state of health of the interface used for PE shall operate with a
5 priority at least as high as the PCD.
6 Otherwise, the SOH of the PE may not be determined in a timely manner, especially on a busy switch.
7
<134539-37> PE SOH Alarm
8
9 Alarms shall be generated for SOH of the PE interface as follows:
10 • If there is no response to the PE SOH Ping message from the PE SOH device a minor
11 alarm shall be generated.
12
• If there is no response to the PE SOH Ping message from the other server, a minor alarm
13
shall be generated.
14
15 • If there is no response from the SOH device and there is also no response from the other
16 server a 3rd alarm shall be generated.
17 • If the PE priority is NOT ignore, the 3rd alarm shall be a major alarm.
18 • If the PE priority is ignore, the 3rd alarm shall be a minor alarm.
19 Alarms shall be generated within 10 seconds after the failure condition occurs and resolved
20 within 10 seconds after the associated test passes.
21
22 <134539-38> PE SOH in Server Command output
23 A new line, labeled "Processor Ethernet:", shall be added to the server command output and
24 shall display the SOH of the PE interface for each server. Values shall be "up" or "down" or
25 "unused". A blank value shall be used to indicate status is unknown and may appear during
26 various transient states while status is being determined. This line shall appear only on duplex
27 main servers. See figure 10.
28
29 SERVER STATUS
30 Cluster ID: 001
31 Duplication: sw
32 Standby Busied? no
Standby Refreshed? yes
33
Standby Shadowing: on
34 Duplication Link: up
35 Elapsed Time since Init/Interchange: 11d 02:30:53
36
37 sv-st10-2 sv-st10-1
38
39 ID: 002 (2) ID: 001 (1)
40 Mode: Active Mode: Standby
41 Major Alarms: yes Major Alarms: no
42 Minor Alarms: yes Minor Alarms: no
Control Network: 0 / 0 / 0 Control Network: 0 / 0 / 0
43
44 Processor Ethernet: up Processor Ethernet: down
45 Server Hardware: okay Server Hardware: okay
46 Processes: okay Processes: okay
47
48
49
50 Figure 10. Enhanced Server Command Output
51
52
53
54
55 <134539-39> Arbiter Interfaces
56
The arbiter shall use all available NICs when attempting to contact its peer on the other server.
57
The priority order shall be:
58
59 1. The NIC assigned to the duplication interface

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 28 -

1 2. The NIC assigned to Control Network A


2
3. The NIC assigned to Control Network B
3
4 4. The NIC assigned to the Customer LAN
5
5. The NIC assigned to the Processor Ethernet (Note that generally PE is assigned to
6
7 an interface supporting one of the other functions in this list. If the feature known as
8 flexible NIC assignment ever gets implemented, there could be only a duplication
9 NIC and a NIC for PE, however.)
10
11 6.1 SAT Form Changes
12
13 7. Integrated WEB Tools Impact
14
15 The web interface is the target of a number of outstanding/queued work items including,
16
17 • Conversion of the web interface implementation from C++ to PHP,
18 • Server/software generalization,
19
20 • Flexible NIC Assignment (CID 126798), and
21 • Enhanced Firewall support (CID 122694).
22
23 If any of these work items appear in a work plan prior to or in the same release as the changes specified herein, the
24 details for web changes specified herein will need to be revisited. The following requirements attempt to minimize
25 the amount of re-work when the above projects are implemented.
26
27 Figure 11, illustrates the current "set identities" web page in the configure server wizard for an S8500 server. A
28 simplex example is shown here because it illustrates the current mechanism for PE assignment. PE assignment is
29 not presented on web pages for duplicated servers, currently. The first change that needs to be made to this page is
30
to convert the syntax of the assignment of PE from assignment to a functional LAN (control network, customer
31
32 LAN) to an assignment of ethernet interface (eth0, eth1). This change is illustrated in figure 12.
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 29 -

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Figure 11. Existing PE Assignment on S8500
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55 Figure 12. New Set Identities Web Page
56
57 There is an additional problem, however. The assignment of the PE interface has a dependency on the role of the
58
59
processor -- whether it is an LSP, ESS or main server. Specifically, the PE interface MUST be assigned on an LSP

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 30 -

1 and on an ESS (although its use is restricted here), but PE assignment is optional on a duplicated main server. The
2 way these pages are arranged, the "set identities" page, where PE assignment is done, occurs before the server role
3
4
is administered (Configure ESS/LSP). The solution that is the simplest to make is to move the "configure ESS/LSP"
5 page to be the first page following the "review notices" page; that is, before the "set identities" page. A more
6 customer friendly change would be to split the "configure ess/lsp" page into two pages as defined in figures 18 and
7 21 of CID 126798. This would minimize later additional churn in customer visible operation of the web pages.
8
9 In addition to the changes described above, assignment of the PE interface on duplicated main servers, requires
10 assignment of an external device to PING in order to determine PE state of health. This PING address defaults to
11 the address of the default gateway which is entered on the "configure interfaces" page in the configure server wizard.
12 Thus the fields are added to that page so entries can be defaulted and checked. See figure 13.
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35 Figure 13. PE Additional Parameters
36
37
38
39 The field the IP address of the PING device appears only for duplicated servers (easy) and only for MAIN
40 servers (harder). This is another reason for a move of the "configure ESS/LSP" page.
41
42 <134539-40> CNA CNB Optional
43 It must be possible to not assign CNA and CNB on main servers. i.e. no control network
44 assigned. In configurations with no port networks, these functions have no purpose.
45
46 This is another reason to move the LSP/ESS decision very early in the configure server web pages. Note also
47 that assignment of the Enterprise LAN to a real NIC is still required. It may be the same NIC as used by the PE
48 interface or not.
49 <134539-41> Configure ESS/LSP
50
(preferred) The configure ess/lsp web page shall be split in two as defined in CID 126798
51
figures 18 and 21. The figure 18 screen shall appear prior to the set identities page and the
52
figure 21 screen shall replace the current configure ess/lsp page.
53
54 (acceptable) The existing configure ess/lsp page shall be moved to appear prior to the set
55 identities page.
56 Either solution is acceptable, but splitting the page now reduces later change for the customer.
57
<134539-42> PE Assignment to NIC
58
59 The semantics for assigning the PE interface in web administration shall be changed from a

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 31 -

1 selection of possible functional LAN types (CNA, CNB, customer LAN) to a selection of
2 possible Ethernet interfaces (eth0, eth1, eth2, eth3, eth4, eth5) as illustrated in figure 12. The
3 PE interface may be assigned to any externally facing server NIC on the server except the
4 ones assigned to the duplication link or the laptop link. An external facing NIC is one which is
5 intended for external customer connections as compared to a NIC which is used internally to
6 connect to things like the SAMP card. See remaining requirements for additional constraints
7
Currently, the PE interface is assigned by the user to a functional interface (control Lan, Enterprise Lan).
8
9 <134539-43> PE Defaults
10 The web page that administers PE to a NIC shall present current assignments when the page
11 is displayed. On a server that has not been previously administered, the values shall be as
12 follows: For simplex servers, the default shall be for the same NIC as for the Customer LAN.
13 For duplex servers, the default shall be "unset".
14
<134539-44> PE Address Data
15
16 New lines shall be created in servers.conf to identify the PE NIC and IP address information
17 as illustrated in table 3 on page 19. These lines shall be created even if they replicate data for
18 another interface. e.g. customer LAN. Also, the server IDs, at the end of the lines for the
19 customer LAN entry shall be copied to the PE entry. On duplicated main servers, an alias
20 address must be assigned.
21 <134539-45> PE Change Notice
22
23 If data for the PE interface is changed via the web interface, CM is already notified to re-read
24 the PE data. This notification shall be extended to include notification of the software that is
25 monitoring PE health.
26 <134539-46> NIC Labels
27 The set identities web page shall display the names of all the functions assigned to a NIC when
28 more than one function is assigned to a NIC. e.g. PE and customer LAN assigned to same NIC.
29
30 See figure 13 for example labeling of PE and Customer LAN on the same NIC.
31 <134539-47> PE Health Check
32
The web interface shall support assignment of an IP address for an external device to be
33
PING’d as part of the health check of the PE interface. The IP address entered shall:
34
35 • be a valid IPV4 address
36 • be on the same subnet as the PE interface
37 • not be an IP address terminating on either server
38 • be saved in the variable PEsohDeviceIP in the file /etc/opt/ecs/ecs.conf.
39
40 The field shall appear only for duplicated MAIN servers, and only if the PE is assigned to a NIC;
41 when presented, the field is a required entry.
42 The help for the screen where this assignment is made shall recommend use of the default
43 gateway for the subnet where the PE resides, although it is not required.
44
Note: It is acceptable for the user to specify a telephony gateway as the ping device so long as it is on the same
45
subnet as the PE interface.
46
47 <134539-48> PE Administration by The User
48 The assignment of the PE to a NIC shall be user administrable on S8400, S8500 and S87xx
49 servers. On S8300, assignment shall be automatic to the non-laptop interface.
50
This requirement may be generalized: If a server has only two ethernet interfaces (laptop + one
51
other) then assignment is automatic to the non-laptop interface. If a server has more than two
52
ethernet interfaces, then the user may assign the PE to any one of the external facing
53
interfaces except the laptop interface or duplication interface.
54
55 Some servers use ethernet interfaces for internal communication to devices such as the SAMP. PE cannot be
56 assigned to an internal interface.
57 Note that the ability to NOT assign the PE interface is new to this release. On duplicated MAIN servers, if an
58 alias is not assigned, then the PE may not be used. This does mean that use of PE on duplicated MAIN servers
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 32 -

1 requires the servers to be on the same sub-net. In the normal case this means co-located, although there are
2 schemes for extending sub-nets over distance.
3
<134539-49> S8300 PE Assignment and Use
4
5 Assignment of the PE interface on the S8300 shall not be administrable. The PE shall be
6 assigned to interface eth1 per requirement 48 except that an alias is never assigned on an
7 S8300. The configure server web page shall display the assignment "read only". THIS
8 REQUIREMENT SIMPLY DOCUMENTS THAT THERE IS NO CHANGE FOR THIS SERVER
9 TYPE.
10 <134539-50> S8400 PE Assignment and Use
11
Assignment of the PE interface on the S8400 shall administrable to interfaces,
12
13 • unused, eth0, eth2 or eth3
14 per requirement 48 except that an alias is never assigned on an S8400.
15
16 <134539-51> Xen Server PE Assignment and Use
17 Assignment of the PE interface on the Xen server shall administrable to interfaces,
18 • unused, eth0 or eth2
19
20 per requirement 48 except that an alias is never assigned on a Xen server.
21 <134539-52> S8500 PE Assignment and Use
22
Assignment of the PE interface on the S8500 shall be administrable to interfaces "unset" or to
23
any non-laptop external facing NIC on the S8500 per requirement 48 except that an alias is
24
never assigned on an S8500.
25
26 <134539-53> S87xx Duplicated Server PE Assignment and Use
27 Assignment of the PE interface on the S87xx servers shall be administrable to interfaces as
28 defined in the following:
29
S87xx Main: The PE interface may be assigned to "unset’ or to any external facing NIC on
30
the S87xx server except the laptop or duplication NICs.
31
32 The procr interface is disabled on duplicated main servers unless an alias
33 address is assigned.
34 When enabled on a duplicated server, the procr interface shall be usable for
35 the following purposes:
36
37 • Incoming registrations from an ESS or LSP
38 • H.323 connections
39 • H.248 connections
40 • SIP Trunks
41 • IP based adjunct connections
42
43 S87xx LSP: LSPs are always simplex servers. Duplicated servers may not be configured
44
as LSPs. (This is not changed from CM4.0)
45
46 S87xx ESS: On an S87xx ESS server, the PE interface must be assigned to one of the
47 external facing ethernet interfaces.
48 Procr shall be assigned to a server unique IP address for the NIC assigned
49 to PE.
50
On an S87xx ESS server the procr interface shall be usable for the following
51
purposes: (This is not changed from CM4.0)
52
53 • Outgoing registrations to the main
54 <134539-54> Reset to Defaults
55
Reset to defaults on all servers shall mark the PE assignment unset.
56
57 The current S8300 reset to defaults web page sets the PE assignment to "unset". The configure server web pages
58 then ensure that the PClanIntf0 value is set properly either via fixed assignment (e.g. S8300 ) or via customer
59 administration.

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 33 -

1 8. Firewall
2
3 There is a major revision to the firewall interface in queue as work item 8674 and defined in Compas ID 122694. If
4 this work is done, the following general rules applied to the INPUT chain of all interfaces would eliminate any
5 special work for the PE interface.
6
7 ACCEPT tcp -- any any anywhere anywhere tcp flags:FIN,SYN,RST,PSH,ACK,URG/SYN listener
ACCEPT udp -- any any anywhere anywhere listener
8 ACCEPT tcp -- any any anywhere anywhere state RELATED,ESTABLISHED
9
10 The "listener" module causes a port to be opened whenever there is a listen socket open on the port. Ports are open
11 dynamically as CM opens listen sockets without having to establish unique rules per port or interface.
12
13 Adding these general rules without the other changes specified in CID122694 won’t work without additional
14 modifications not related to PE support. For example, the "server access" web page manipulates the firewall and that
15 page would have to be modified as well. Instead, the specific ports needed on the PE interface must be handled
16 individually.
17
18 <134539-55> Firewall INPUT Chain
19 Firewall ports in the INPUT CHAIN, needed by the PE interface on duplicated main servers,
20 shall be opened on the NIC to which the PE is assigned for those ports shown in tables 4 and
21 5 with "yes" in the PE column. PE assignment on other server types shall not be altered.
22
23 Ports shall be opened on the INPUT chain using a "listener" module.
24 Note that typical administration ports (e.g. ssh) are not opened on this interface. The customer LAN is still
25 required to be administered and may or may not be on the same Ethernet interface as the PE.
26
27
28
29 Table 4. "Standard" Services
30
PE Service Port Protocol Comment
31
32 yes echo/ping 8 icmp
33 ftp 21 tcp File transfer. Disabled by default
34 ssh 22 tcp Used for server administration and SAT access.
35 telnet 23 tcp Used for server administration and SAT access. Dae-
36 mon disabled by default
37 DNS 53 udp Domain Name Service
38 bootps 67 tcp/udp Used when IPSIs configured for DHCP
39 bootpc 68 tcp/udp Used when IPSIs configured for DHCP
40 yes tftp 69 udp Used in some configurations for Announcement fetch
41 by gateways
42 http 80 tcp Server administration home page -- redirects to https.
43
sunrpc 111 tcp/udp Used by IA770
44
yes ntp 123 udp CM servers act as client to network time server and
45
also act as source to gateways
46
snmp 161 udp SNMP administration access
47
48 snmp traps 162 udp CM receives some traps from Gateways and UPS
units.
49
50 ldap 389 tcp Used by IA770 and by LDAP based Login processing
51 https 443 tcp Server administration via the web
52 syslog 514 udp CM servers may be configured to send logs to a cen-
53 tral server. CM servers may also receive logs from
54 gateways.
55 ldaps 636 tcp Used by LDAP based Login processing
56 radius 1812 udp Used by RADIUS based Login processing
57 radius accounting 1813 udp Used by RADIUS based Login processing
58 ssh 2222 tcp High Priority ssh. Used to gain control of server by
59 Avaya services in cases of high server occupancy

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 34 -

1 Table 4. "Standard" Services


2
3 PE Service Port Protocol Comment
4 SafeWord 5030 tcp Used by Secure Computing Corporation SafeWord
5 based Login Processing. The default port is shown.
6 Each installation could choose a different port.
7 securID 5500 udp Used by RSA Corporation SecurID based Login Pro-
8 cessing. The default port is shown. Each installation
9 could choose a different port.
10
11 Table 5. CM Specific Services
12
13 PE Service Port Protocol USE
14 samp-http 80 tcp Used between CM server and SAMP
15 samp-https 80 tcp Used between CM server and SAMP
16
http-ipphone 81 tcp IP phone firmware download; ports opened/closed by
17 web pages based on use.
18
https-ipphone 411 tcp Secure IP phone firmware download; ports
19 opened/closed by web pages based on use.
20
vphone 1037 tcp Used for older "double connect" IP phones
21
yes encrypted-h248 1039 tcp Encrypted links to gateways
22
23 samp 1234 tcp Used between CM server and SAMP
24 vaa 1320 tcp Used by IA770
25 arbiter 1332 udp duplicated systems only; Note this port must be opened
26 on the PE interface in PE-only systems.
27 arbiter 1333 udp duplicated systems only; Note this port must be opened
28 on the PE interface in PE-only systems.
29 yes h323gatestat 1719 udp ESS registration. In on Main servers, out on ESS
30 yes h323hostcall 1720 tcp IP phone call control link
31 ipsi-cmds 1956 tcp resetipsi, loadipsi, telnetenable commands for IPSI
32 yes h248message 2945 tcp Non-encrypted link to gateways
33 pcd-ipsi 5010 tcp Interface between CM and the IPSI for call processing
34 ipsi version 5011 tcp IPSI version port
35 ipsilic 5012 tcp IPSI license port
36
secure-sat 5022 tcp direct SAT access using ssh
37
def-sat 5023 tcp direct SAT access using telnet
38
39 yes sip 5060 tcp public sips
40 yes sip-tls 5061 tcp secret sips
41 dupmgr-swdup 5098 tcp software duplication
42 licsvr 5423 tcp should not be open on firewall
43 Enterprise wide 5424 tcp should not be open on firewall as EWL has not been
44 Licensing adopted by CM.
45 audix 5500 tcp Used by IA770; see also securID in previous table
46 AE Services 8765 tcp Application Enablement Service
47 samp-ssh 10022 tcp Used between CM server and SAMP
48 dupmgr 12080 tcp duplicated servers only
49 samp 19121 udp Used between CM server and SAMP
50
filesync (1.3) 514 tcp CM 1.3 used this port
51
filesync (2.x) 21873 tcp Older servers used this port. It is left open for back-
52
wards compatibility only.
53
filesync (3.0 and 21874 tcp input on LSPs and ESSs output on main servers
54
later)
55
56
57 IMAPI 55000 tcp Used by IA770
58 message waiting 1024:65535 udp Used by IA770 to message manager
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 35 -

1 Table 5. CM Specific Services


2
3 PE Service Port Protocol USE
4 yes ip-signaling-1 5000:5021 tcp Split into two parts, ip-signaling-1 and 2 due to use of
5 5022 and 5023 for SAT access.
6 yes ip-signaling-2 5024:9999 tcp
7 yes ip-signaling 2048:65535 udp
8
yes H.323 Signaling 1024:65531 tcp
9
yes h245 59000:59200 tcp Gateway interface.
10
11
12
13 <134539-56> Special Port Handling
14
15 The following ports shall be opened in the INPUT CHAIN with conditions as follows:
16 ACCEPT tcp -- PE * tcp dpt:2945 sklimit: avg 50/sec burst 100 port 2945 listener
DROP tcp -- PE * tcp dpt:2945
17
18 ACCEPT tcp -- PE * tcp dpt:1720 flags:0x3F/0x02 acl port: 1720 listener
19 LOG tcp -- PE * tcp dpt:1720 flags:0x3F/0x02 limit: avg 5/min burst 5 LOG flags 0 level 4
DROP tcp -- PE * tcp dpt:1720 flags:0x3F/0x02
20
21 ACCEPT tcp -- PE * tcp dpt:81 flags:0x3F/0x02 listener#conn/0 < 100
22 ACCEPT tcp -- PE * tcp dpt:411 flags:0x3F/0x02 listener#conn/0 < 100
23
24 This should represent no change to how the ports are opened currently, except they will be open on the PE
25 interface as well as potentially other interfaces.
26 <134539-57> Firewall OUTPUT chain
27
TCP ports 1024 through 65535 and UDP port 1719 shall be opened in the OUTPUT CHAIN on
28
the NIC assigned to the PE interface.
29
30
31
9. Other Systems Impacted
32 • Services systems -- none
33
34 See the discussion in CID 122596 for impacts to external devices using PE on duplicated servers.
35
36 10. Special Requirements
37
38 The following requirements were in the original SRAD (122596) and in the initial drafts of this SRAD. Code for
39 these requirements is already in the base as part of the increased capacity of the S8510 servers. However, this feature
40 was deleted from this release. So although the code is already in place it needs testing.
41
42 <134539-58> Socket Increases (Testing only)
43 The number of sockets, the number of simultaneous socket requests being processed, the
44 number of simultaneous socket attempts (new socket establish) and the number of file
45 descriptors shall be increased as necessary to meet the increased capacity and performance
46 needs of the PE interface. This may involve an increase in both Linux and in the Telephony
47 Application as necessary.
48
49 The amount of the increase will be discovered during design and initial testing. The number of sockets
50 supported in the GIP is already 24,575 and may be sufficient.
51 <134539-59> S8500 LSP License Truncation Limits (Testing only)
52 The S8500 LSP servers shall have their license truncation limits raised to match those of the
53 non-XL S87xx servers (Stingray) for all limits in order to allow an S8500 LSP to provide backup
54 for an entire system. It is desirable that "XL" limits are supported if there is sufficient memory
55 available.
56
57 In configurations without port networks, there is no ESS. So this change allows the customer to essentially have
58 an enterprise backup solution. License truncation limits are defined in CID 88705. The differences are
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 36 -

1 highlighted in the following table.


2
3 Table 6. License_trunc[] From CM4.1
4
5 (9)
(6) (12) (21)
6 CHAWK_ Use
STINGRAY S8500 S8500_LSP
7 LSP
8 1 0,48000, 0,3200, 0,48000, 0,48000, /* PORT */
9 3 0,14, 0,14, 0,14, 0,14, /* VERSION */
10 5 0,27, 0,27, 0,27, 0,27, /* CCRELEASE */
11
7 0,7000, 0,1000, 0,7000, 0,7000, /* LOGGIN */
12
9 0,7000, ✰ 0,1000, 0,1000, ✰ 0,450, /* ADVOCATE */
13
14 11 0,9, 0,9, 0,9, 0,9, /* MODEL */
15 13 0,2, 0,2, 0,2, 0,2, /* ILOCATION */
16 15 0,36000, 0,2400, 0,36000, 0,36000, /* XMOB_NUM */
17 17 0,12000, 0,800, 0,12000, 0,12000, /* RO_TRK */
18 19 0,12000, 0,800, 0,12000, 0,12000, /* IP_TRK */
19 21 0,12000, ✰ 0,2400, 0,2400, ✰ 0,12000, /* IP_STN */
20 23 0,7000, ✰ 0,1000, 0,1000, ✰ 0,450, /* IPA */
21 25 0,2, 0,2, 0,2, 0,2, /* OFFER */
22 27 0,12000, 0,2400, 0,12000, 0,12000, /* RO_STN */
23
29 0,688, 0,80, 0,688, 0,688, /* DS1_ECHO */
24
31 0,128, 0,10, 0,128, 0,128, /* VAL_BRD */
25
26 33 0,3, 0,3, 0,3, 0,3, /* LBSTAR_CFF */
27 35 0,414, 0,68, 0,414, 0,68, /* IP_ATTD_CON */
28 37 0,6, 0,12, 0,6, 0,9, /* PLATFORM */
29 39 0,0, 0,0, 0,0, 0,0, /* IPSOFTPHONE */
30 41 0,250, 0,250, 0,250, 0,250, /* MGA_SRC */
31 43 0,5000, 0,800, 0,5000, 0,5000, /* SIP_TRK */
32 45 0,36000, 0,2400, 0,36000, 0,36000, /* OPT_EC500 */
33 47 0,36000, 0,2400, 0,36000, 0,36000, /* OPT_SSE */
34 49 0,36000, 0,2400, 0,36000, 0,36000, /* OPT_OPS */
35
51 0,36000, 0,2400, 0,36000, 0,36000, /* STA */
36
53 0,0, 0,0, 0,0, 0,0, /* 2602_VC */
37
38 55 0,300, 0,300, 0,300, 0,300, /* EMMC_PORTS */
39 57 0,12000, ✰ 0,2400, 0,2400, ✰ 0,12000, /* VC_H323 */
40 59 0,12000, ✰ 0,2400, 0,2400, ✰ 0,12000, /* VC_IPSP */
41 61 0,12000, ✰ 0,2400, 0,2400, ✰ 0,12000, /* UNAUTH_EPT */
42 63 0,128, 0,128, 0,128, 0,128, /* 2602_80VC */
43 65 0,128, 0,128, 0,128, 0,128, /* 2602_320VC */
44 67 0,36000, 0,2400, 0,36000, 0,36000, /* OPT_PBFMC */
45 69 0,36000, 0,2400, 0,36000, 0,36000, /* OPT_PVFMC */
46 71 0,12000, 0,1600, 0,12000, 0,12000, /* AVC_PORTS */
47
73 0,2500, ✰ 0,100, 0,100, ✰ 0,50, /*SIPEAS */
48
.. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_Agent */
49
50 .. {0,0,414} {0,0,68} {0,0,414} {0,0,68} /* IP_eCons */
51 .. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_Phone */
52 .. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_ROMax */
53 .. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_Soft */
54 .. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_API_A */
55 .. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_API_B */
56 .. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_API_C */
57 .. {0,0,12000} ✰ {0,0,2400} {0,0,2400} ✰ {0,0,450} /* IP_IR_A */
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers - 37 -

1 <134539-60> S8500 Boot Time Parameters (Testing only)


2
The S8500 shall have boot time parameters not smaller than those of an S87xx processor. It
3
is desirable that the S8500 boot time parameters support "XL" configurations if sufficient
4
memory is present.
5
6 Comments from Tam:
7 Currently the S8500 has the same btm_parm (Large) as the S87xx. As far as the XL part goes... The S8500C has 1GB of memory &
therefore has enough capacity to run with the X Large Memory Configuration. If 512MB of RAM is added to an S8500B/C server, it too
8
has enough capacity to run with XL. See Table 4 in Modhumita’s XL SRAD (117073)
9 http://compasfs.dr.avaya.com/cgi-bin/wwwcompas?prodid=117073&dformat=pdf
10 The desirable portion of this requirement would make the S8500 an XL server. For the CM4.0 XL SRAD, product management wanted
11 ONLY the S8720 to be XL. For CM4.1, when opportunity presented to make the S7810 XL, product management demurred. We might
12 want to run this past Debbie Goldman.
13
<134539-61> Ip-Interface Procr Form (Testing only)
14
15 The "target socket load" field on the "IP-Interface Procr" SAT form shall be increased to 5
16 digits (currently 4 digits).
17
18 11. ACKNOWLEDGEMENTS
19
20 Special thanks to Tam Noirot for guidance in ESS configurations, to Ji Ran for help in PE in general, and to Jim
21 Rhodes for inventing the GIP/SUSER changes to build on current work by Bob Brown, and on-going mentoring.
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers List of Requirements -38-

1 <134539-1> Which Gateways - - - - - - - - - - - - - - - - - - - - - - - - - 21


2
<134539-2> Accept Incoming Socket - - - - - - - - - - - - - - - - - - - - - 21
3
4 <134539-3> Accept Response - - - - - - - - - - - - - - - - - - - - - - - - 21
5 <134539-4> SA9009 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 22
6 <134539-5> FEAT_PRETH License Bit - - - - - - - - - - - - - - - - - - - - 22
7
8
<134539-6> H.248 Gateway License Support - - - - - - - - - - - - - - - - - 22
9 <134539-7> PClanIntf0 and PEInterface - - - - - - - - - - - - - - - - - - - 22
10 <134539-8> PE on Duplicated Main Servers Requires An Alias Address - - - - 22
11 <134539-9> PE on Duplicated ESS Servers - - - - - - - - - - - - - - - - - - 22
12
13 <134539-10> VoIp Resource - - - - - - - - - - - - - - - - - - - - - - - - - 22
14 <134539-11> IP Takeover - - - - - - - - - - - - - - - - - - - - - - - - - - 23
15 <134539-12> Flooding Prevention - - - - - - - - - - - - - - - - - - - - - - 23
16
<134539-13> PE Capacity - - - - - - - - - - - - - - - - - - - - - - - - - - 23
17
18 <134539-14> Server Capacity - - - - - - - - - - - - - - - - - - - - - - - - 23
19 <134539-15> PE and CLAN Mix (testable=n) - - - - - - - - - - - - - - - - - 23
20 <134539-16> Supported Adjuncts - - - - - - - - - - - - - - - - - - - - - - - 23
21
22
<134539-17> System Initialization - - - - - - - - - - - - - - - - - - - - - - 23
23 <134539-18> PE and Processor Interchange - - - - - - - - - - - - - - - - - - 24
24 <134539-19> Delay of Normal Operation - - - - - - - - - - - - - - - - - - - 24
25 <134539-20> SUSER Connection Types - - - - - - - - - - - - - - - - - - - 24
26
27 <134539-21> Socket Port Numbers and Firewall Values - - - - - - - - - - - - 24
28 <134539-22> SUSER Interchange Behavior - - - - - - - - - - - - - - - - - - 24
29 <134539-23> GIP Connection Types - - - - - - - - - - - - - - - - - - - - - 24
30
<134539-24> GIP Interchange Behavior - - - - - - - - - - - - - - - - - - - - 25
31
32 <134539-25> Socket Shadowing - - - - - - - - - - - - - - - - - - - - - - - 25
33 <134539-26> H.248 Gateway Session Restore - - - - - - - - - - - - - - - - - 25
34 <134539-27> H.248 Registration Preserved - - - - - - - - - - - - - - - - - - 25
35
36
<134539-28> H.248 Gateway Throttling - - - - - - - - - - - - - - - - - - - 25
37 <134539-29> Upgrade/Restore of Duplicated Main Servers - - - - - - - - - - 25
38 <134539-30> Upgrade/Restore of Simplex Main/ESS/LSP Servers and Duplicated ESS Servers 26
39 <134539-31> SOH with No IPSIs - - - - - - - - - - - - - - - - - - - - - - - 26
40
41 <134539-32> PE SOH Effect on Interchange - - - - - - - - - - - - - - - - - 26
42 <134539-33> PE SOH Determination - - - - - - - - - - - - - - - - - - - - - 26
43 <134539-34> Exchange of PE SOH - - - - - - - - - - - - - - - - - - - - - - 26
44
<134539-35> PING Device Unreachable - - - - - - - - - - - - - - - - - - - 26
45
46 <134539-36> PING Priority - - - - - - - - - - - - - - - - - - - - - - - - - 27
47 <134539-37> PE SOH Alarm - - - - - - - - - - - - - - - - - - - - - - - - - 27
48 <134539-38> PE SOH in Server Command output - - - - - - - - - - - - - - - 27
49
50
<134539-39> Arbiter Interfaces - - - - - - - - - - - - - - - - - - - - - - - - 27
51 <134539-40> CNA CNB Optional - - - - - - - - - - - - - - - - - - - - - - 30
52 <134539-41> Configure ESS/LSP - - - - - - - - - - - - - - - - - - - - - - 30
53 <134539-42> PE Assignment to NIC - - - - - - - - - - - - - - - - - - - - - 30
54
55 <134539-43> PE Defaults - - - - - - - - - - - - - - - - - - - - - - - - - - 31
56 <134539-44> PE Address Data - - - - - - - - - - - - - - - - - - - - - - - - 31
57 <134539-45> PE Change Notice - - - - - - - - - - - - - - - - - - - - - - - 31
58
<134539-46> NIC Labels - - - - - - - - - - - - - - - - - - - - - - - - - - - 31
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions
PE for Duplicated Main Servers List of Requirements -39-

1 <134539-47> PE Health Check - - - - - - - - - - - - - - - - - - - - - - - - 31


2
<134539-48> PE Administration by The User - - - - - - - - - - - - - - - - - 31
3
4 <134539-49> S8300 PE Assignment and Use - - - - - - - - - - - - - - - - - 32
5 <134539-50> S8400 PE Assignment and Use - - - - - - - - - - - - - - - - - 32
6 <134539-51> Xen Server PE Assignment and Use - - - - - - - - - - - - - - - 32
7
8
<134539-52> S8500 PE Assignment and Use - - - - - - - - - - - - - - - - - 32
9 <134539-53> S87xx Duplicated Server PE Assignment and Use - - - - - - - - 32
10 <134539-54> Reset to Defaults - - - - - - - - - - - - - - - - - - - - - - - - 32
11 <134539-55> Firewall INPUT Chain - - - - - - - - - - - - - - - - - - - - - 33
12
13 <134539-56> Special Port Handling - - - - - - - - - - - - - - - - - - - - - - 35
14 <134539-57> Firewall OUTPUT chain - - - - - - - - - - - - - - - - - - - - 35
15 <134539-58> Socket Increases (Testing only) - - - - - - - - - - - - - - - - - 35
16
<134539-59> S8500 LSP License Truncation Limits (Testing only) - - - - - - - 35
17
18 <134539-60> S8500 Boot Time Parameters (Testing only) - - - - - - - - - - - 37
19 <134539-61> Ip-Interface Procr Form (Testing only) - - - - - - - - - - - - - - 37
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

Compas ID 134539 Avaya Inc. - PROPRIETARY Last Modified: February 4, 2009


Issue 1.2 Use pursuant to Company Instructions

You might also like