IPC Software Upgrade Procedure

IPC Software Upgrade Procedure

1 INTRODUCTION....................................................................................................................................... 2
2 MANUAL PRECHECKS............................................................................................................................. 2
3 GENERAL............................................................................................................................................... 3
3.1 Exchange SSH user keys.................................................................................................. 3
3.2 IPCCLI Interface to issue SU commands...........................................................................4
3.3 Delete Versions Precheck.................................................................................................. 5
3.4 Delete Versions.................................................................................................................. 6
4 SU UPGRADE ACTIONS........................................................................................................................... 6
4.1 SU Commit Precheck......................................................................................................... 6
4.2 SU Commit......................................................................................................................... 6
4.3 SU Download Precheck..................................................................................................... 6
4.4 SU Download..................................................................................................................... 7
4.5 SU Install Precheck............................................................................................................ 8
4.6 SU Install........................................................................................................................... 8
4.7 SU Firmware Update Precheck.......................................................................................... 8
4.8 SU Firmware Update......................................................................................................... 9
4.9 SU Activate Precheck........................................................................................................ 9
4.10 SU Activate.................................................................................................................. 10
5 SU DOWNGRADE ACTIONS................................................................................................................... 10
5.1 SU Backout Precheck...................................................................................................... 10
5.2 SU Backout...................................................................................................................... 11
5.3 SU Fallback Precheck...................................................................................................... 11
5.4 SU Fallback...................................................................................................................... 11
6 Applying Private Patches.................................................................................................................... 12
IPC Software Upgrade Procedure

1 Introduction

Purpose of this document is to assist only Alcatel-Lucent Test and Field engineers to
perform Software Upgrade actions.

This document explains in brief the steps that are involved in carrying out the IPC
Software Upgrade.

Before starting with the download of new load on to the IPC, operator must exchange the
ssh user keys for swdnload user.

Precheck
All the SU Action has an associated Precheck action to be completed before performing
SU Action. If Precheck Action returns warning then operator must understand the
warning and if required to proceed further then actual SU action should be executed with
FORCED option as YES.

Some of the actions need same input values to be specified for precheck and actual
action. Make sure that this input values match for precheck and actual action.

Monitoring SU Actions
All the below SU actions invoked through IPCCLI will return the status as ACCEPTED
or FAILED to indicate the acceptance of action. Operator must monitor the SU action
status through suReport file after action is accepted through IPCCLI.

suReport will only be available on the solaris/fms blade where SUMgr is currently
available. So during Activate/Backout/Fallback actions wherein there is switchover of
OAMServers suReport should be monitored where SUMgr is available.

SU Report file is located at /var/spool/su/suReport.dat

For SU Actions like SU Firmware Update, SU Delete Versions, SU Commit status of the
command can also be checked through IPCCLI command.

/var/spool/fm/log/logSU/*.su file can be checked for additional information in case of

failure during Software Upgrade.

2 Manual Prechecks

It is recommended to perform following manual prechecks before proceeding with SU

Actions.
IPC Software Upgrade Procedure

1. Make sure that both solaris/fms blades are in LEAD/ACT state by issuing
RCCcstat command on solaris/fms blade.
2. Make sure that all the 17 OAMServers are running successfully using
rnccheckservers command.
3. Make sure that following ICC processors are pingable from both solaris/fms blade
 GICC-3 or GICC-4 from both shelf (GICCpair should be pingable)
 GICC-19 or GICC-20 from both shelf (GICCpair should be pingable)
 CICC1-12-0 and CICC2-12-0 (SPs)
 Atleast 2 TPs per shelf

4. Make sure that all the CMMs and ESC cards are pingable from both solaris/fms
blade.
5. Make sure that “resetRequired” attribute of LIPC is set to “false”. If
resetRequired attribute is set to “true” then perform LIPC Reset through IPCCLI
before proceeding with SU Action.

3 General
3.1 Exchange SSH user keys

This step needs to be carried out on either fms01 or fms02 only once per IPC for the
specific remote host.

1. Generate the SSH user keys for ‘swdnload’ user on IPC using following
command
/flx/RNCsec/current/bin/GenerateSSHUserKeys.sh swdnload

User keys will be created in

/flx/data/RNCusers/swdnload/.ssh directory.

2. Create ‘swdnload’ user on the remote machine from where new IPC load will be
picked. To create the user execute following command on remote machine
useradd -m swdnload
passwd swdnload

On some unix machines, “passwd swdnload” may throw errors. In that case use
following command to give password.
passwd –r files swdnload

3. Create .ssh directory under ~swdnload on remote machine

cd ~swdnload
mkdir .ssh
IPC Software Upgrade Procedure

touch authorized_keys

4. Add the contents(SSH key) of

/flx/data/RNCusers/swdnload/.ssh/id_rsa.pub from IPC to
~swdnload/.ssh/authorized_keys on remote machine.
Note: There should not be any newline character in the SSH key when it is copied
to remote machine.

5. To make sure that SSH User keys are successfully added to remote host,
Login to IPC as swdnload user and then try sftp or ssh to remote machine.
sftp swdnload@<IP of remote host>

Above sftp command should not ask for password.

Note: It may prompt user to say ‘yes’ to add the host key to known host file. In
that case operator should say yes.

3.2 IPCCLI Interface to issue SU commands

All the SoftwareUpgrade commands mentioned below needs to be invoked as follows.

[IPC1314:fms01]-> IPCCLI menu

----------( IPC CLI Main Menu )----------

1. Batch Job Scheduling

2. Fault Management
3. Log and File Management
4. Object Management
5. IPC Audits
6. Software Upgrade
7. System Management

Select from menu or (E)xit (H)elp (F)ile? 6

-----------( Software Upgrade )-----------

1. Get Software State

2. Get Software Versions
3. Get Software Versions Status
4. Get Software Package Info
5. Get Software Package Info Status
6. Perform Software Upgrade Action
7. Get SU Action Progress
8. Get Software Upgrade Precheck Status
9. Software Update Database Audit Sync
IPC Software Upgrade Procedure

10. Get SU Commands History

11. Get SU Action Report

Select from menu or (B)ack (M)ain (E)xit (H)elp (F)ile? 6

-------------------( Perform Software Upgrade Action )-------------------

1. SU Download PreCheck 13. SU Backout PreCheck

2. SU Download 14. SU Backout
3. SU Install PreCheck 15. SU Commit PreCheck
4. SU Install 16. SU Commit
5. SU Apply PreCheck 17. SU Commit Status
6. SU Apply 18. SU Delete Versions PreCheck
7. SU Deapply PreCheck 19. SU Delete Versions
8. SU Deapply 20. SU Delete Versions Status
9. SU Activate PreCheck 21. Abort in Progress SU Action
10. SU Activate 22. SU Firmware Update PreCheck
11. SU Fallback PreCheck 23. SU Firmware Update
12. SU Fallback 24. Get SU Firmware Update Status

Select from menu or (B)ack (M)ain (E)xit (H)elp (F)ile?

3.3 Delete Versions Precheck

Delete Versions action is used to remove one or more IPC software versions from the IPC

Note: Regardless of the options specified, the delete versions command does not
allow you to delete a current or committed software version.

-----( SU Delete Versions PreCheck Parameter Selection )-----

1. type=IPC
2. version=allVers

type= should always be specified as IPC

Specify one of the following for version=,

Select Number to check that a specific software version can be deleted, enter
the version number (the Generic Descriptor File name without the .gdf extension).

version=obvers
To check that all versions except the current, previous, committed, newdownloaded and
newinstalled versions can be deleted.

version=allvers
IPC Software Upgrade Procedure

To check that all versions except the current and committed versions can be deleted.

3.4 Delete Versions

-----( SU Delete Versions Parameter Selection )-----

1. type=IPC
2. version=allVers
3. forced=NO

Specify type and version as specified in the corresponding precheck action above.

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully

4 SU Upgrade Actions
4.1 SU Commit Precheck

For carrying out IPC Upgrade through SUMgr, atleast one load should exist as committed
load on IPC.

Note: In general whenever SU Upgrade is carried out without DM (Data Migration)

any of the previous or current load can exist as committed load. But when SU
Upgrade is carried out with DM, it is mandatory to commit the current load before
activating the new load.

SU Commit Precheck does not prompt for any input from operator.

4.2 SU Commit

-----( SU Commit Parameter Selection )-----

1. forced=NO

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully
IPC Software Upgrade Procedure

4.3 SU Download Precheck

-----( SU Download PreCheck Parameter Selection )-----

1. sourceFile=
2. server=
3. install=NO

Do not put quotes around sourceFile= and server= when specifying the input parameter.

Specify sourceFile= with the gdf file name without path. Specify only gdf filename
Eg. IPC-02.02.00.02.gdf

Specify server= with the IP Address of the remote host

Specify install= with following,

YES, To continue with Install after download
NO, To only perform Download Operation.

4.4 SU Download

-----( SU Download Parameter Selection )-----

1. sourceFile=
2. server=
3. install=NO
4. forced=NO

Do not put quotes around sourceFile= and server= when specifying the input parameter.

Specify sourceFile= with the same value that was specified for corresponding precheck
Action.

Specify server= with the same value that was specified for corresponding precheck
Action.

Specify install= with the same value that was specified for corresponding precheck
Action..

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully
IPC Software Upgrade Procedure

4.5 SU Install Precheck

This STEP needs to be followed only if install= option was set to NO in the Download
action above.

-----( SU Install PreCheck Parameter Selection )-----

1. sourceFile=

Do not put quotes around sourceFile= when specifying the input parameter.

Specify sourceFile= with the gdf file name without path. Specify only gdf filename.
Eg. IPC-02.02.00.02.gdf

4.6 SU Install

-----( SU Install Parameter Selection )-----

1. sourceFile=
2. forced=NO

Do not put quotes around sourceFile= when specifying the input parameter.

Specify sourceFile= with the same value that was specified for corresponding precheck
Action.

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully

4.7 SU Firmware Update Precheck

-----( SU Firmware Update PreCheck Parameter Selection )-----

1. fwUpdateOperation=UPGRADE

Specify fwUpdateOperation= with,

UPGRADE, This will update the firmware with the new firmware available in newly
installed load.
IPC Software Upgrade Procedure

ROLLBACK, Use this option only if firmware update has to be done from the current
load

Note: It is recommended to do Firmware Update with

fwUpdateOperation=UPGRADE after the Install is complete and before Activate is
done.

4.8 SU Firmware Update

-----( SU Firmware Update Parameter Selection )-----

1. fwUpdateOperation=UPGRADE
2. forced=NO

Specify fwUpdateOperation= with the same value that was specified for corresponding
precheck Action.

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully

4.9 SU Activate Precheck

-----( SU Activate PreCheck Parameter Selection )-----

1. autobackout=NO

Specify autobackout= with following options,

YES, Incase of some failure during Activation, Backout will be performed automatically.
NO, No Autobackout will be performed. IPC will remain in Activate failed state incase
of failure during Activation.

Activate Precheck may throw some warnings, because of improper number of healthy
CICC, GICC processors. In that case you have to activate using forced=YES as
mentioned above.

Note1: We recommend performing SU Activate action with autobackout option set

to NO. In case of Activate failure, autobackout=NO allows the operator to
investigate the reason for failure. If the failure was because of some faulty card, then
Operator can continue with the activate procedure after replacing the faulty card
with proper one.
IPC Software Upgrade Procedure

Note2: If SUActivate action fails for some reason and if the issue can be fixed, then
SUActivate action can be reissued. One needs to do SU Activate Precheck and then
issue SU Activate which will proceed from the same step where it failed earlier.

4.10 SU Activate

-----( SU Activate Parameter Selection )-----

1. forced=NO
2. autobackout=NO

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully

Specify autobackout= with the same value that was specified for corresponding
precheck Action

5 SU Downgrade Actions

SU Backout and SU Fallback actions are required incase of Activation failure. Also if
operator is not satisfied with the newly activated load then Backout and Fallback actions
can be performed.

Note that SUBackout will go back to the previous load that is present on the IPC whereas
SUFallback will go back to the committed load.

5.1 SU Backout Precheck

SU Backout can be performed when operator is not satisfied with the newly activated
load or incase of failure during SU Activate.

Note: SUBackout will be automatically issued, if autobackout option was set to YES
during SU Activate action.

SU Backout Precheck will not prompt user for any input.

Backout Precheck may throw some warnings, because of improper number of healthy
CICC, GICC processors. In that case you have to backout using forced=YES as
mentioned above.
IPC Software Upgrade Procedure

Note: If Backout action fails for some reason, then backout cannot be reissued.
Operator has to do Fallback, which will move to the previously committed IPC load.

5.2 SU Backout

-----( SU Backout Parameter Selection )-----

1. forced=NO

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully

5.3 SU Fallback Precheck

SU Fallback can be performed when operator is not satisfied with the newly activated
load or incase of failure during SU Activate.
Difference between Backout and Fallback is that, Fallback will always go back to
committed load, whereas Backout will go back to previous load.

SU Fallback Precheck will not prompt user for any input.

Fallback Precheck may throw some warnings, because of improper number of healthy
CICC, GICC processors. In that case you have to fallbackactivate using forced=YES as
mentioned above

Note1: If SUFallback action fails for some reason and if the issue can be fixed, then
SUFallback action can be reissued. One needs to do SU Fallback Precheck and then
issue SU Fallback which will proceed from the same step where it failed earlier.

5.4 SU Fallback

-----( SU Fallback Parameter Selection )-----

1. forced=NO

Specify forced= with following,

YES, If Precheck returned warning
NO, If Precheck completed successfully
IPC Software Upgrade Procedure

6 Applying Private Patches

Any patches applied on IPC before SUActivate, SUBackout or SUFallback will not be
available after the completion of SU Actions. Operator must reapply the patches if they
are required.