EMC DOCUMENTUM JAVA

METHOD SERVER HIGH AVAILABLITY

CONFIGURATION
JMS HA configuration for Fail over Support

ABSTRACT
This white paper explains how to configure the JMS HA/failover and configuring
methods for failover. The white paper describes the steps to track the execution
of methods from logs.

November 2013

EMC WHITE PAPER

TABLE OF CONTENTS
ABSTRACT ............................................................................................................................................. 1
TABLE OF CONTENTS ............................................................................................................................. 2
INTRODUCTION ..................................................................................................................................... 4
Java method server ............................................................................................................................. 4
How does the method server work? ....................................................................................................... 4
Its a web application ..................................................................................................................... 4
Java Method Server for High-Availability ................................................................................................ 5
Failover ........................................................................................................................................ 5
Load balancing .............................................................................................................................. 5
Audience ............................................................................................................................................ 5
JAVA METHOD SERVER FOR HIGH AVAILABILITY .................................................................................. 6
TYPE1: SINGLE HOST SETUP .................................................................................................................. 7
STEPS FOR CONFIGURING JMS HA 7.0................................................................................................... 8
1.
Preparing for Installation/Prerequisite: ......................................................................................... 8
Content Server in HA ..................................................................................................................... 8
Packaging WebApps ....................................................................................................................... 8
2.
Creating second JMS Instance ..................................................................................................... 9
3.
Verification of Instance .............................................................................................................. 15
dm_jms_config : .......................................................................................................................... 15
4.
Associating Content Servers with JMS Instance ............................................................................ 16
5.
Refresh Content Server Cache .................................................................................................... 17
6.
DUMP_JMS_CONFIG_LIST .......................................................................................................... 18
Configuring HA parameters ................................................................................................................. 22
Enabling logs and Understanding logs ................................................................................................... 22
Error Messages: ........................................................................................................................... 23
Sample Logs ..................................................................................................................................... 23
Server.log ................................................................................................................................... 23
Docbase.log ................................................................................................................................. 24
System Methods details ...................................................................................................................... 25
Options for method ............................................................................................................................ 26
Verification steps ............................................................................................................................... 26
Limitations ........................................................................................................................................ 27
Custom Method configurations ............................................................................................................. 27
TYPE2: TWO HOST HA SETUP ............................................................................................................... 28
Digital Signature Verification ............................................................................................................... 28
TYPE3: MULTIPLE HOSTS SETUP .......................................................................................................... 30
STEPS FOR CONFIGURING JMS HA 6.7 SP2 .......................................................................................... 31
TYPE1:SINGLE HOST SETUPFOR 6.7 SP2 ............................................................................................. 31
1.
Preparing for Installation/Prerequisite: ........................................................................................ 31
Content Server in HA .................................................................................................................... 31
Packaging WebApps ...................................................................................................................... 31
2.
Creating second JMS Instance .................................................................................................... 31

3.

Verification of Instance and Associating Content Server with JMS Instance ...................................... 32

TYPE 2: TWO OR MULTI HOST SETUP FOR 7.0 ..................................................................................... 32

STEPS FOR CONFIGURING JMS HA 7.1................................................................................................. 32
CONCLUSION ....................................................................................................................................... 33
References ........................................................................................................................................ 33
CONTACT US ........................................................................................................................................ 33

INTRODUCTION
Java method server
The Documentum 5 platform introduced the Java method server as a means of improving the performance and scalability of the
Documentum method architecture. In the past couple of years, Documentum has focused on Java as a cornerstone for its
technology. Documentum clients are being written in Java more and more frequently. The core Documentum libraries are
implemented in Java. Running Java-based methods, however, has been the most painful of experiences. The Java method
server aims to fix this.
The method server runs as a process distinct from the Content Server and is responsible for the invocation of the individual
methods. Each method invocation results in a call to the method server which manages the actual execution of the Java-based
method implementation. As the method server is a continuously running process, there is no overhead involved in loading and
tearing-down a Java virtual machine with each invocation. This yields an enormous benefit in overhead.
As methods are typically executed over and over again, the classes involved in the implementation of the method only need be
loaded once. Other overhead, such as the creation of docbase sessions may also be reduced as the method server may reuse
sessions with connection pooling. The result is methods that execute in milliseconds instead of seconds and which use
significantly fewer system resources.

How does the method server work?

Its a web application
The method server itself is a Java-based web application. It communicates with the Content Server via HTTP calls. Each time a
method is invoked, the Content Server makes an HTTP request passing the name of the Java class which implements the
method along with any specified arguments to a servlet which knows how to execute the specified method.
Methods in Documentum are defined via dm_method objects. To indicate that a Java method should execute via the Java
method server, set the dm_method.method_type attribute tojava, the dm_method.use_method_server attribute to 1, and the
dm_method.method_verb attribute to the classname of the method implemenation. When the Content Server sees a method
matching this pattern, it opens a HTTP request to the Java method server requesting executing of the method with the specified
class.
This simple architecture solves the overhead problem and also introduces a convenient mechanism for dramatically improving
the scalability of the Content Server. Although it is not configured out-of-the-box to support this, Method servers may be
executed on a machine physically separate from the Content Server. It is even possible to run the method server on a platform
different from the Content Server (ie. running the Content Server on Solaris and the method server on Windows). This offloads
work from the Content Server which might enable the Content Server to handle more users.
If the methods are expensive to execute, it is also possible to load-balance method servers using industry standard HTTP load
balancers. There is nothing special or complicated about the method server. It simply requires a J2EE-compatible servlet
container

Java Method Server for High-Availability

Content Server and Java Method Server (JMS) each support high-availability (HA). Content Server supports two HA types:
failover and load balancing. JMS supports failover only.

Failover
In a failover setup, if one of the Content Servers fails, the other Content Servers in the failover setup continues with the service.

Load balancing
Load balancing involves operating redundant Content Servers where the service load is balanced between Content Servers to
maximize performance. In a standard Content Server load-balancing scenario, proximity values are used to determine which
Content Server processes an item. In a cluster scenario, third-party load-balancers are used.

Audience
The information in this guide is for application and system administrators who use CS and JMS software to deploy and custom
method. The intended audience is EMC support personnel, professional services, and customers.
Terminology

JMS: Java Method Server

CS: Content Server

CLI: Command-Line Interface

GUI: Graphical User Interface

JAVA METHOD SERVER FOR HIGH AVAILABILITY

JMS supports the following HA configurations:

Content Server and JMS on a single host

Content Server and JMS on two hosts

Content Server and JMS on multiple hosts

Content Server HA deployment involves two or more Content Servers. JMS HA involves adding JMSs to additional Content
Servers such that each Content Server has a dedicated JMS. JMS HA is automatically enabled by associating each Content
Server with their dedicated JMS.
JMS HA flowchart

Multiple content servers on different hosts serving single repository

Multiple content servers on same host serving single repository

To extend the system, you must add more content servers. Each content server has its own default local embedded JMS.
Therefore, you can have as many JMSs as the number of CSs in the system.

Limited to one JMS per content server, other CSs JMS are used as backup.

JMS HA is not intended to address scaling issues.

Under normal operation, Content server always communicates to its own default local embedded JMS. It will use failover
JMS only if its default embedded JMS becomes unavailable. Content server will resume using its own default JMS when that
it comes back online.

TYPE1: SINGLE HOST SETUP

JMS HA on Single Host

For a Single Host setup both the Primary and Secondary Content Server are installed on the same Host.
The Primary JMS will be installed and configured during the Primary Content Server Installation.
After the Content Servers are installed the second JMS is installed and configured.

STEPS FOR CONFIGURING JMS HA 7.0

1. Preparing for Installation/Prerequisite:
Content Server in HA
To Achieve JMS HA, a second (or another) Content Server associated with JMS. Single Content server does not support
multiple JMS. To Install Content Server in HA refer to the Content Server Enterprise Edition 7.0 Installation Guide.

Packaging WebApps
The WebApps are deployed on the first JMS. Using the jmsConfig.exe, package WebApps to be deployed on the second JMS.
ServerApps.ear - WebApp has to be deployed for the JMS server.
In the webappsToInclude.txt, add the names of the WebApps to be packaged. This file is found under
$Documentum\jmsTools\config\webappsToInclude.txt
Run the jmsConfig.exe tool under $Documentum\jmsTools\bin.
Select Package only option. And Click Next

After the Packaging is complete the WebApps are located at $Documentum\jmsTools\webapps\jmsWebApps.jar.

Copy the jmsWebApps.jar file on the machine to install second JMS under $Documentum\jmsTools\webapps path.

2. Creating second JMS Instance

Run the jmsConfig.exe under C:\Documentum\jmsTools\bin folder.
Select Jmsn Config to create a new JMS instance and click Next.

On the Select Configuration mode page, select Create Instance and click Next.
Provide an Instance Name for the JMS and Click Next.

10

Set the password for the Application Server - Admin User.

Provide the port details to deploy JMS. JMS requires consecutive 20 ports.
By default 9080 to 9099 ports will be selected for JMS. Port numbers can be changed, ensure consecutive 20 ports are
available.

11

On Configure repository domain name, by default the fully qualified DNS is given. Verify and click next.
Provide the Primary Instance Installation Owner and Primary Connection Broker details:

12

Installation Owner: Username and Password

Primary Connection Broker:

Connection Broker host: Machine Name/IP

Connection Broker port: Broker port (default port 1489)

Select the Repository and click next.

13

Select Yes in the below screen since the primary content server is installed on same host.
For multi host setup - select No as the primary CS and secondary CS are on different host.

Click on Done. Configuration process is complete.

14

3. Verification of Instance

The new instance of JBoss application server is created under C:\Documentum\jboss5.1.0\server.

Folder named DctmServer_instancename is created.

Verify the packaged applications are deployed under C:\Documentum\jboss5.1.0\server\DctmServer_secondJMS\deploy.

Start the new JMS instance. Run the startsecondJMS.bat or from Services start Documentum Java Method Server
instancename.

Access the URL:

http://SecondHost:port/DmMethods/servlet/DoMethod
(eg: http://localhost:9100/DmMethods/servlet/DoMethod ) from IE. Browser will display Documentum Java Method Server.

Each JMS instance has a dm_jms_config object associated with it.

dm_jms_config :
This object represents an instance of Java method server. It contains information about servlet_names, base_uris, its
associated content server(s), and respective jms to cs proximity value(s)
Run Iapi query:
?,c,select r_object_id,object_name from dm_jms_config
An additional object will be created. Verify the object name displays JMS secondHost:port
List of jms_config objects

15

4. Associating Content Servers with JMS Instance

Associating the Java Method Server with Content Server

16

Login to DA and go to Administration>Basic Configuration>Java Method Servers.

Select first JMS and right click > Properties

By Default the first JMS will be associated with primary/first Content Server.

Set the Second Content Server as secondary

Select second JMS and right click > Properties

Set the Second Content Server as primary

Set the First Content Server as secondary

5. Refresh Content Server Cache

In a normal operational environment, refreshing content servers JMS config list cache is needed only when a new jms config
object is added to, or an existing one is updated or deleted from the database. Content server cache update is handled by the
servers change checker process/thread. That is, whenever theres a jms config object change in the docbase, the change
checker will cause all content servers that are serving the same docbase to update their respective server cache automatically.
The server cache might not be updated immediately due to change checkers frequency of checking changes.
REFRESH_JMS_CONFIG_LIST RPC function is introduced to allow administrators to refresh the content servers cached JMS list
immediately after a change is made to a jms_config object. However, issuing REFRESH_JMS_CONFIG_LIST will only update
the connected servers cache.

It does not update other non-connected servers caches.

At run time, content server will figure out which JMS object defined in database is considered as default local embedded JMS,
which one is for load balancing, and which one is for failover.
In a multi-server-single-docbase environment, issuing this RPC call on one content server will only refresh that connected
servers cache. It does not refresh other content servers caches.
To force refresh a target content servers JMS config list kept in its cache, this RPC call must be issued from within a connected
session (an established session to the target content server).

17

6. DUMP_JMS_CONFIG_LIST
RPC function DUMP_JMS_CONFIG_LIST is introduced to allow administrators to see the current status of the JMS list kept in
content servers cache (i.e. shared memory), in real time. The typed object returned from this RPC call contains information
about the last time the list was refreshed, which jms are marked as dead in server cache (even though their is_disabled
attribute in dm_jms_config object is set to F as not-disabled, in the database).
If a JMS is marked as dead in server cache, DUMP_JMS_CONFIG_LIST will show when was the last failure that caused server
to believe its dead, and when will the server re-try to POST to this dead JMS again, how many times have we failed to send
HTTP POST requests to that JMS.
In a multi-server-single-docbase environment, running DUMP_JMS_CONFIG_LIST on different servers could result in different
views/output. The same jms config object could be considered as default local embedded JMS for a primary content server, but
it is also considered as a failover jms for secondary content server, but it would not have been considered a failover jms for 3rd
content server if the 3rd is an RCS
Connect to each content server, run REFRESH_JMS_CONFIG_LIST RPC to refresh content server cache, then run
DUMP_JMS_CONFIG_LIST RPC to examine the JMS failover information stored in content server cache. Connect to CS1, then
CS2, run these two RPCs one by one.

API> connect,Repository.CSconfiguration,CS-Administrator,Password
...
s1
API> apply,c,NULL,REFRESH_JMS_CONFIG_LIST
...
q0
API> close,c,q0
...
OK
API> apply,c,NULL,DUMP_JMS_CONFIG_LIST
...
q0
API> next,c,q0
...
OK
API> dump,c,q0
...
USER ATTRIBUTES

jms_list_last_refreshed

: Fri Sep 13 09:18:46 2013

incr_wait_time_on_failure
max_wait_time_on_failure
current_jms_index
jms_config_id

18

: 30
: 3600
: -1

[0]: 0800009080000cc3

[1]: 0800009080000d77
jms_config_name

[0]: JMS CS1DNS: Port for RepoJMS1.RepoJMS1

[1]: JMS CS2DNS: Port for RepoJMS1

server_config_id

[0]: 3d00009080000102
[1]: 3d00009080000d2e

server_config_name

[0]: RepoJMS1

[1]: csjmsha_RepoJMS1
jms_to_cs_proximity

[0]: 1

[1]: 1
is_disabled_in_docbase

[0]: F

[1]: F
is_marked_dead_in_cache

[0]: F

[1]: F
intended_purpose

[0]: DM_JMS_PURPOSE_DEFAULT_EMBEDDED_JMS
[1]: DM_JMS_PURPOSE_FOR_FAIL_OVER

last_failure_time

[0]: N/A
[1]: N/A

next_retry_time

[0]: N/A
[1]: N/A

failure_count

[0]: 0
[1]: 0

SYSTEM ATTRIBUTES

APPLICATION ATTRIBUTES

INTERNAL ATTRIBUTES

API> close,c,q0
...
OK

19

Content Server 1

20

Content Server 2

21

Configuring HA parameters

Two new configuration parameters are introduced in server.ini for JMS failover.
o

incremental_jms_wait_time_on_failure=30

jms_max_wait_time_on_failures=3600

These two config params are optional, if not present in server.ini, default values are 30 and 3600 seconds (3600 seconds is 1
hour) respectively.

incremental_jms_wait_time_on_failure config value is used for content server to determine what time should it retry to
POST to a previously failed Java method server again. By default, the incremental time interval is set to 30 seconds.

jms_max_wait_time_on_failures config value is to put a cap on wait time so that we dont end up waiting for days or
weeks for the next retry time. Default jms_max_wait_time_on_failures is 1 hr (i.e. 3600 seconds).

Enabling logs and Understanding logs

To turn on debug tracing on JMSs web.xml, This file is under

C:\Documentum\jboss5.1.0\server\DctmServer_secondJMS\deploy\ServerApps.ear\DmMethods.war\WEB-INF for methods
(this option requires app server restart)
In the web.xml, set the trace parameter value tot
<init-param>
<param-name>trace</param-name>
<param-value>t</param-value>
</init-param>
The trace details is logged in Server.log under $Documentum\jboss5.1.0\server\DctmServer_MethodServer\log

To turn on trace_launch at content server level, run the below commands from IAPI
For HTTP POSTs:
apply,c,NULL,SET_OPTIONS,OPTION,S,trace_http_post,VALUE,B,T
For Complete Launch:
apply,c,NULL,SET_OPTIONS,OPTION,S,trace_complete_launch,VALUE,B,T
The debug tracing details are logged in server.log under $Documentum\dba\log\RepositoryName.log

To turn on trace_launch at method level, while running the method enable trace launch option.
The debug tracing details are logged in server.log under $Documentum\dba\log\RepositoryName.log

Installer log files

JMS Configuration Program logs

C:\Documentum\jmsTools\bin\ or $DOCUMENTUM_SHARED/jmsTools/bin/
install.log
setupError.log
C:\Documentum\dba\config\docase\dm_jms_config_setup.out

22

Error Messages:

DM_METHOD_E_HTTP_COMMUNICATION
The app server which the Content Server tried to send POST request to, is unavailable/down

DM_METHOD_E_HTTP_POST_APP_SERVER_NAME_NOTFOUND
Theres no matching app_server_name/servlet_name value found in any of the dm_server_config/dm_jms_config objects

DM_METHOD_E_NO_JMS_AVAILABLE2
Content server believes all of the Java method servers are currently down and not accepting any POST requests. No one is
available.

DM_METHOD_E_HTTP_POST_FAILED
Content server failed to send POST request to app server

Sample Logs
Server.log
03:09:00,212 INFO [stdout] (http--0.0.0.0-8080-2) <DCTM><DEBUG> HTTP request from 10.31.69.85, invoking
com.emc.documentum.server.method.rename.UserRename.execute(), ARGUMENTS []
03:09:00,233 INFO [stdout] (http--0.0.0.0-8080-2) <DCTM><INFO> [http--0.0.0.0-8080-2:Method
name=com.emc.documentum.server.method.rename.UserRename] Completed executing. Total Time Taken: 0 secs
03:09:28,186 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString():
signatureParams=method_verb,__dm_docbase__,__dm_server_config__
03:09:28,189 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString(): paramName=method_verb
03:09:28,192 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString():
paramVals[0]=com.emc.documentum.server.method.rename.UserRename
03:09:28,194 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString():
encodedParamValue=com.emc.documentum.server.method.rename.UserRename
03:09:28,197 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString():
paramName=__dm_docbase__
03:09:28,199 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString(): paramVals[0]=Repo1
03:09:28,200 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString(): encodedParamValue=Repo1
03:09:28,202 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString():
paramName=__dm_server_config__
03:09:28,204 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString(): paramVals[0]=csd70_Repo12
03:09:28,206 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString():
encodedParamValue=csd70_Repo12
03:09:28,209 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> constructQueryString():
queryString=method_verb=com.emc.documentum.server.method.rename.UserRename&__dm_docbase__=Repo1&__dm_serve
r_config__=csd70_Repo12
03:09:28,212 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validateDigitalSignature(): docbaseName=Repo1
03:09:28,213 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validateDigitalSignature():
signature=My6j6Mgcbxac1XDy/B1aL64CYnyyH1E4UIPb/yhoHcq3Uet43KvYVIMYEpGrByaSMJvft6wbHoZWs14qmlB6V+ihSFVi+3rN
n9+dnRAUH1ksY0L66qyr8FJbV3alcscOOpaJlPy6VsymCwAv02iptkmw9C3er3p83b5aShJVrF0=, signature.length()=172
03:09:28,218 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validateDigitalSignature():
installOwnerName=Administrator

23

03:09:28,220 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> validateDigitalSignature():

queryString=method_verb=com.emc.documentum.server.method.rename.UserRename&__dm_docbase__=Repo1&__dm_serve
r_config__=csd70_Repo12, queryString.length()=117
03:09:28,224 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> verifyDigitalSignature(): docbaseName=Repo1:
installOwnerName=Administrator:
inputData=method_verb=com.emc.documentum.server.method.rename.UserRename&__dm_docbase__=Repo1&__dm_server_
config__=csd70_Repo12:
encodedDigitalSignature=My6j6Mgcbxac1XDy/B1aL64CYnyyH1E4UIPb/yhoHcq3Uet43KvYVIMYEpGrByaSMJvft6wbHoZWs14qmlB
6V+ihSFVi+3rNn9+dnRAUH1ksY0L66qyr8FJbV3alcscOOpaJlPy6VsymCwAv02iptkmw9C3er3p83b5aShJVrF0=:
encodedDigitalSignature.length()=172
03:09:28,233 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> verifyDigitalSignature(): success
03:09:28,260 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:Method
name=com.emc.documentum.server.method.rename.UserRename] Successfully located java method/module
03:09:28,264 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:Method
name=com.emc.documentum.server.method.rename.UserRename] Is an instance of IDfMethod
03:09:28,267 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:Method
name=com.emc.documentum.server.method.rename.UserRename] Preparing to execute.
03:09:28,271 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><DEBUG> HTTP request from 10.31.69.85, invoking
com.emc.documentum.server.method.rename.UserRename.execute(), ARGUMENTS []
03:09:28,278 INFO [stdout] (http--0.0.0.0-8080-3) <DCTM><INFO> [http--0.0.0.0-8080-3:Method
name=com.emc.documentum.server.method.rename.UserRename] Completed executing. Total Time Taken: 0 secs

Docbase.log
013-10-22T03:08:58.072000 5596[7388]

0100007b800025a3

JMS DO_METHOD TRACE LAUNCH: user:

Administrator, session id: 0100007b800025a3, JMS id: 0800007b80000cc7, method: dm_UserRename_Java,

host:csd70.emcccsa.com, port:9080, path:/DmMethods/servlet/DoMethod, arguments:-method_verb
com.emc.documentum.server.method.rename.UserRename -__dm_docbase__ Repo1 -__dm_server_config__ csd70_Repo12
2013-10-22T03:08:59.121000 5596[7388]

0100007b800025a3

JMS DO_METHOD TRACE LAUNCH: do_method

launch: failed: user: Administrator, session id: 0100007b800025a3, JMS id: 0800007b80000cc7, method:
dm_UserRename_Java, host:csd70.emcccsa.com, port:9080, path:/DmMethods/servlet/DoMethod
2013-10-22T03:08:59.127000 5596[7388]

0100007b800025a3

JMS TRACE LAUNCH: Failed to post to JMS

0800007b80000cc7 (JMS csd70.emcccsa.com:9080 for Repo1.Repo1): Error is "Failed to obtain socket for
host:csd70.emcccsa.com, at port:9080", :Marking this JMS config as dead server cache.
2013-10-22T03:08:59.127000 5596[7388]

0100007b800025a3

JMS DO_METHOD TRACE LAUNCH: user:

Administrator, session id: 0100007b800025a3, JMS id: 0800007b8000151e, method: dm_UserRename_Java,

host:csd70.emcccsa.com, port:8080, path:/DmMethods/servlet/DoMethod, arguments:-method_verb
com.emc.documentum.server.method.rename.UserRename -__dm_docbase__ Repo1 -__dm_server_config__ csd70_Repo12
2013-10-22T03:09:00.241000 5596[7388]

0100007b800025a3

JMS DO_METHOD TRACE LAUNCH: do_method

launch: successful: user: Administrator, session id: 0100007b800025a3, JMS id: 0800007b8000151e, method:
dm_UserRename_Java, host:csd70.emcccsa.com, port:8080, path:/DmMethods/servlet/DoMethod
2013-10-22T03:09:00.286000 5596[7388]

0100007b800025a3

DO_METHOD TRACE LAUNCH: method launch:

successful, user: Administrator, session id: 0100007b800025a3, method: dm_UserRename_Java

24

System Methods details

Below is the list of system methods and supportability for failover.
Method Name

JMS Location Constrain

dm_DistOperations_dmcl40

JMS_LOCATION=MAIN

dm_RefreshReplicaMethod

JMS_LOCATION=MAIN

dm_bp_transition_java

JMS_LOCATION=ANY

dm_bp_schedule_java

JMS_LOCATION=ANY

dm_bp_batch_java

JMS_LOCATION=ANY

dm_bp_validate_java

JMS_LOCATION=ANY

dmc_MigrateDbExprsToJava

JMS_LOCATION=ANY

dmc_MigrateDbExprsToJavaForType

JMS_LOCATION=ANY

dmc_SetJavaExprEnabled

LOCAL_ONLY

dmc_AddCustomJavaExpr

LOCAL_ONLY

CTSAdminMethod

JMS_LOCATION=ANY

dm_ldif_export

NA (does not use java method server)

dm_ldif_copy

NA (does not use java method server)

dm_ldif_import

NA (does not use java method server)

dm_ldif_status

NA (does not use java method server)

dm_LDAPSynchronization

LOCAL_ONLY

LDAPSync

LOCAL_ONLY

LDAPSyncOnDemand

LOCAL_ONLY

dm_LDAPCertDbAutomation

LOCAL_ONLY

dm_dcfEdit

LOCAL_ONLY

dm_run_dependent_jobs

LOCAL_ONLY

dm_getClientDocbrokers

LOCAL_ONLY

dm_webcache_publish

LOCAL_ONLY

dm_bpm_transition

JMS_LOCATION=ANY

dm_bpm_timer

JMS_LOCATION=ANY

dm_WfmsTimer

JMS_LOCATION=ANY

TCFActivityMethod

JMS_LOCATION=ANY

TCFWorkitemActivityMethod

JMS_LOCATION=ANY

dm_FTCreateEvents

JMS_LOCATION=ANY

dm_FTStateOfIndex

JMS_LOCATION=ANY

dm_FTIndexAgentBoot

JMS_LOCATION=ANY

dm_FTOptimizeIndex

LOCAL_ONLY

dm_event_template_sender

JMS_LOCATION=ANY

dm_AsynchronousWrite

JMS_LOCATION=ANY

dm_PreCacheContent

JMS_LOCATION=ANY

dm_noop_auto_method_java

JMS_LOCATION=ANY

BPMCreateFolder

JMS_LOCATION=ANY

25

BPMLifecycleAttach

JMS_LOCATION=ANY

BPMLifecycleDemote

JMS_LOCATION=ANY

BPMLifecyclePromote

JMS_LOCATION=ANY

BPMLinkToFolder

JMS_LOCATION=ANY

BPSFTIntegration

JMS_LOCATION=ANY

BPSHTTPIntegration

JMS_LOCATION=ANY

BPSJMSIntegration

JMS_LOCATION=ANY

BPSMQJMSIntegration

JMS_LOCATION=ANY

BPSSMTPIntegration

JMS_LOCATION=ANY

BPSWebServiceIntegration

JMS_LOCATION=ANY

BPSIntegration

JMS_LOCATION=ANY

dm_bps_mail

JMS_LOCATION=ANY

dm_QmCheckVerificationThreshold

JMS_LOCATION=ANY

dm_QmPriorityAging

JMS_LOCATION=ANY

dm_QmPriorityNotification

JMS_LOCATION=ANY

dm_QmSetRequiredSkill

JMS_LOCATION=ANY

dm_QmThresholdNotification

JMS_LOCATION=ANY

dm_WFCheckPoint

JMS_LOCATION=ANY

dm_WFExportData

JMS_LOCATION=ANY

dm_WFReporting

JMS_LOCATION=ANY

dm_WFReportPackageData

JMS_LOCATION=ANY

dm_WFSuspendTimer

JMS_LOCATION=ANY

DmSampleActivityTemplate

JMS_LOCATION=ANY

ECISMethod

JMS_LOCATION=ANY

ObservationPoint

JMS_LOCATION=ANY

PostEventtoParentWorkflow

JMS_LOCATION=ANY

StartSub-Process

JMS_LOCATION=ANY

XSLTransformation

JMS_LOCATION=ANY

Options for method

For a method to support JMS failover set the below parameters:

o

Is_restartable to True (default is false)

a_extended_properties[0] to JMS_LOCATION=ANY or leave it blank (default value is blank)

Verification steps

Configure a Java Method to support JMS failover.

Bring down first JMS.

Run the Method from first Content Server.

Method should be executed via second JMS. Verify the logs. Logs for second JMS would contain the method execution
information.

26

Bring down second JMS.

Run the Method from second Content Server.

Method should be executed via first JMS. Verify the logs. Logs for first JMS would contain the method execution information.

Limitations

Not all java methods can be failover

o

Some java method must be run on primary content server, some must be run on same host as originating content
servers host

Default JMS for Content File Server/Remote Content Server (aka RCS) do not participate in JMS failover

JMS node must be bound to a content server host

o

It cannot be installed on a stand-alone independent machine

We do not support JMS load balancing. A single content server cannot be explicitly associated to multiple Java method
servers.

Limitations inherited from single JMS model

o

JBoss cannot handle large number of HTTP POST requests

If JMS is crashed during ASYNC method execution, content server cannot detect the crash, hence will not failover to
another JMS

Custom Method configurations

If custom methods wish to take advantage of the JMS failover feature, make sure the method is failover aware. Set
Is_restartable parameter to True (default is false

If the method is failover aware, but method must be run on a JMS thats on the same host as originating content server host,
then set a_extended_properties[0] = JMS_LOCATION=ORIGINAL

If method can be run on any CS-bound JMS, then set a_extended_properties[0] = JMS_LOCATION=MAIN

If method can be run on any JMS, then leave a_extended_properties attribute blank or assign value JMS_LOCATION=ANY. If
left blank, it is assume the method can be run on any JMS. Default value is blank.

27

TYPE2: TWO HOST HA SETUP

JMS HA on two hosts

Two Host HA Setup the primary and secondary Content Servers are installed on different hosts.
On the Second Content Server, the second content server has to be installed and configured.
For creating a second JMS, the same steps as single host JMS HA are followed. During the screen Is Primary Content Server,
select the option NO.
To Install Content Server in HA refer to Content_Server_Enterprise_Edition_7.0_Installation_Guide.

Digital Signature Verification

In a single JMS model, JMS and CS are always hosted on same machine.

Before running the Java method, to prevent malicious cyber-attacks, to make sure the HTTP POST request is legit, JMS
Do_Method servlet performs IP validation to make sure request is originated from the same host as JMSs host.

IP validation would always succeed in single JMS model environment.

In a JMS failover environment, IP validation would fail if content server tries to send method requests to a failover JMS on a
different host.

28

To address this IP security restriction, weve introduced digital signature validation instead.

6.6 onwards, content Server generates digital signature, JMS verifies it

Signature is signed by using the repositorys private key, and is then verified using the repositorys public key.

JMS uses the following parameters :

$Documentum_shared\JBOSSx.x\server\DctmServer_MethodServer\deploy\ServerApps.ear\DmMethods.war\WEB-INF, make
sure the init value for docbase and docbase_install_owner_name are present in web.xml.
<init-param>
<param-name>docbase_install_owner_name</param-name>
<param-value>xxxxxx</param-value>
</init-param>
<init-param>
<param-name>docbase</param-name>
<param-value>xxxx</param-value>
</init-param>

29

TYPE3: MULTIPLE HOSTS SETUP

JMS HA on Mulitple hosts

More than 2 Content servers can be clustered. Similarly the Java method server associated with each Content Server can be
configured for High Availability.
Same steps are followed for configuring the JMS HA as for 2 nodes.

30

STEPS FOR CONFIGURING JMS HA 6.7 SP2

TYPE1:SINGLE HOST SETUPFOR 6.7 SP2
1. Preparing for Installation/Prerequisite:
Content Server in HA
To Achieve JMS HA, a second (or another) Content Server associated with JMS. Single Content server does not support multiple
JMS. To Install Content Server in HA refer to Content_Server_Enterprise_Edition_6.7_Installation_Guide.

Packaging WebApps
The WebApps are deployed on the first/primary JMS. Using the jmsPackager.bat, package WebApps to be deployed on the
second/additional JMS.
ServerApps.ear - WebApp has to be deployed for the JMS server.
In the webappsToExclude.txt, add the names of the WebApps to exclude from packaging. This file is found under
$Documentum\jmsTools\config\webappsToExclude.txt.
Run jmsPackager.bat file under $Documentum\jmsTools\bin.
After the Packaging is complete the WebApps are located at $Documentum\jmsTools\webapps\jmsWebApps.jar.
Copy the jmsWebApps.jar file on the machine to install second JMS under $Documentum\jmsTools\webapps path.

2. Creating second JMS Instance

The steps are similar as for creating second JMS instance for 7.0.

Run the jmsconfig.exe under $Documentum\jmsTools\bin folder.

Select Create Instance and click Next.

Provide JMS Instance Name

Set the password for the Application Server - Admin User.

Provide the port details to deploy JMS. JMS requires consecutive 20 ports.
By default 9080 to 9099 ports will be selected for JMS. Port numbers can be changed, ensure consecutive 20 ports are
available.

31

On the primary Installation owner screen :

Provide the Installation Owner: Username and Password.

Click on Next.

The WebApps deploy and configuration process completes. Click on Done.

3. Verification of Instance and Associating Content Server with JMS Instance

Verify the JMS Instance following the same steps as Single Host JMS HA for 7.0
Associate the Content Servers with JMS Instances, same steps as Single Host JMS HA for 7.0 is followed.

TYPE 2: TWO OR MULTI HOST SETUP FOR 7.0

Two Host HA Setup the primary and secondary Content Servers are installed on different hosts.
On the Second Content Server, the second content server has to be installed and configured.
For creating a second JMS, the same steps as single host JMS HA for 6.7 Sp2 are followed.

STEPS FOR CONFIGURING JMS HA 7.1

To Install Content Server in HA refer to Content_Server_Enterprise_Edition_7.1_Installation_Guide.
For Single Host Setup, the same steps as for configuring JMS HA 7.0 are followed.
For Two or more host setup, the Java Method Server Instance is created by the installer. Associate the Content Servers with JMS
Instances, same steps as for configuring JMS HA 7.0 is followed.

32

CONCLUSION
This whitepaper explains configuration of JMS High Availability with failover support setup. Describes the steps to enable
methods for failover and track the execution of methods from logs.

References

Documentum Content Server 7.0 Installation Guide

EMC Documentum Content Server Load Balance and FailOver

EMC Documentum Content Server 7.0 Administration and Configuration Guide

CONTACT US
To learn more about how
EMC products, services, and
solutions can help solve your
business and IT challenges,
contact your local
representative or authorized
reselleror visit us at
www.EMC.com.

Copyright 2013 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information
is subject to change without notice.
The information in this publication is provided as is. EMC Corporation makes no representations or
warranties of any kind with respect to the information in this publication, and specifically disclaims
implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any EMC software described in this publication requires an
applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on
EMC.com.

33
www.EMC.com

EMC2, EMC, the EMC logo, and the RSA logo are registered trademarks or trademarks of EMC
Corporation in the United States and other countries. VMware a registered trademark of VMware,
Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property
of their respective owners. Published in the USA. 08/12 White Paper H12673