Performance Testing Requirements

Questionnaire Document
(Proprietary and Confidential)

Introduction: Apache JMeter is a 100% pure Java desktop application designed to load test
client/server software (such as a web application). JMeter can be used to simulate a
heavy load on a server, network or object to test its strength or to analyze overall
performance under different load types. Additionally, Jmeter can helps you regression
test your application by create test scripts with assertions (content check points) to
validate that your application is returning the results you expect

Pre-requisites: Install latest version of JAVA/JRE 1.5 or later followed by set the JAVA bin and
JAVA_HOME path as shown below
My ComputersRight ClickPropertiesAdvance TabEnvironment Variables

Jmeter HTTP Performance Testing Process

Page 2 of 32

Getting started: 1) Download

the

zipped

binary(Apache

JMeter

2.7)

from

http://jakarta.apache.org/jmeter/
2) Extract the ZIP files to any specified location in the local host
3) On Windows, launch JMeter by running jmeter.bat

4) After running jmeter.bat, Jmeter console will open up by default Test Plan,
Workbench nodes shown up as seen in the below snapshot

Jmeter HTTP Performance Testing Process

Page 3 of 32

Note: TestPlan Node Permanent workspace for created scripts (Refer Terminology section for Test
Plan)
Workbench
Workbench)

Temperary work space for creating scripts (Refer Terminology section for

Jmeter supports the blow list of protocols for performance testing of any web/client
server applications
HTTP/HTTPS, FTP, JDBC, JMS, LDAP, SOAP, RMI
This whitepaper will explains the HTTP/HTTPS performance testing process for Google
Web Search Application with effective utilization of Jmeter features/add-ons facility

What is HTTP/HTTPS Protocol: HTTP Protocol stands for Hyper Text Transfer Protocol. It is the protocol used to convey
information

of

World

Wide

Web

(WWW).

HTTP protocol

is a

stateless and

connectionless protocol. HTTP/HTTPS protocol is based on a request/response

paradigm. In this protocol the communication generally takes place over a TCP/IP
protocol

Jmeters Terminology: Before we go further it would probably be helpful to discuss some basic Jmeter
terminology. You may also want to refer to the Jmeter GUI for more information
Thread: For all intents and purposes, a thread can be thought of as a user
(virtual user in LR)
Thread Group: Think of a thread group as the scenario. A Business Scenario
that has a user log in to Web Tours application, book a flight, and logout might
be recorded in a test script called one thread group. During a test there might
be one or several hundred users executing a thread group. The thread group
might contain conditional logic that loops through parts of the web application
Test plan: A test plan is one or more thread groups. In the above, we discussed
a thread group that performing some actions as part of Booking a Flight
Jmeter HTTP Performance Testing Process

Page 4 of 32

scenario. Like this if we have multiple Business Scenarios can also be called as
Test Plan
Sampler: A sampler is something that sends a request to a server. There are
many types of samplers for all the protocol. One can Create custom request for
specific protocol (web_custom_request)

Listener: A listener listens to the responses generated by samplers. Listeners

can be used to compile the results of the test run and to check that a sampler
generates the expected response

Assertion: An Assertion is used to specify what a response should or should

not contain. There are several different types of assertions depending on what
kind of response you are expecting and what sort of check you'd like to perform
(web_reg_find in LR)
Logic Controller: Logic controllers provide a mechanism for controlling the
flow of the thread group. Adding a logic controller to your thread group is like
adding if-then or do-while logic to a program. Your test script can make run-time
decisions about how to execute. Suppose you want to open every object in a list
but you do not know how many objects will be in the list ahead of time. A "Loop
Controller" could be used to execute an action (opening an object) a certain
number of times without requiring you to hard-code the requests into the script
Workbench: The Workbench is like a working area. When recording scripts,
JMeter places the automatically-generated samplers under the workbench. They
will not get executed as part of a thread group until you move them from the
workbench to a thread group (Virtual User Generator)
Transaction Controller: The Transaction Controller generates an additional
sample which measures the overall time taken to perform the nested test
elements. Note that this time by default includes all processing within the
controller scope, not just the samples

JMeter Test Plan Process:

Jmeter HTTP Performance Testing Process

Page 5 of 32

There are several steps for achieving end-to-end web performance testing using HTTP
Sampler

Test Plan Creation: 1) Add a Thread Group to Test Plan

(Right click on Test Plan and select Add Threads (Users) Thread Group)

2) Add HTTP Proxy server to the Workbench

(Right click on WorkBench and select Add Non-Test Elements HTTP Proxy Server)

Jmeter HTTP Performance Testing Process

Page 6 of 32

3) HTTP Proxy Server node is added under WorkBench node

Jmeter HTTP Performance Testing Process

Page 7 of 32

4) Open the internet browser and configure the proxy setting as shown in below

snapshot. Make sure that the port number should be same as the port number
given in the Global Settings in Jmeter.
ToolsInternet OptionsConnectionsLAN Settings

5) Select Put each group in new transaction controller as Grouping option. Therefore

every single click can be recorded as one business transaction (lr_start_transaction

in LR)

6) Select the HTTP Proxy Server node under WorkBench

Select HttpClient4 as Type in HTTP Sampler Settings (Recording Options in LR)

Jmeter HTTP Performance Testing Process

Page 8 of 32

Select

TestPlan > Thread Group as Target Controller in Test plan content

(lr_start_transaction in LR)
Click on Start button

7) Now

start

recording

by

opening

the

browser

and

enter

http://www.google.com which will access the Google Home page

8) Enter any search string in the text box and click on Google Search

Jmeter HTTP Performance Testing Process

Page 9 of 32

the

URL

9) Click on first link

10)

Select the HTTP Proxy Server node under WorkBench and click on Stop button

Jmeter HTTP Performance Testing Process

Page 10 of 32

Till the above step, Recording process is completed.

11)

All the above business scenario can be recorded in jmeter as shown in the

below snapshot, where all the Transaction Controller names can be renamed in the
following way:
/

01_Google_Homepage

/search

/url

03_Google_ClickfirstLink

02_Google_Search and

Correlation:
Correlation is the most important aspect of scripting. It generally includes fetching
dynamic data from preceding requests/calls and posting it to the subsequent requests
1) Correlate the URL (http://jmeter.apache.org/) URL(dynamic content) will be
different for every iteration based on Search string

Jmeter HTTP Performance Testing Process

Page 11 of 32

2) Copy the URL(Dynamic Content)and go to Response tab (Text mode) in View

Result Tree listener of search operation. In order to see the Response Data in
View Results Tree listerner, we need to execute script by at least one time
followed by adding View Results Tree listener
Right click on Thread Group Listerners View Results Tree

Note:- In LR, there is a Tree View mode to see the server response where as in
Jmeter, Thread Group (script) should be executed at least once to see the server
response

3) Dynamic
Content
from
q=http://jmeter.apache.org/&

Response

tab

is

<a

href="/url?

Capture the Dynamic value that is to be correlated along with its left and right
boundaries. In this example
Jmeter HTTP Performance Testing Process

Page 12 of 32

Dynamic Value: -

http://jmeter.apache.org/

Left bound: -

<a href="/url\?q=

Right bound: -

&

Replace the dynamic value with a regular expression i.e., (.*?) by adding Regular
Expression Extractor as explained in the next step
RegularExpression:

<a href="/url\?q=(.*?)&

The regular expression is used to parse the response data. This must contain at least
one set of parentheses "()" to capture a portion of the string
4) Go to http request under 02_Google_Search transaction controller->
Add -> Post Processor -> Regular Expression Extractor

5) Select the Regular Expression Extractor and enter the details such as Reference
name, Regular Expression, Template, Match No. as shown in the below snapshot.

Jmeter HTTP Performance Testing Process

Page 13 of 32

Reference Name: It is the name of the JMeter variable which stores the dynamic
value.
Match No. : Indicates which match to use. The regular expression may match
multiple times.

Use a value of zero to indicate JMeter should choose a match at

random.
A positive number N means to select the nth match.
Negative numbers are used in conjunction with the ForEach controller
Default Value: If the regular expression does not match, then the reference variable
will be set to the default value. This is particularly useful for debugging tests
Template: attribute should be default as $1$

A small example to understand depth of Correlation : (Impact of

Template Attribute)

Some times we want to capture multiple strings which are changing dynamically in the
same LINE OF CODE like;
<name=jojo&amp;age=25&amp;height=5.7&end> (RED colored indicates
the dynamica values which need to be captured in the same Reference
Name)
RegularExpression Declaration is :
<name=(.*?)&amp;age=(.*?)&amp;height=(.*?)&end>

Dynamic values are stored in the variable names as shown below

Jmeter HTTP Performance Testing Process

Page 14 of 32

test_g1=
test_g2=
test_g3=
test_g=

jojo
name
25 age
5.7
height
3 Number of groups in regular expression

Parameterization:
Parameterize the search string Jmeter

1) Add CSV Data set Config element to the thread group.

AddConfig ElementCSV Data Set Config

2) Open a notepad (inside bin) and enter your Search Strings. Save the file as

search.csv. The extension csv makes it a comma separated Value (CSV) file.
Jmeter HTTP Performance Testing Process

Page 15 of 32

3) Select the CSV Data set Config element and fill in the fields as follows

Filename: search.csv
Variable Name: Search String

Jmeter HTTP Performance Testing Process

Page 16 of 32

4) Go to 02_GoogleSearch transaction and change the value of variable q and oq

to ${keyword}. For each iteration the value of q and oq will be replaced as

fetched by variable Search String

Assertions: (web_reg_find in LR)

Assertions allows us to assert facts about responses received from the server being
tested
Add an assertion for the HomePage page.
1) Right-click the sampler named " / " then click AddAssertionResponse Assertion

Jmeter HTTP Performance Testing Process

Page 17 of 32

2) Add " Advanced search " as a check point in the Pattern to Text field

2. Now enter
the pattern to
test

1. Click on add
button

3) To know when an assertion fails add an Assertion Listener (Error Handling)

( Right-click on Thread Group AddListener Assertion Results)

Jmeter HTTP Performance Testing Process

Page 18 of 32

4) Run your script. If any of the assertions fail, a message will show up in the

Assertion Results listener as shown in below snapshot (lr_output_message &

lr_error_message in LR)

Jmeter HTTP Performance Testing Process

Page 19 of 32

Cookie Manager:
Many Web applications use cookies. JMeter provides cookie capabilities through a
Cookie Manager. Adding this element to a thread group allows us to send cookies to
the application being tested, just as Web browsers do.
Add HTTP Cookie Manager(Right click the Thread Group and select Add Config
ElementHTTP Cookie Manager) and set Cookie policy of the cookie manager to be

compatibility. Remember also to set the "Clear cookies each iteration" setting
to checked

Timers: (lr_think_time in LR)

By default jmeter thread sends request without pausing between each reques.It is
recomended to set delay by adding timers to the thread group. Timers are used to
create delays between requests made by each simulated user. Without a timer, your
requests hit the server back to back, as fast as possible. Timers allow for a more
realistic load on the server. There are different types of timers that allow you to
produce either constant or random delays.If you want to have each thread pause for
the same amount of time between requests, use Constant Timer

Jmeter HTTP Performance Testing Process

Page 20 of 32

Give the constant delay as shown in below snapshot

User Defined Variables:

User Defined Variable element is used to initialize set of variables.
Add User Defined Variables element under Thread Group
(RightClick on Thread Group AddConfig ElementUser Defined Variables)

In our example we can define the Server Name as User Defined Variable.

Jmeter HTTP Performance Testing Process

Page 21 of 32

Go to User Defined Variables element and fill the fields as shown in the below
snapshot

Replace www.google.co.in with ${ServerName} in HTTP Request as show in the

below snapshot

After above enhancements, execute the script for multiple iteration to validate the
funcitonlity of the application
1) Add View Results Tree Listener element to the thread group
Add->Listener->View Results Tree
2) Run the script(ctr+R) and observe the result in result tree

Jmeter HTTP Performance Testing Process

Page 22 of 32

Results Output:- Every iteration it will fetch different search string followed by
accessing Their respected search string First Link only

Thread Group:
A Thread Group defines a pool of users that will execute a particular test case against
your server. In the Thread Group GUI, you can control the number of threads (number
of users), the ramp ramp-up period (how long it takes to start all the threads), The
loop count (how many times the thread group will be executed), and optionally, a start
and stop time for the test.

Jmeter HTTP Performance Testing Process

Page 23 of 32

SetUp Thread Group:

A special type of thread group that can be utilized to perform Pre-Test Actions. The
behavior of these threads is exactly like a normal Thread Group element. The
difference is that these types of threads execute before the test proceeds to the
executing of regular Thread Groups.
Example: - Consider two thread groups of Web tours scenario i.e. Flight creation and
Flight deletion. Therefore, we need to execute the test plan with these two thread
groups. The first thread group is Flight creation, where the user reserves the flights.
The second thread group is Flight deletion, where the user cancels all the reserved
flights. In this case, the thread group Flight deletion must have always a specific
number of reserved flights to be cancelled. So to schedule a test plan with these two
thread groups, we have to make the Flight creation thread group as Setup Thread
Group, where a specific number of flights will be reserved by the user and the thread
Jmeter HTTP Performance Testing Process

Page 24 of 32

group Flight deletion can be made as actual thread group. Therefore, when the test
plan is scheduled Flight creation thread group will execute followed by Flight Deletion
thread group will be executed where the Created Flights will be DELETED.

Teardown Thread Group:

A special type of Thread Group that can be utilized to perform Post-Test Actions. The
behavior of these threads is exactly like a normal Thread Group element. The
difference is that these types of threads execute after the test has finished executing
its regular Thread Groups. In other words we can call it as vice versa of setup thread
group.

Remote Load Testing: Step 1: Configure the nodes

Same version of Java installed on all slave machines for best recommendation

Configure the CSV param files in the appropriate directory on Jmeter Master
Machine

Configure the number of Load Generators machine IPs in the properties.txt to

simulate the Threads activity during the test. These properties will be picked up

Jmeter HTTP Performance Testing Process

Page 25 of 32

when the server is started and may be used in the test plan to affect its behavior
(e.g. connecting to a different remote server)
Step 2: Start the servers
To run JMeter in remote node, start the JMeter server component on all machines you
wish to run on by running the JMETER_HOME/bin/jmeter-server.bat (windows)
script.
Step 3: Add the server IP to your client's Properties File
Edit the properties file on the controlling JMeter machine. In /bin/jmeter.properties, find
the property named, "remote hosts", and add the value of running Jmeter slave
server's IP address. Multiple such salve servers can be added, comma-delimited. If we
define the JMeter property server.exitaftertest=true, then the server will exit after it
runs a single test.
Step 4: Start the JMeter Client from a GUI client
Now you are ready to start the controlling JMeter client. For MS-Windows, start the
client with the script "bin/jmeter.bat". You will notice that the Run menu contains two
new sub-menus: "Remote Start" and "Remote Stop" (see below figure). These menus
contain the client that you set in the properties file. Use the remote start and stop
instead of the normal JMeter start and stop menu items

Figure 1 - Run Menu

Add On to Jmeter:
Jmeter has several add-ons which will be providing the options as compare with
Licensed performance test tools.
UniversalServerAgent : - Windows and Unix servers monitoring
Jmeter HTTP Performance Testing Process

Page 26 of 32

Stepping Thread Group, Utilmate thread group : - Types of Thread groups for
simulating different load test ramp up patterns during test
Active thread over time : - Type of Listener to see the number of users status during
test
Discussed above add-ons installtion procedure followed by their Features;
Copying JMeterPlugins.jar file inside the JMETER_INSTALL_DIR/lib/ext. We can
observe two thread groups (ie.,Stepping Thread Group,Ultimate Thread Group) are
added to Test Plan.

Stepping Thread Group:

JMeter have only one out-of-the-box option for threads (users) scheduling: simple
ramp-up. But many users, especially with HP LoadRunner experience miss more
flexible thread scheduling algorythm. Stepping Thread Group adds to JMeter thread
scheduling similar to LoadRunner's. It allows to set up the Ramp up, Ramp down,
Duration explicitly where as in Normal Thread Group allows ONLY to configure Duration
along with the starting of Total Threads
Snapshot as follows

Ultimate Thread Group: It allows to seeing the expected behavior of the test which was designed for different
Ramp up, Duration, Ramp down for single thread group
Jmeter HTTP Performance Testing Process

Page 27 of 32

Ex: - If we have single Thread group (one script) whereas different scheduler needs it
(50% of threads simulate different ramp up, duration, ramp down from total
number of threads)

Active Threads Over Time: It allows seeing the Threads (Virtual Users) running status as per design the load
pattern schedulers in Normal Thread or Stepping up thread group or Set up or Tear
Down thread groups (Virtual Users graph in LR)

Listeners:

Jmeter HTTP Performance Testing Process

Page 28 of 32

These are used to view and store the results of our test plan.Running a load test
without capturing performance metrics is kind of pointless. JMeter offers a variety of
listeners to capture metrics.
Right click Thread Group then click Add ListenerView Results Tree

Similarly we can add the listners such as View Resuits Tree,Summary Reports,
Aggregate Reports etc.
View Results Tree(Tree view in LR)

View Results in Table

Jmeter HTTP Performance Testing Process

Page 29 of 32

Summary Report

Some additional listeners are added to the Jmeter GUI by copying JMeterPlugins.jar file
from JMeterPlugins-VERSION.zip to JMETER_INSTALLED_DIRECTORY/lib/ext. Some of the
listeners are listed below
jp@gc - PerfMon Metrics Collector: By using this listener we can monitor the CPU
utilization, Memory, Swap, Disks I/O, Networks I/O and so on. For monitoring CPU
utilization, Memory, Disk I/O and so on, we have to start the startAgent.bat file in
JMeterPlugins-0.5.1\ServerAgent

Jmeter HTTP Performance Testing Process

Page 30 of 32

jp@gc - Active Threads over Time: Active Threads over Time is a listener showing
how many active threads are there in each thread group during test run. The plugin
produces graph like shown below:
jp@gc - Auto Stop Listener: Auto Stop Listener is used when we want to stop the
test on some runtime criteria. Currently 3 criteria available: average response time,
average latency and error rate. These criteria are used in OR logic, the component
will ask JMeter to stop test if one of the criteria has been met.
i)

Auto Stop on Response Time/Latency: There is combo-box switching which

result parameter to use in conditions: response time or latency. Test will be
stopped only if specified response time/latency exceeded for sequentially N
seconds. To disable auto-stop on time criteria, just set time value to zero

ii)

Auto Stop on Error Rate: Error rate specified in percent. Rate can be float
number. Test will be stopped only if specified error rate exceeded for
sequentially N seconds. To disable auto-stop on rate criteria, just set error
rate to zero

jp@gc - Bytes Throughput Over Time: This graph will display the amount of bytes
sent and received by JMeter during the load test

Jmeter HTTP Performance Testing Process

Page 31 of 32

jp@gc - Hits per Second: This graph will display the hits generated by the test plan
to the server per second
jp@gc - Transactions per Second: This graph shows the number of transactions per
second for each sampler

Jmeter HTTP Performance Testing Process

Page 32 of 32