Session 21-23

Experimentation
Experimentation
Copyright Dorling Kindersley India Pvt. Ltd
C
p
a
h
8
r
te
Learning Objectives
Understand the concept of experimentation and

causality
Understand the internal and external validity
issues in experimentation
Understand the threats to internal and external
validity in experimentation
Understand the ways to control extraneous
variables
Understand and compare a laboratory experiment
and field experiment
Understand the concept of experimental designs
and deal with various types of experimental
designs
Understand the limitations
of experimentation
Experimentation
Upon completion of this chapter, you will be able to:
Experiments can be defined as the systematic

study in which a researcher controls or
manipulates
one
or
more
independent
(experiment) variables to test a hypothesis about
the independent variable.
Independent variables are the variables that are
manipulated or controlled by the researcher.
On the other hand, dependent variables are
variables on which a researcher has little or no
control over the research process, but the
researcher is keen to note a change in it with
the corresponding change in the independent
variables.
Tull and Hawkins (1984) stated that the essence of
experimentation is the manipulation of one or
more variables by Experimentation
the experimenter in such a
Defining Experiments
Concept of Causality
Before detailing experimentation, we will first focus

on the concept of causality, a prerequisite for
experimentation. We do things in the world by
exploiting our knowledge of what cause what
(Hobbs, 2005).
In fact, causality is a conditional phenomenon
between variables in the form if x, then y.
Causality is an important aspect of how we
construct reality (Cavazza et al., 2007).
There exist four formal conditions for causality:
covariation, time order of occurrence of
variable, systematic elimination of other
causal variable, and experimental designs.
Experimentation
Covariation is the extent to which a caused

variable occurs with the causal variable
together or vary together as the framed
hypothesis under consideration. For example,
sales (dependent variable) move upward with the
lowered status of the price (independent variable).
The second condition, time order of occurrence
of variable, explains that the causal variable
changes prior to or simultaneously with the caused
variable; hence, it cannot occur afterwards. For
example, when prices are increased on the first day
of the month, sales go down on the remaining days
of the month.
The third condition, systematic elimination of other
causal variables, indicates that the variable being
investigated should be the only causal explanation
Experimentation
of any change in the dependent variable.
Concept of causality (Cont.)
O = Observation or measurement made on

dependent variable as a part of the experiment
X = Exposure of the test unit under investigation to
independent variable or experimental manipulation
or treatment
EG = Experimental group that is exposed to the
experimental manipulation or treatment
CG = Control group participating in the experiment
but has no exposure to the experimental
manipulation or treatment
R = Random assignment of test units and
experimental manipulation or treatments to the
groups
M = Match of experimental group and control group
Experimentation
Some Basic Symbols and Notations

in Conducting Experiments
Internal and External Validity in the

Experimentation
If the design and structure of a study are such that
one can confidently conclude
that
the
independent
variable
caused
systematic
changes in the dependent variable, then the
study is said to have a high internal validity.
On the other hand, if the study gives us a plausible
alternative interpretation of the observed
relationship between the independent and
dependent variable, then it is said to have a low
internal validity.
The external validity typically refers to the
generalizability of the results of a study to other
(usually real world) settings or populations
(Anderson & Bushman, 1997).
Experimentation
The problems of the external validity generally

relate to the possibility that a specific, but time
limited, set of experimental conditions may
not deal with the interpretations of untested
variables in the real world (Zikmund, 2007).
The lack of external validity puts a researcher in
a difficult situation to repeat the experiment for a
different set of subjects in different time span. If an
experiment lacks the internal validity, then there
is no meaning in generalizing the result of the
experiment.
Factors that are of serious concern for the
internal validity of the experiment may also
jeopardize the external validity of the
experiment. The major
source of threat to the
Experimentation
Internal and External Validity in the

Experimentation (cont.)
Threats to the Internal Validity

of the Experiment
History
Maturation
Testing
Instrumentation
statistical regression
selection bias
mortality
Experimentation
History effect refers to a specific event in the

external environment that occurs between the
commencements of experiment and when the
experiment ends.
In an experiment, maturation takes place when
the
subjects
become
older,
bored,
experienced, or
disinterested during the
experiment.
A testing effect occurs when a pre-test
measurement sensitizes the subjects to the
nature of the experiment.
The instrumentation effect is said to be occurred
in an experiment when either the measuring
instrument or the observer changes during
Experimentation
the experiment.

of the Experiment
10

of the Experiment
Statistical regression is the tendency of the
subjects with extreme scores to migrate (regress)
towards the average scores during the experiment.
Selection bias occurs when an experimental
group significantly differs from the target
population or control group.
Mortality effect occurs when the subjects drop
out while the experiment is in progress. The
subjects refuse to participate in the experiment
because of various reasons such as lack of time, loss
of interest in the experiment, and so on.
Experimentation
11
Threats to the External Validity

of the Experiment
If an experiment lacks the internal validity,
then there is no meaning in generalizing the
result of the experiment.
In other words, the internal validity can be
viewed as a necessary, but not sufficient,
condition for external validity (Parasuraman et
al., 2004).
Four biases, such as reactive effect, interaction
bias, multiple treatment effect, and nonrepresentativeness of the samples, that
seriously jeopardize the external validity of an
experiment.
Experimentation
12
Reactive effect occurs when the respondents

exhibit an unusual behaviour knowing that they are
participating in an experiment.
Interaction bias occurs when a pre-test increases
or decreases the sensitization of the respondent to
the experimental treatment.
Multiple treatment effect occurs when a
participant is exposed to multiple treatments.
This is quite obvious that the impact of previous
treatments cannot be fully erased from the
mind of the respondents.
Sometimes, it happens that a researcher
knowingly or unknowingly selects the subjects
who may not be a true representative of the
Experimentation
population. When this
happens in an experiment, it
Threats to the External Validity

of the Experiment
13
Ways to Control Extraneous Variables

Presence of extraneous variable poses a
serious threat to the internal as well as external
validity of the experiment.
The four ways to control the extraneous variable are
Randomization
Matching
Statistical control
Design control
Experimentation
14
Randomization refers to the random assignment of

the subjects and experimental treatment to
experimental group to equally distribute the effect of
extraneous variables.
When a researcher suspects that the extraneous
variables may affect the dependent variable, he or
she applies the technique of Matching, which
involves matching each group on some pertinent
characteristics or some pertinent background
variables.
With the help of a Statistical control, a researcher
measures the effect of extraneous variable and
adjusts its impact with a sophisticated statistical
analysis. In this context, a statistical technique such
as ANCOVA can be applied.
Experimentation
Ways to Control Extraneous Variables

(cont.)
15
Laboratory Versus Field Experiment
The laboratory experiment is conducted in a

laboratory or artificial setting. A researcher applies
or controls the experimental manipulation or
treatment in an artificial environment.
A field experiment is conducted in the field or a
natural setting. In the field experiment, the effect of
experimental manipulation or independent variables
on dependent variable is observed in a natural
setting.
The main objective of experimentation is to detect
or confirm the causal relationship between the
dependent variable and independent variables and
quantify them.
In every study, there is a trade-off between
internal and external validity.
Experimentation
16
Experimental Designs and its

Classification
An experimental design is a sketch to execute an
experiment where a researcher is able to control or
manipulate at least one independent variable. The
experimental designs can be broadly segregated
into two groups: classical experimental designs
and statistical experimental designs.
Classical experimental designs consider the
impact of only one treatment level of independent
variable taken for the study at a time, whereas
statistical experimental design considers the
impact of different treatment levels of independent
(explanatory) variable as well as the impact of two
or more independent variables.
Experimentation
17
Figure 8.1 : A classification of experimental designs
Experimentation
18
Classical Experimental Designs

1. Pre-Experimental Design
Pre-experimental design is an exploratory type
of research design and has no control over
extraneous factors.
These designs are exploratory in nature and are
mainly used to frame the hypotheses about the
casual relationship and that is why a detailed
description of these designs is of paramount
importance.
There are four commonly used pre-experimental
designs. These are one-group, after-only design;
one-group, before-after design; non-matched
with control group design, and matched with
control group design.
Experimentation
19
1 (a) One-Group, After-Only Design
Experimentation
One-group, after-only design is the most basic

experimental
design.
One-group,
after-only
experimental design involves the exposure of
single group test unit to a Treatment X and then
taking a single measurement on the dependent
variable (O).
20
1 (b) One-Group, Before-After Design

One-group, before-after design involves testing

the test units twice. The first observation is
made without exposing the test units to any
treatment, and the second observation is made
after exposing the test unit to treatment.
The treatment effect can be determined as the
difference between the first observation and the
second observation, that is, O1 O2.
Experimentation
21
1 (c) Non-Matched With Control

Group Design
Non-matched with control group design involves
the introduction of control group in the
experiment. This group does not receive any
experimental treatment. In this design, the
control group is introduced, so that it can be
compared with the experimental group.
The treatment effect can be determined as the
difference between the observations from the
experimental group that receives the treatment
(O1) and the observations from the control group
that receives no treatment (O2). Hence, the result
of interest is O1 O2.
Experimentation
22
1 (d) Matched with Control Group Design

To address the problem of selection bias,

matched with control group design involves the
matching of experimental group and control
group
on
the
basis
of
some
relevant
characteristics.
M indicates that experimental group and control
group are matched on the basis of some relevant
characteristics.
Experimentation
23

2. True-Experimental Design
Randomization is the key of difference
between
pre-experimental
design
and
true
experimental design.
True experimental design involves the random
assignment of test units to the experimental group
and various treatments to the experimental groups.
True experimental designs are commonly classified
as two-group, before-after design; two-group,
after-only design; and Solomon four-group
design.
Experimentation
24
2 (a) Two-Group, Before-After Design

Two-group, before-after design is also known as

pre-testpost-test control group design. This
design involves the random assignment of test
units to either the experimental group or the
control group.
The control group is treated with extraneous
variable only and not with the experimental
treatment. Let the impact of experimental
variable be EV and impact of extraneous variable
be UV. Hence, the impact of the experimental
treatment can be computed as
Experimentation
25
2 (b) Two-Group, After-Only Design
This design is mainly susceptible to

extraneous
variables:
selection
bias
mortality.
Experimentation
Two-group, after-only design is similar to the

matched with control group design with one
difference in terms of assignment of units (or
treatments) to experimental group and control
group in a random manner.
two
and
26
2 (c) Solomon Four-Group Design
Experimentation
To handle the problems of two supplement

groups, before-after design is supplemented by
an after-only design and is referred to as
Solomon four-group design. This design is also
known as four-group-six-study design.
27

3. Quasi-Experimental Designs
In quasi-experimental design, a researcher
lacks full control over the when and whom
part of the experiment and often non-randomly
selects the group members.
The quasi-experimental designs are useful because
these are less expensive and they save time.
These designs can also be used when true
experimental designs cannot be used. The widely
used quasi-experimental designs are time
series designs and multiple time series
designs.
Experimentation
28
3 (a) Time Series Designs

Time series designs are like one-group, beforeafter

design
except
that
the
periodic
measurement is employed on the dependent
variable for a group of test units.
In the time series designs, treatments are either
administered by the researcher or it occurs
naturally.
Experimentation
29
Figure 8.2: Time series design
Experimentation
30
3 (b) Multiple Time Series Designs
Experimentation
In a multiple time series design, another group of

test units is incorporated to serve as a control
group. This design may be a better alternative as
compared with the time series designs subject to
a cautious selection of the control group.
31
Statistical Experimental Designs
Statistical experimental designs involve the

conduction of a series of experiments to
statistically control the extraneous variables and
to measure the impact of multiple independent
variables.
As different from the classical experimental designs,
statistical
experimental
designs
allow
a
researcher to examine the impact of different
treatment levels of the experimental variable.
These designs also provide an opportunity to
determine the impact of two or more independent
variables. The most widely and commonly used
statistical designs are completely randomized
design, randomized block design, Latin square
design, and factorial design.
Experimentation
32
1. Completely Randomized Design

In
a
completely
randomized
design,
the
experimental treatments are randomly assigned
to the test units. By randomly assigning
treatments to the test units, a researcher tries to
check the impact of extraneous variables through
manipulation of the treatment variable.
The statistical technique applied to analyse the

result of this type of experimental design is
known as Analysis of Variance commonly
known as ANOVA.
Experimentation
33
2. Randomized Block Design
Randomized block design involves in random

assignment of treatments to the experimental group
and control group.
A randomized block design is useful when there is
one major external variable, such as sales, store
size, or income of the respondent that might
influence the dependent variable (Malhotra & Dash,
2009).
In fact, the randomized block design is a
combination of randomization and matching.
Thus, control variable defines groups and the
randomized experiment is conducted within each
group.
Experimentation
34
Randomized Block Design (Cont.)

A randomized block design can be symbolically

represented as
For understanding the concept of randomized

block design, let us take a simple example of a
company that considers the difference in sales in
four different price levels. It is also believed by
the company that the geographic regions may
also have an impact on the sales.
Experimentation
35
Experimentation
The research question may be do the regions

significantly differ in terms of generating sales.
The second question may be do the price levels
significantly differ in terms of generating sales.
This example can be symbolically represented as
36
Experimentation
The sales (in thousands) results are exhibited in

the Table 8.2.
37
3. Latin Square Design
Latin square design allows a researcher to control two

external variables (non-interacting) along with the
manipulation of independent variable.
In Latin square design, the test units are grouped
according to the two external variables considered in
the study. The test units are systematically blocked in two
directions provided by two external variables.
Let us assume that another external variable size of the
showroom may provide the basis for difference in
quarterly sales (in million rupees) of the company.
In addition, the company has also considered one more
region south region as the basis for difference in sales
volume. The company has four different sizes of
showrooms across the country. So, the design based on
the two external variables different regions and size of
the showroom will appear as shown in Table 8.3.
Experimentation
38
Latin Square Design (Cont.)

Treatments (levels of independent variable) are

applied to the cells in such a way that each level
of independent variable is applied to each cell
only once as exhibited in Table 8.3.
In Latin square design, each cell is treated with
only one treatment level.
Experimentation
39
Factorial Design
In the statistical designs discussed so far, the effect of

only one independent (experimental) variable on
dependent variable was studied.
In a factorial design, two or more experimental
variables are simultaneously considered.
In a factorial experiment, more than one type of
independent variables are varied at a time but in a
structured way (Shaw et al., 2002); for example, a
fast moving consumer goods company wants to test a
new product in 30 cities.
Three different customer groups with respect to
income: high-income group, middle-income group, and
low-income group are to be tested. In addition, two
price levels: high price and moderate price are
considered in the experiment. Hence, this will be 3 2
factorial, as there are three different levels of income
and two different treatment
levels.
Experimentation
4.
40
Factorial Design (Cont.)

So, this design will
experimental groups:
X1
X2
X3
X4
X5
X6
=
=
=
=
=
=
have
the
following
six
High-income group; high price

High-income group; moderate price
Middle-income group; high price
Middle-income group; moderate price
Low-income group; high price
Low-income group; moderate price
Experimentation
41
Factorial Design (Cont.)

The effect of each independent variable on

dependent variable is referred to as the main
effect. Hence, the impact of different income
levels on sales and price levels on sales is
termed as the main effect.
Interaction takes place when the simultaneous
effect of two or more experimental variables is
different from the sum of their separate effect.
Experimentation
42
Limitations of Experimentation
Time
Cost
Secrecy
Implementation problems
Experimentation
43
Test Marketing
Test marketing means conducting an experiment in

a field setting.
Test markets are generally classified into four types:
standard,
controlled,
electronic,
and
simulated.
In standard test market, a company uses its own
distribution channel network to test a new product
or market mix variables.
In controlled test market, a company hires an
outside research agency to conduct the study.
An electronic test market gathers data from the
consumers who agree to carry an identification card
that they present when buying goods and services
at participating retailers in the selected cities (Hair
et al., 2002).
Experimentation
44
Test Marketing (Cont.)

Simulated test market is an artificial technique
of test marketing. A simulated test market occurs
in a laboratory, where the potential consumers of a
particular product are exposed to a new product or
competitive product or any other marketing stimuli.
Experimentation
45

Session 21-23

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Session 21-23

Uploaded by

Copyright:

Available Formats

Experimentation

Copyright Dorling Kindersley India Pvt. Ltd

Understand the concept of experimentation and

Copyright Dorling Kindersley India Pvt. Ltd

Upon completion of this chapter, you will be able to:

Experiments can be defined as the systematic

Copyright Dorling Kindersley India Pvt. Ltd

Before detailing experimentation, we will first focus

Copyright Dorling Kindersley India Pvt. Ltd

Covariation is the extent to which a caused

Copyright Dorling Kindersley India Pvt. Ltd

Concept of causality (Cont.)

O = Observation or measurement made on

Copyright Dorling Kindersley India Pvt. Ltd

Some Basic Symbols and Notations

Copyright Dorling Kindersley India Pvt. Ltd

Internal and External Validity in the

The problems of the external validity generally

Copyright Dorling Kindersley India Pvt. Ltd

Internal and External Validity in the

Copyright Dorling Kindersley India Pvt. Ltd

Threats to the Internal Validity

History effect refers to a specific event in the

Copyright Dorling Kindersley India Pvt. Ltd

Threats to the Internal Validity

Copyright Dorling Kindersley India Pvt. Ltd

Threats to the Internal Validity

Copyright Dorling Kindersley India Pvt. Ltd

Threats to the External Validity

Reactive effect occurs when the respondents

Copyright Dorling Kindersley India Pvt. Ltd

Threats to the External Validity

Copyright Dorling Kindersley India Pvt. Ltd

Ways to Control Extraneous Variables

Randomization refers to the random assignment of

Copyright Dorling Kindersley India Pvt. Ltd

Ways to Control Extraneous Variables

Laboratory Versus Field Experiment

The laboratory experiment is conducted in a

Copyright Dorling Kindersley India Pvt. Ltd

Copyright Dorling Kindersley India Pvt. Ltd

Experimental Designs and its

Copyright Dorling Kindersley India Pvt. Ltd

Figure 8.1 : A classification of experimental designs

Copyright Dorling Kindersley India Pvt. Ltd

Classical Experimental Designs

1 (a) One-Group, After-Only Design

Copyright Dorling Kindersley India Pvt. Ltd

One-group, after-only design is the most basic

1 (b) One-Group, Before-After Design

One-group, before-after design involves testing

Copyright Dorling Kindersley India Pvt. Ltd

1 (c) Non-Matched With Control

1 (d) Matched with Control Group Design

To address the problem of selection bias,

Copyright Dorling Kindersley India Pvt. Ltd

Classical Experimental Designs

2 (a) Two-Group, Before-After Design

Two-group, before-after design is also known as

2 (b) Two-Group, After-Only Design

This design is mainly susceptible to

Copyright Dorling Kindersley India Pvt. Ltd