Reliability and Validity of the Test

Reliability and Validity

Reported by: Opamin, Sheryl D.

Validity
Denotes the extent to which an instrument
is measuring what it is supposed to
measure.
The degree to which a test measures what
it intends to measure of the truthfulness of
the response.
Refers to the extent to which a test
measures what it purports to measure.

Types of Validity
Content Validity
is the degree to which test items match
some objective criterion.
Refers to the adequacy and
representativeness of learning outcomes
to be measured. (Gutierrez 2007)
Commonly used in assessing
achievement test. (Calmolin 2012)
Depends on the relevance of the
individuals responses to the behavior
area in consideration rather than on the
apparent relevance of item content.
( Calmolin 2012)

Types of Validity
Construct Validity

Predictive Validity

Is a degree to which a
test
measures
an
attribute or quality it is
supposed to measure.
This involves such test as
those of understanding,
appreciation,
and
interpretation of data.

Refers to the degree or

extent to which scores on
a test can predict later
behavior or test scores.
Determines by showing
how well predictions made
from the test are proven
by proof collected at some
succeeding time.

Types of validity
Concurrent Validity
Refers to the relationship between scores on a
test or scale
and scores on another measure of established
validity given at about the same time.
It is the degree to which the test agrees or
correlates with a criterion set up as an acceptable
measure.

Reliability
Refers to the consistency of a test in measuring
whatever it measures ( Wiersma & Jurs, 1992).
It is the degree to which a test will yield similar or
comparable results for the same students when
administered at different times.
This is usually measured through the computation
of a reliability coefficient.

Reliability Coefficient
Is a measure of the consistency of a test
( Wiersma 1991)
1.00 to 0.80 indicates high reliability
0.79 to 0.40 indicates fair reliability
0.39 to below indicates low reliability
In general, the higher the reliability the less effect
of chance upon test. The lower the reliability, the
more the test reflects chance factors.

Errors of measurement
Test Takers
Test Itself
Test administration
Test Scoring

Methods of Determining the Test Reliability

Test-Retest Method
- In this method test reliability is determined
from the repetition of the same test. Thus, a test is
administered twice.
- Scores obtained by the students in both test
administration are rank ordered and a rank
correlation coefficient is obtain with the use of
Spearman rho.

Test-Retest Method
Spearman rank correlation coefficient or Spearman
rho
- Is the statistical tool used to measure the
relationship between paired ranks assigned to
individual scores on two variables X and Y.

-( X )will be the first administration and (Y) is the
second administration . To obtain the value of
spearman rho (r) , consider the formula.
Rho= 1-

Steps to follow on Test-Retest Method

Step 1- Rank the scores of the respondents from
highest to lowest in the first set of administration
(X) and mark this rank as rank x. The highest
score receives rank 1, second highest 2; third
highest 3 and so on.
Step 2- Rank the second set of scores in the
second administration (Y) in the same manner as
in Step 1 and mark this is rank y.
Step 3 Find the difference for every pair to get
D.
Step 4- Multiply the difference by itself to get D2.
Step 5- Add or total D2 to get .

Test-Retest Method
Student
1
2
3
4
5
6
7
8
9
10

Scores- FA
33
28
44
36
26
35
42
40
20
28

Scores- SA
29
26
42
30
28
35
39
41
20
28

Test-Retest Method

Test-Retest Method Spearman rho

Ss-FA
X

Ss- SA
Y

Rank
x

Rank
Y

33
28
44
36
26
35
42
40
20
28

29
26
42
30
28
35
39
41
40
28

6
7.5
1
4
9
5
2
3
10
7.5

7
10
1
6
8.5
5
4
2
3
8.5

-1
-2.5
0
-2
0.5
0
-2
1
7
-1

1
6.25
0
4
0.25
0
4
1
49
1

N = 10

= 66.5

Test-Retest Method
Computation of Spearman rho
Formula: Rho= 1= 1- = 1

= 1- = 1- 0.4030
= 0.597 Or 0.60 fairly reliable

Parallel Form Method

In this method , two equivalent form of the test
are formulated and administered to the same
group students during a single time period.
The correlation between the two sets of scores on
the two forms of the test is considered an
estimate of reliability.
Pearsons r is the measure of relationship that has
to be used in calculating the coefficient of
reliability in this method.

Parallel Form Method

Computation of Reliability Coefficient for the parallel Method
using Pearsons r
X

90
90
85
85
80
80
75
75
70
70
65
65
60
60
55
55
50
50
45
45
SX=
SX=
675
675

80
80
72
72
70
70
65
65
68
68
55
55
60
60
50
50
53
53
44
44
SY= 617
SY= 617

XY
8100
8100
7225
7225
6400
6400
5625
5625
4900
4900
4225
4225
3600
3600
3025
3025
2500
2500
2025
2025

6400
6400
5184
5184
4900
4900
4225
4225
4624
4624
3025
3025
3600
3600
2500
2500
2809
2809
1936
1936

S= 47625

= 39203

7200
7200
6120
6120
5600
5600
4875
4875
4760
4760
3575
3575
3600
3600
2750
2750
2650
2650
1980
1980
SXY= 43110
SXY= 43110

Parallel Form Method

Pearsons r Computation

N SXY= 10 (43110) = 431,100

233,908,125
(SX)(SY)=(675)(617)= 416,475

=
=

N SXY- (SX)(SY) = 431,100- 416,475 = 14,625

=
15294.05522
=
10(47625) = 476,250
N
r=
( = (675)(675) = 455,625
= 0.956 or 0.96
(N - () = 476,250-455,625 = 20,625
=0.96 Highly Reliable
N = 10(39203)= 392,030
( = (617)(617) = 380,689
(- (= 392,030-380,689 = 11,341
(N -( (N - )- = (20,625)(11,341)

Split Half Method

In this method a single test is divided into two
equivalent halves and is used as if they were two
separate tests to estimate reliability coefficient.
One common method of dividing the test is by scoring
the even-numbered and odd numbered items
separately.

( Ornstein, 1900)
Spearman-Brown formula is used to measure Split-Half
Method Reliability. R =
R= Split-half reliability and roe= Correlation between
odd and even items.

Split Half Method

Computation of Correlation for Odd and Even Scores using
Pearsons r
Total
Score
40
35
32
46
39
42
37
45
38
41

Odd
Even
Score (X) Score (Y)

XY

18
19
20
23
27
20
18
24
19
22

22
16
12
23
12
22
19
21
19
19

324
361
400
529
729
400
324
576
361
484

484
256
144
529
144
484
361
441
361
361

396
304
240
529
324
440
342
504
361
418

SX= 210

SY= 185

= 4488

= 3565

SXY= 3858

Split-Half Method
Pearsons r Formula Computation:
N=10
N SXY= 10 (3858) = 38,580
=1,111,500
(SX)(SY)=(210)(185)= 38,850
=
N SXY- (SX)(SY) = 38,580- 38,850 = 270
= 1054.277
N = 10(4488) = 44,880
r=
r= 0.256 or 0.26
( = (210)(210) = 44,100
(N - () = 44,880-44,100 = 780
0.26 Low level of
Reliability
N = 10(3565)= 35,650
( = (185)(185) = 34,225
(- (= 35,650-34,225 = 1425
(N -( (N - )- = (780)(1425)

Improving test Reliability

Increase the number of items.

Making the test of average difficulty.
Scoring a test objectively.
Setting time limits for answering the test.

Wrap- Up
Choose one and answer
1. What is Content Validity?
2. What is the formula use to compute Test-Retest
Method?
3. Define what is Reliability Coefficient.
4. What are the errors in measurement that affect
the reliability?
5. Give the range of reliability Coefficient.

Scriptures for today

Where there is strife, there

is pride,
but wisdom is found in
those who take advice.
Proverbs 13:10

References

Evaluating and Measuring learning outcomes

Assessment of Student learning 1 Laurentina
Paler-Calmorin Ph.D.
Assessment of learning Outcomes (Cognitive
Domain)- Danilo S. Gutierrez, Ph.D.