400755 Evidence Based Nursing 1

Reliability & Validity

Prepared by: Dr Liz Halcomb & Judy Mannix 2008

Revised by: Dr Kath Peters 2009

Objectives
● By the end of this session & with additional
readings you will be able to:
– Differentiate between reliability and validity
– Explain how reliability and validity relate to
quantitative research
– Understand key aspects in evaluating
quantitative design and methods

Evaluating Research

● Does the research provide valid and reliable

evidence?

● Measures of this in quantitative research are

reliability and validity

1
 Validity & Reliability
(Borbasi, Jackson, Langford, 2008, pp. 105-6)

● A valid instrument measures what it is

supposed to measure.

● A reliable instrument measures the variable

consistently, dependably and accurately.

Schneider et al. 2003, p.335

Internal & External Validity

(Schneider et al., 2007, p. 199-200)

● Internal validity = whether the intervention

(independent variable) has a measurable
effect on the outcome (dependant variable).

● External validity = the extent to which

findings can be generalised beyond the
sample.

2
 Types of Quantitative Validity

● Content validity

● Construct validity

● Criterion-related validity

Content Validity
● The extent to which “the research instrument
represents the phenomena under study”
(Dempsey & Dempsey, 2000, p.191).

● Assessed by a panel of experts

● Face validity: Does the research instrument
look as though it is measuring the intended
phenomena?

Construct Validity
● “Degree to which the measuring instrument
measures a specific hypothetical trait or
construct” (Dempsey & Dempsey, 2000, p. 191).

● Establishing construct validity will involve the

use of statistical techniques such as factor
analysis.

3
 Criterion-Related Validity
● Relationship of the instrument to another
relevant instrument or measure
– Predictive validity: relationship of
instrument to future behaviour or
performance (Beanland et al., 1999)

– Concurrent validity: relationship to

another valid research instrument that
measures similar construct

Reliability

● “Degree of consistency with which the

instrument measures what it is supposed to
measure” (Polit & Hungler, 1999, p. 411)

● Includes stability, homogeneity (internal

consistency) and equivalence (Borbasi et al., 2008)

Stability
● The extent to which the same scores are
obtained when an instrument is used on the
same people on different occasions.

● Test-retest reliability
– statistical analysis of results
– reliability coefficient (r). High coefficient =
high reliability. Perfect is +1.00 (difficult to
obtain). Scores over 0.70 may be
acceptable

4
 Homogeneity
(Borbasi et al., 2008; Schneider et al, 2007).

● Internal consistency of items

● Do all items of an instrument measure the
same characteristic?
● Some methods of establishing this:
− Item to total correlations
− Kuder-Richardson KR-20
− Cronbach Alpha
− Split–half technique

Equivalence
● “the consistency or agreement among different
observers using the same or alternate forms of
a measuring instrument” (Schneider et al., 2007, p. 213)

● Methods of establishing this:

– Inter-rater reliability
– Parallel or alternate form (also examines stability)

Evaluating Quantitative Methods

● The title and abstract
● Structuring the study
● The sample
● Data collection
● Data analysis
● Findings
(Schneider, 2007, p. 283).

5
 Evaluating Quantitative Methods
● The title and abstract
– Title congruent with text?
– Aims and objectives stated?
– Sufficient information about research process?
● Structuring the study
– Literature review adequate?/Gaps highlighted?
– Hypothesis stated?/Type?
● Sample size
– Adequate?/How was size determined?
– Appropriate for analysis used?
(Schneider, 2007, p. 283)

Evaluating Quantitative Methods

● Data collection
– How and by whom were data collected?
– Are measurement instruments described?/validated?
– Reliability and ethical issues addressed?
● Data analysis
– Appropriate?
– Outcomes of all analyses identified and explained?
● Findings
– Adequate information?
– Limitations identified?
(Schneider, 2007, p. 283)

References
Baldry E, Bratel J, Dunsire M & Durrant M. (2005). Keeping children with a
disability safely in their families. British Association of Social Workers,
17(3):143-156.
Beanland, C., Schneider, Z., LoBiondo-Wood, G. & Haber, J. (1999). Nursing
research: Methods, critical appraisal and utilisation (First Australasian
Edition; reprint 2000). Marrickville: Harcourt Australia.
Borbasi, S., Jackson, D. & Langford, R.W. (2008). Navigating the maze of
nursing research: An interactive learning adventure (2nd ed.). Marrickville:
Elsevier.
Burns, N. & Grove, S.K. (1999). Understanding nursing research. Philadelphia:
Saunders.
Dempsey, P. A., & Dempsey, A. D. (2000). Using nursing research: Process,
critical evaluation and utilization. Philadelphia: Lippincott.
Kendall, S. & Bloomfield, L. (2005). Developing and validating a tool to measure
parenting self-efficacy, Journal of Advanced Nursing. 51(2): 174-81.
Kong, S K F & Lee, D T F. (2004). Factors influencing decision to breastfeed.
Journal of Advanced Nursing, 46(4):369-379.

6
 References continued
Mactavish J B & Schleien S J. (2004). Re-injecting spontaneity and balance in
family life: Parents' perspectives on recreation in families that include
children with a developmental disability, Journal of Intellectual Disability
Research, 48(2):123-141.
Polit, D.F. & Hungler, B.P. (1999). Nursing research: principles and methods
(6th ed.). Philadelphia: Lippincott
Polit, DF., Beck, C.T. & Hungler, B.P. (2006). Essentials of nursing research:
Method, appraisal and utilisation (5th ed.). Philadelphia: Lippincott.
Schneider, Z. (2007). Critical evaluation of research. In Z Schneider, D
Whitehead, D Elliott, G LoBiondo-Wood & J Haber (Eds.), Nursing
research: Methods, critical appraisal and utilisation (3rd ed.) (pp. 268-300)
Marrickville: Elsevier.
Schneider, Z., Elliott, D., LoBiondo-Wood, G. & Haber, J. (2003). Nursing
research: methods, critical appraisal and utilisation. (2nd ed.). Marrickville:
Elsevier.
Schneider, Z., Whitehead, D., Elliott, D., LoBiondo-Wood, G. & Haber, J.
(2007). Nursing research: Methods, critical appraisal and utilisation (3rd
ed.). Marrickville: Elsevier.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed methods in
social and behavioral research. California: Sage Books.