Running Head: MDDE 602 RESEARCH ARTICLE CRITIQUE

Critique of a Research Project

Terralyn McKee
Student Number 2653678
MDDE 602
Assignment #4

MDDE 602 RESEARCH ARTICLE CRITIQUE

Introduction
Research, whether it is qualitative, quantitative or mixed methods, is a systematic
process of investigation seeking to add to or confirm previously established truths about
the world in which we live. The research community has established a set of protocols
to guide this investigative process in order to ensure its integrity, rigor, validity,
generalizability, etc. these protocols, or methodologies, have been established to limit
bias and error throughout all phases of research project (Neuman, 2006).
The process of research critique acts as a vital and necessary part of the
research environment as it provides a forum for scholarly review and commentary on
the premises, processes and outcomes of research. Critiques support a process of
analysis, examining the strengths and limitations of the research process and
outcomes, and by extension, how this information might expand current knowledge in
the field or be used to enhance practice (Boswell & Cannon, 2011). Overall, a critique
should be a vehicle for feedback toward the improvement of knowledge. This paper will
use the forum of critique to review a qualitative research study by Lynn Feist (2003) on
removing the barriers to professional development in online distance education.
Statement of the Problem
The research article reviewed, Removing Barriers to Professional Development
(Feist, 2003) has been framed within the methodology of a qualitative research project
more specifically, as applied research using an evaluation research focus to establish
the project efficacy in meeting the outcomes of more relevant professional development
for online instructional staff. This project used a case study approach to explain
(explanatory purpose) the relationship between pairing online course development with

MDDE 602 RESEARCH ARTICLE CRITIQUE

technology training as a collaborative professional development approach toward

improving instructor efficacy and satisfaction
The concepts of professional development, course design for online vs face to
face environments, integration of learning technologies delineated the starting point for
focusing the research question and for connecting points in learning theory and distance
education emerging themes were defined for use in coding (Feist, 2003. P. 28) and
extended/refined the original concepts to include
1) effective professional development,
2) time,
3) leadership,
4) curriculum,
5) role of the instructor, and
6) special projects
These correspond to the key terms and concepts highlighted in the research of
online education, instructional course design and technology training for faculty (e.g.
Garrison, 2000; Moore & Kearsley, 2005; Shulman,1981).
After a targeted literature review of professional development for instructional
staff in online learning environments it was determined that there were a number of
barriers in traditional professional development models that rendered the process
ineffective in addressing the needs of instructors and technology training, and that these
barriers were particularly problematic for part-time faculty and faculty working in a
decentralized campus environment such as hers (Grant MacEwan University). As noted
by respondents in the focus group, the research focus lacked clarity with regards to
project expectations (Feist, 2003, p.19).

MDDE 602 RESEARCH ARTICLE CRITIQUE

Research Design
The methodology employed for this research project was a case study which
focused in-depth attention on the unique and complex experiences of a select number
of participants, while integrating additional information from a larger group of
participants who were also included in the provision of professional development
activities and collaborative model design for integrated course design and technology
applications. According to Feist (2003, p. 28), her choice of a case study design
provide(s) in-depth information about the experiences and opinions of instructors
involved in online course development. Given that Feist is examining the role of
professional development at Grant MacEwan University, the use of a case study
methodology, with its in-depth analysis of a typical process, constraints on time and
funding, and limited need for generalization of results beyond the institution, make this
an appropriate methodological choice (Cohen, Manion & Morrison, 2007).
While over 100 teachers/administrators were included in this two year project, the
sample size for the in-depth interviews included ten pre-selected teachers
representing a nonrandom theoretical sample. Data collection included the standard
case study tools of field notes, interviews, and surveys - content analysis was applied to
produce rich, descriptive thematic narratives. The data produced both formative and
summative evaluations of the lived experiences of the respondents and were used
throughout the project to construct the collaborative model process and guidebook.
Interestingly enough however, Feist (2003) included pre and post tests into her
design, which are hallmarks of experimental methodology. This reader questions the
rationale of this process given that the results of the pre-post test were compromised

MDDE 602 RESEARCH ARTICLE CRITIQUE

given their administrations times and completion rates, as well as the researchers own
admission of their value based on the inability to make any meaningful comparisons to
pre-post skill, knowledge or behaviour changes in respondents. Was the inclusion of this
element into the research design an attempt by the researchers to improve process and
data rigor since Muketha, Ghani, Selamat, and Atan (2010) state that experiments
provide more rigor and control over research variables than case studies? If that is the
case, then its inclusion was questionable at best.
A final note on data collection and analysis design, Feist (2003) states that the
target audience for this study included teachers (full and part-time) and administrators.
However, no data has been included that demonstrates administrator perspective or if
and how the experiences of part-time teachers differed from that of full time teachers.
The linkage of the design, process or data collection and analysis that incorporates the
perspective of the administrator and part-time teachers is found only in the section for
research inference. Validation of this inference is needed as there is no mechanism that
provides a check back process prior to the section on findings regarding the reliability,
replication or validity of this inference in the research data or process.
Findings
The researcher in this project felt that the data supported her hypothesis
regarding the nature of barriers in professional development for online staff. However, it
is difficult to ascertain from the project report if this is truly the case as the findings
appear to be based solely on the focus group results from five of the ten respondents. If
additional considerations were incorporated into the overall findings, they have not been
directly attributed to the other project activities or participants 100 teachers, project

MDDE 602 RESEARCH ARTICLE CRITIQUE

consultants or design team members or the methods for their design, collection,
analysis and reporting.
In fact, Lincoln and Guba (1985) provide case study guidelines to direct the
process for reporting findings from case study research; they state that reporting should:
- strive for an informal nature and tone to reflect the respondent style;
-be factual except in sections on interpretation, evaluation and inference;
-endeavor toward over-inclusion vs under-inclusion of key concepts/input;
-adhere to research ethics of anonymity;
-reference data as a means of check back for reliability, validity and inference.

Feist (2003) presents the research findings of the content analysis process in summary
form - she does identify the themes that emerged from the descriptive narratives,
though she does not include a coding scheme sample in the appendix. This process
would have provided more validity for her findings by allowing reviewers to examine the
tools, their definitions and levels of measurement. Based on what Feist (2003) has
chosen to include in this report, it does appear that the data does support the research
hypothesis.
Discussion
The research document focuses discussion on the models of online course
development and technology learning as key aspects of professional development for
teachers in post-secondary education, the barriers of traditional design development

MDDE 602 RESEARCH ARTICLE CRITIQUE

and delivery, and teacher feedback on experiences within a case study project to
explore a collaborative development model.
Project highlights included the development of a collaborative model of
professional development and resulting guidebook, the development of more
responsive and effective professional development strategies for respondents, and the
inclusion and value of onsite, staffed technology sandboxes for respondents.
The research discussion does examine the process and outcomes in terms of the
project objectives in a systematic and logical manner. The challenges, successes and
unintended outcomes that emerged throughout the project activities were included in an
honest and reflective manner, making assessments about what should have happened
and adaptations to methods and resources next time activities are undertaken. Given
that qualitative research methods allow for an evolution in the research question and
investigation processes, the trajectory and conclusions of this project were not unusual
or surprising to this reviewer.
Application of the findings however might be overly inclusive given the limited
amount of information that was provided in this research with regards to its application
to administrators and part-time faculty members.
Conclusions
Feist (2003) concludes that these findings support her original research assumptions
regarding barriers to professional development for teaching faculty; time constraints, the
value of just-in-time technology training, and easily-accessible support mechanisms are
the key to impactful professional development services.

MDDE 602 RESEARCH ARTICLE CRITIQUE

This research project was framed within the methodology of a case study, and as such,
the findings are directly applicable to the experiences of the respondents and their
educational environments within Grant MacEwan University. Careful consideration of its
generalizability need to be taken within this context, and examination of the elements
and concepts from this study need to be explored before judgment of transferability to
other environments or populations can be made. However, that being said, the project
concepts were grounded within the definitions of educational pedagogy regarding
curriculum design, faculty development, online delivery, and the impacts of technology
on education. The research results supported much of the assertions made previously
with regards to these concerns allowing more comfort in the generalization of outcomes
to other educational environments and populations.
Implications for Future Research and Knowledge Base
While tentatively supporting theoretical bases of existing knowledge in the field,
Feists study does little to add to the overall body of knowledge in this area. As
previously identified, this study was plagued issues that included lack of clarity in
purpose, structuring of some design elements, and incomplete data supports for
inferences. However, Feist does make an important assertion when she identifies
existing gaps within the literature that fail to focus on learning to teach with technology
as a concept and the error of past models that were built on the experiences of
educational technologists and not on teacher experiences with technology. This
observation is in line with research by Mishra & Koehler (2006, p. 1018) that states,
Developing theory for educational technology is difficult because it requires a detailed
understanding of complex relationships that are contextually bound. Their support of

MDDE 602 RESEARCH ARTICLE CRITIQUE

emerging research methodology known as design experiments - specifically tailored to

include implementation detail as well as the pedagogical impacts of the study
interventions - would qualify as a valid and potentially rich source of new knowledge.
Overall, I believe there was value for this project for the institution and the
respondents that if sustained, could provide a basis for another review to assess longerterm impacts of adopting the Collaborative Model and associated use of the guidebook.

MDDE 602 RESEARCH ARTICLE CRITIQUE

References
Boswell, C. & Cannon, S. (2011). Introduction to nursing research: Incorporating
evidence based practice (2nd ed.). Sudbury: MA. Jones & Bartlett Publishers.
http://www.jblearning.com/samples/0763744379/Critique_Process.pdf
Cohen, L., Manion, L., and Morrison, K. (2007). Research methods in education (6th
ed.). New York: NY. Routledge.
Feist, L. (2003). Removing barriers to professional development. T.H.E. Journal, 30(11),
30-36. Retrieved from
http://it.cofc.edu/tlt/services/FTI/ftiprojects/resources/remove_barriers.pdf
Garrison, R. (2000). Theoretical challenges for distance education in the 21 st century.
International Review of Research in Open and Distance Learning 1(1). Retrieved
from www.irrodl.org/index.php/irrodl/article/download/2/22
Lincoln, Y.S. and Guba, E. (1985). Naturalistic inquiry. Beverly Hills: CA. Sage p365-6
McMillan, J. H. & Schumacher, S. (2001). Case study design. In Research in education:
A conceptual introduction. Toronto: Addison Wesley Longman, Inc.
Mishra, P., & Koehler, M.J. (2006). Technological pedagogical content knowledge: A
framework for integrating technology in teacher knowledge. Teachers College
Record, 108(6), 1017-1054. Retrieved from
http://citeseerx.ist.psu.edu/viewdoc/download?
doi=10.1.1.91.7990&rep=rep1&type=pdf

MDDE 602 RESEARCH ARTICLE CRITIQUE

Moore, M. and Kearsley, G. (2005). Distance Education: A Systems View (2nd ed.).
CA:Thomson Wadsworth.
Muketha, G.M., Ghani, A.A.A., Selamat, M.H., and Atan, R. (2010). A survey of
business process complexity metrics. Journal of Information Technology, 9:
1336-1344. DOI: 10.3923/itj.2010.1336.1344 Retrieved from
http://scialert.net/abstract/?doi=itj.2010.1336.1344
Neuman, W. L. (2006). Social research methods: Qualitative and quantitative
approaches (6th ed.). Toronto: Allyn & Bacon
Shulman, L.E. (1981). Disciplines of inquiry in education: An overview. Educational
Researcher, v10 n6 p5-12, 23, Jun-Jul 1981. Retrieved from
http://www.indiana.edu/~educy520/readings/shulman97.pdf