Professional Documents
Culture Documents
Lay B. Boal
Walden University
20 February 2011
Running Head: Final Project: Developing a Quantitative Research Plan 2
Introduction
On January 8, 2002 President George W. Bush signed the No Child Left Behind Act of
2001 (NCLB), a reauthorization of the Elementary and Secondary Education Act, into law. One
of the effects of NCLB is focusing on academic performance by students, including students with
disabilities, in content areas such as social studies and science (in addition to reading, language
arts, and mathematics) by measuring such performance with statewide standardized testing
(NCLB, 2002). The state of Texas implements the Texas Assessment of Knowledge and Skills
(TAKS) as its statewide assessment system for measuring how well students are progressing.
Students with disabilities in Texas had participated in a "Three tiered model" of assessment
(TAKS-Alt) (Texas Education Agency, 2010). On the content area of social studies, the students
are only required to be tested in Grades 8, 10, and 11 (exit level) and it is not listed under a
separate reporting category. In 2011, a new assessment program, State of Texas Assessments of
Academic Readiness (STAAR) will be implemented where a new social studies curriculum will
be adopted which will include areas of understanding and analyzing economic impact on
international events, multinational corporations, government policies, and individuals on the 21st
century economy besides knowledge of U.S. history (Texas Education Agency, 2010).
In view of the future changes to the social studies curriculum, it appears that there is a
higher standard being implemented in terms of the students learning process where it is not just
knowing the subject matter content but applying a higher level of learning such as analyzing and
evaluating a particular issue that is learnt. Therefore for low achieving students who suffer from
this harsher effects of the NCLB could lose their motivation in passing their standardized
Running Head: Final Project: Developing a Quantitative Research Plan 3
assessments and educators have to make an concerted effort to motivate students on the benefit
of learning (Francis, Goheer, Haver-Dieter, Kaplan, Kerstetter, Kirk, Liu, Thomas, & Yeh, 2004)
and communities, such as schools, churches, athletes’ clubs, Boys & Girls clubs, Boy Scouts and
Girl Scouts. James-Burdumy, Dynarski & Deke (2005) stated that after school programs have
grown rapidly in recent years because both parents are in the work force, the pressure of increase
academic achievement, and concerns on the effect of unsupervised students after school hours.
The federal government has played an increasing role in the development of after school
programs as early as 1994 where Congress authorized the creation of the 21st Century
Community Learning Centers (21st CCLC Program), an organization that awards grants to after
school based programs that focuses on education, health, cultural and recreational needs of rural
and inner city public schools (Francis et al., 2004). NCLB provides for “supplemental
educational services”, and defined the program’s first purpose as being to “provide
opportunities for academic enrichment, including providing tutorial services to help students,
particularly students who attend low-performing schools, to meet state and local student
academic achievement standards in core academic subjects, such as reading and mathematics”
outcomes. Bergin, Hudson, Chryst & Resetar (1992) found positive associations between
Running Head: Final Project: Developing a Quantitative Research Plan 4
after school participation and higher achievement scores. Afterschool participation is also
associated with higher classroom grades, higher math and reading scores, increased day school
attendance, lower dropout rates, higher homework completion rates, and higher graduation rates
(Goerge, Cusick, Wasserman & Gladden, 2007; Little & Harris, 2003; Sheley, 1984). While at
the same time, others have reported mixed, insignificant or negative outcomes regarding
academic performances, school retention, feelings of safety, and behavior to name a few
(Cooper, Charlton, Valentine & Muhlenbruck, 2000; Dynarski, et al., 2003; James, 1997;
The study on the relationship between participation in after school programs and
academic achievement in 8th grade Social Studies is important to show that after school programs
are integral to students’ academic achievement not only in math, reading and science.
This study would be important to provide parents, educators, administrators, and policy
makers on the possible advantages and disadvantages of after school programs in relation to
academic achievement.
Problem statement
The problem addresses the mixed outcome of previous research studies on afterschool
programs as a potential powerful resource that can help increase student’ academic achievement,
and there is lack of research showing a relationship between students’ academic achievement in
The purpose of this study is, by conducting a quantitative quasi-experimental design, the
study will examine whether participation in after school programs increase 8th grade students’
The study will be examined based on two theoretical frameworks. The first theoretical
framework is that of Voelkl (1997) where failure to identify with school has been suggested as
one explanation for why some students persistently fail to meet academic expectations.
Identification with school has been conceptualized as involving a sense of belonging and a
valuing of school and school related outcomes (Voelkl, 1997). Students who failed to identify
with school often experience a host of problems ranging from behavioral problems, social and
emotional withdrawal, and academic failure. The second theory framework that applied is the
social capital model (Portes, 1998) which is defined as the ability to secure benefits through
1. To determine the relationship between participation in after school programs and 8th
2. To determine the effect of participation in after school programs with 8th grade
The variable that will be measured in this study will be academic achievement which is
the dependent variable, by recording benchmark scores of 8th grade social studies which will be
collected every six weeks. The independent variable is whether the student participated in after
school program.
Running Head: Final Project: Developing a Quantitative Research Plan 6
The research questions presented in this study are as follows: Is there a relationship
between participation in after school programs and 8th grade students’ academic achievement in
social studies? If there is a relationship, does participation in after school programs increases 8th
The null hypothesis for the first research question is that there is no relationship between
participation in after school programs and 8th grade students’ academic achievement in social
studies. The alternative hypothesis for the first research question is that there is a relationship
between participation in after school programs and 8th grade students’ academic achievement in
social studies.
The null hypothesis for the second research question is that participation in after school
programs has no effect on the increase or decrease in 8th grade students’ academic achievement in
social studies. The alternative hypothesis for the second research question is that participation in
after school programs caused the increase or decrease in 8th grade students’ academic
Burdumy, Dynarski & Deke (2005) where these studies have reported positive, negative and
James-Burdumy, Dynarski & Deke (2005) also reported that most studies were conducted of
after-school programs have used non-experimental designs with varying degrees of validity,
adding to the difficulty of synthesizing the literature’s findings. On the other hand, Huang et al.
Running Head: Final Project: Developing a Quantitative Research Plan 7
(2008) stated that frequent critique of afterschool studies is selection bias because afterschool
program participation is voluntary; students self-select themselves into participation and non-
participation groups. Huang et al. (2008) further explained that in comparing participating
students to nonparticipating students in the same school, there are inherent biases that researchers
need to balance or control in order for the findings to be valid. The use of experimental designs,
including randomization and control groups would be ideal but in reality, it is often difficult, and
potentially unethical, for most afterschool programs to randomize their participants unless the
Based on the arguments above, the quasi-experimental design will be the appropriate
design for this study. Frank-Nachmias & Nachmias (2008) stated that the rigid structure of an
experimental design often cannot be adapted to social science research and social scientists
generally use the quasi-experimental design in social science research even though the quasi-
experimental design is weaker in drawing causal inferences. Huang et al. (2008) argued that by
mean refusing to accept some students into the program so that they could serve as samples in
the control group, and students who are refused enrollment may end up unsupervised and
without the homework help they desperately need. Social scientists use the quasi-experimental
designs due to a problem in social science research cases when researchers cannot randomly
Nachmias, 2008).
The lack of randomization to assign students to control groups may cause vulnerability to
using a quasi-experimental design, researchers can reduce the risk of being wrong when making
Running Head: Final Project: Developing a Quantitative Research Plan 8
time regarding the differences they have predicted in their hypothesis (Frank-Nachmias &
Nachmias, 2008). In this study, by making use of a statistical control, consistent with Huang et
al. (2008), a propensity score is used to reduce self-selection bias by removing pre-existing
category differences. Propensity scores are estimated in order to account for potential
differences in student background characteristics (like gender) and by reducing initial differences
across different groups, one can more confidently attribute differences in achievement outcomes
to treatment intensity (Huang et al., 2008). Huang et al (2008) explained that propensity scores
were estimated to account for potential differences in student background characteristics, and
these scores are computed from a large reservoir of potential controls by applying a systematic
weighting procedure. In other words, the propensity score is the conditional probability of being
assigned to the treatment condition given a set of observed covariates and it is commonly
Research is also conducted over a period of time and time poses a dilemma for social
scientists , namely humans and social environment are not static entities; they change in response
to internal processes and external events and they are not willfully controlled by the researcher
(Frank-Nachmias & Nachmias, 2008). Thus, social scientists may have to modify variables over
time and quasi-experimental designs which incorporate panels and time-series designs are
flexible enough to accommodate such variation if needed (Frank-Nachmias & Nachmias, 2008).
Population
The focus population in the study would be all 8th grade students in an identified junior
high school in the Copperas Cove Independent School District. This study’s population is a
Running Head: Final Project: Developing a Quantitative Research Plan 9
finite population, which contains a countable number of 8th grade students (Frank-Nachmias &
Nachmias, 2008). The population size in this study is approximately 350 8th grade students. We
are targeting 8th graders because the benchmark tests for social studies are implemented every 6-
week grading period to all students, and they are mature enough to think about the effects of
having a low grade and the consequences of not able to pass the TAKS tests, which may be a risk
of being held back a grade or advancing to high school, and having the cognitive ability to
Sampling
This study refers to the population of 8th graders in a junior high school in Copperas
Cove, Texas. 8th grade students who participated and had not participated in after school
programs are subsets of the population and they are called samples (Frankfort-Nachmias &
requirement that researchers must ensure that they attempt to secure a representative sample,
where the sample analysis would produce a similar results to those that would be obtained had
the entire population been analyzed. Landreneau (2011) explained that errors are possible in
where there likely to result in biased samples and there is no guarantee of a representative
sample. In order to minimize or control errors, researchers could use a probability sample design
that would make it possible to estimate the degree the findings of one sample is likely to differ
from those obtained by studying the entire population (Frankfort-Nachmias & Nachmias, 2008).
Four common probability sample designs are simple random sampling, systematic sampling,
stratified sampling, and cluster sampling (Frankfort-Nachmias & Nachmias, 2008). Upon
analyzing the characteristics of the probability sample designs, the cluster sample design can be
applied to draw out the sample of 8th graders who participated in after school programs as the
Running Head: Final Project: Developing a Quantitative Research Plan 10
sampling population can be defined precisely and the list of sampling population can be obtained
by the researcher. Stage 1 of the cluster sample design would produce a simple random or
systematic sample of eighth-grade students who participated in after school programs. Stage 2 of
the cluster sample design would draw the sample of 8th grade students who did not participate in
purposive sampling where the researcher uses his subjective judgment and attempt to select
Nachmias, 2008). This means that the researcher specifies the characteristics of the population
interest and then locates individuals who match those characteristics, such as a sample of only 50
8th graders who participated in after school programs. Frankfort-Nachmias & Nachmias (2008)
explained that a nonprobability sample does not fully represent the sampling population and
affects the generalization. Nevertheless it is used by social scientists under certain circumstances
like exploratory research, convenience and economy outweigh the advantages of using
Sample size
The sample size in a research study has to be appropriate to validate the results of the
research. Frankfort-Nachmias & Nachmias (2008) explained that to determine the size of a
sample properly, researchers need to determine what level of accuracy is expected of their
estimates which is the size of the standard error acceptable to them. In determining the sample
size, Frankfort-Nachmias & Nachmias (2008) introduced the concept of standard error which is a
statistical measure that indicates how closely the sample results reflect the true values of a
Running Head: Final Project: Developing a Quantitative Research Plan 11
parameter and construct a confidence interval where the wider the interval, the smaller the risk
the researcher is willing to take on being wrong. The sample size in the research of this
application can be determined by determining the statistical power, alpha and effect size, as
explained by Burkholder (2011). As the research in this application is to examine the correlation
between academic motivation and academic achievement, I would apply the following in the
calculation of sample size, where alpha is 0.5; power is 0.80; and estimated size effect would be
about 0.75. I would use the correlation table and locate the sample size with the correlation at a
power of 0.75 and I would find the sample size of 10. As the estimated size of the population in
this study is 350 students, I would select a maximum sample size of 30 whereas 15 would give
The concept of validity for measurement relates to the validity of the measuring
instrument. Frankfort-Nachmias & Nachmias (2008) explained that content validity means that
the measurement instrument covers all the attributes of the concepts that are being measured;
empirical validity is concerned with the relationship between a measuring instrument and the
measured outcome; and construct validity is relating to the measurement instrument to a general
theoretical framework.
There will be a measuring instruments used in the study which are the social studies
benchmark tests that are being administered to all 8th grade students in the junior high school. In
order to ensure content validity of the test, each test will be administered every six weeks and
there will be a set curriculum that will be covered during the six weeks prior to administering the
test. The curriculum will also include guided instructions which are used during the before-
Running Head: Final Project: Developing a Quantitative Research Plan 12
school program and it will be designed to ensure that the content would be covered by the test
administered at the end of that 6 weeks. All tests will be multiple choices and will only have 40
questions. There will be a minimum of 2 questions per subject are covered in the curriculum.
This will ensure empirical validity of the scoring of the test as the student cannot be awarded a
discretionary grade such as bonus points. Construct validity would apply with reference to the
tests as the tests are designed to measure academic achievement based on the test scores. The
tests would have reliability as it can be applied to any studies conducted by other researchers if
the hypothesis is the same. The curriculum and tests can be applied to a different research group
provided that it is administered to students in the State of Texas and the same grade level which
Records of attendance would be collected from the after school program administration
and attendance for this study will be coded as “present” and “absent”.
The first data analysis would be a descriptive statistical analysis. Descriptive statistics
enable a researcher to explain to his readers the results of his research and the reader would be
able to understand and make sense of the results presented. Statistical results can be explained
explained that statistical measures which has the typical characteristics of frequency distribution,
such as mode, median, and the mean, are referred to as measures of central tendency. Frankfort-
Nachmias & Nachmias (2008) also explained that measures of dispersion such as range, and
Running Head: Final Project: Developing a Quantitative Research Plan 13
standard deviation are important as they explain the distribution of the data where the extent of
dispersion about the central value is measured. In this study, participation of after school
program, indicated by “present” and “absent”, is the variable selected as the nominal scale
measurement. The variable selected as a continuous variable is the test scores on 8th grade social
studies benchmark tests. The statistical assumption that is applied to this application of
descriptive statistical analysis is the standard univariate assumption. The univariate assumption
is based on three assumptions, namely; the dependent variable is normally distributed in the
population for each level of the within-subjects factor; the population variance of difference
scores computed between any two levels of a within-subjects factor is the same value regardless
of which two levels are chosen; and the cases represent a random sample from the population,
and there is no dependency in the scores between participants (Green & Salkind, 2011).
Descriptive information that will be calculated would be the mean, modes, range, and standard
The second data analysis that will be conducted is to determine whether a relationship
exist between participation in after school program and academic achievement by 8th grader in
social studies. The Pearson product moment correlation coefficient will be used to determine
such relationship. Frankfort-Nachmias & Nachmias (2008) explained that the principle of
covariation is basic to the notion of association and relation, and there are various statistical
techniques which researchers can use to assess the extent to which variables are associated.
Correlation coefficient reflects the strength and the direction of association between the variables
and the degree to which one variable can be predicted from the other (Frankfort-Nachmias &
Nachmias, 2008). The notion of prediction is inherent in the concept of covariation, and when
Running Head: Final Project: Developing a Quantitative Research Plan 14
variables which are analyzed are at least intervals, researchers can describe the nature and form
of the relationship in terms of a linear function rule (Frankfort-Nachmias & Nachmias, 2008).
Pearson's product-moment correlation coefficient (r) procedure which would compute the linear
relationship between test scores and the participation in after school program. Green & Salkind
(2011) explained that the significant test for r evaluates whether there is a linear relationship
between the two variables in the population. The statistical assumptions underlying the
significance test for the Pearson correlation are laid out by Green & Salkind (2011). The first
assumption is that the variables are bivariately normally distributed which indicates that the only
type of statistical relationship that can exist between the two variables is a linear relationship;
and the second assumption is that the cases represent a random sample from the population and
the scores on variables for one case is independent of scores on these variables for other cases
(Green & Salkind, 2011). If p is less than 0.05, the null hypothesis can be rejected and there is a
statistical significant relationship between participation in after school program and academic
Independent-sample t tests
The third data analysis conducted is to examine any significant difference among scores
and moderator variables. In order to determine any differences in the academic achievement in
8th grade social studies measurement by using test scores and attendance collected on
Independent-samples t test is one method of statistically testing a hypothesis. Green & Salkind
(2011) explained that the independent-samples t test evaluates the difference between the means
of two independent groups, and each case must have scores on two variables, the grouping
variable and the test variable. In this study, participation in after school programs has been
Running Head: Final Project: Developing a Quantitative Research Plan 15
selected as the grouping variable or also known as the independent variable. The grouping
variable divides cases into two mutually exclusive groups, in this case, “present” and “absent”.
The academic achievement variable has been selected as the dependent variable which is also
known as the test variable, which describes each case on a quantitative dimension. Green &
Salkind (2011) explained that the t test evaluates whether the mean value of the test variable for
one group differs significantly from the mean value of the test variable for the second group.
The statistical assumptions that are applied to this test are the assumptions underlying the
independent-samples t test as laid out by Green & Salkind (2011). The first assumption is that
the test variable is normally distributed in each of the two populations as defined by the grouping
variable; second assumption is that the variances of the normally distributed test variable for the
populations are equal; and the third assumption is the cases represent a random sample from the
population, and the scores on the test variable are independent of each other (Green & Salkind,
2011). The result indicates that we fail to reject the null hypothesis if p is less than .05. This
would mean that there is no significant difference in academic achievement in 8th grade social
statistically testing a hypothesis. Green & Salkind (2011) explained that a one-way analysis of
variance (one-way ANOVA) can be used where each case have two variables: a factor that is
divided into two or more groups or level; and the dependent variable differentiates individuals on
between academic achievement and participation of after school program. The independent
Running Head: Final Project: Developing a Quantitative Research Plan 16
variable, participation of after school program included two groups: present, and absent. The
Green & Salkind (2011) explained that an overall ANOVA test is conducted to assess
whether means on a dependent variable are significantly different among groups, and it is
significant and a factor has more than two levels, follow-up tests which are known as post hoc
multiple comparisons, are conducted by comparing pairs of group means. Green & Salkind
(2011) also added that the ANOVA F test evaluates whether the group means on the dependent
variable differ significantly from each other. The statistical assumptions that are applied to this
application are the assumptions underlying the one-way ANOVA as laid out by Green & Salkind
(2011). The first assumption is that the dependent variable is normally distributed for each of the
populations as defined by the different levels of the factor; the second assumption is that the
variances of the dependent variable are the same for all populations; and the third and final
assumption is that the cases represent random samples from the populations and the scores on the
test variable are independent of each other (Green & Salkind, 2011).
The result indicates that we fail to reject the null hypothesis if p is less than .05. This
would mean that there is no significant difference in academic achievement in 8th grade social
studies according to participation in after school program. If the overall F test was significant,
follow-up test was conducted to evaluate pairwise differences among the means. By using the
post hoc procedures, the Tukey HSD test was conducted and the post hoc comparison is
analyzed.
Limitations
Running Head: Final Project: Developing a Quantitative Research Plan 17
The study will be limited in terms of its generalizability to the total population of students
from grade K to 12. The study was only conducted on 8th grade students and the effect on
In agreement with Huan st al. (2008), outcome studies of afterschool programs typically
are designed to compare participants with non-participants based on any program attendance.
Consequently, participants may attend one day in an afterschool program and still be included in
the treatment group. Furthermore, non-participants may have been enrolled in other afterschool
activities and still be included in a control group. As stated in a report by Frankel and Daley
(2007), two very important issues are ignored by most studies: First, “How did the non-
participants spend their time afterschool?” and second, “How intensive was the participants’
program attendance?” (p. 12). Expanding on Frankel and Daley’s strategy and addressing their
concerns, this study used statistical strategies to reduce selection bias, this study could be
The administration of the tests can pose a threat to validity some students may require
modification or accommodations when taking these tests. External evaluators can be employed
Ethical concerns
This study will be collecting data from 8th grade students. The data collection and
handling would be designed in accordance with the National Institutes of Health Protecting
(Protecting Human Research Participants, 2008). These guidelines maintain the overarching
themes of Respect for Persons, Beneficence, and Justice (Protecting Human Research
Running Head: Final Project: Developing a Quantitative Research Plan 18
Participants, 2008) and conform to the U.S. Department of Health and Human codes (U.S.
This study measures test scores of social studies as academic achievement. This should
not be the whole construction of the meaning of academic achievement and readers and
participants of this study must not unduly discriminate when there are other forms of benefits
that relates to participation of after school programs, which are also associated with higher
classroom grades, higher math and reading scores, increased day school attendance, lower
dropout rates, higher homework completion rates, and higher graduation rates.
school program in the control group. Students who needs after school assistance may be denied
such services and be left unsupervised. The statistical control, such as removal of the sample can
be considered.
The findings of this study could have far reaching significant consequences. The study
is conducted on the relationship between participation in after school program and academic
achievement, which will assist in showing consistency in the direction the results have produced,
despite the Huang et al (2008) contention that previous research in after school programs is
inconclusive.
My study would fill the gap of providing data on academic achievement specifically on
8th grade students and the social studies subject. The results would provide additional data to
future research and given the opportunity to expand my study by analyzing a different variable
such as motivation to learn. My research can be used to support additional funding on improving
Running Head: Final Project: Developing a Quantitative Research Plan 19
after school programs, which may reduce juvenile crimes, parental stress, increase student
social relationships with adults and peers, lessening truancy, and increase parents involvement.
The ultimate goal is to establish a feasible after school program that will benefit students
all over United States or even a model for students all over the world. Such benefits may be
identified by utilizing education as a tool to improve the quality of life for these children and
them being our future, we want to instill good practices, safety, morals and values so that the
References
Bergin, D. A., Hudson, L. M., Chryst, C. F., & Resetar, M. (1992). An afterschool intervention
program for educationally disadvantaged young children. The Urban Review, 24(3), 203–
217.
Cooper, H., Charlton, K., Valentine, J. C., & Muhlenbruck, L. (2000). Making the most of
summer school: A meta-analytic and narrative review. Monographs of the Society for
Dynarski, M., Moore, M., Mullens, J., Gleason, P., James-Burdumy, S., Rosenberg, L., et al.
(2003). When schools stay open late: The national evaluation of the 21st Century
Policy Research.
Francis, A., Goheer, A., Haver-Dieter, R., Kaplan, A.D., Kerstetter, K., Kirk A.L.M., Liu, S.,
Thomas, A.M., & Yeh, T. (2004) Promoting academic achievement and motivation: a
Running Head: Final Project: Developing a Quantitative Research Plan 20
Frankfort-Nachmias, C., & Nachmias, D. (2008). Research methods in the social sciences (7th
Goerge, R. M., Cusick, G. R., Wasserman, M., & Gladden, R. M. (2007). After-school programs
and academic impact: A study of Chicago’s After School Matters. Chicago: Chapin Hall
Green, S.B. & Salkind, N.J. (2011). Using SPSS for windows and macintosh: analyzing and
Huang, D., Leon, S., La Torre, D., & Mostafavi, S. (2008). Examining the relationship between
LA’s BEST program attendance and academic achievement of LA’s BEST students.
James-Burdumy, S., Dynarski, M., & Deke, J. (2005). When elementary schools stay open late:
Results from the national nvaluation of the 21st Century Community Learning Centers
Program. U.S. Department of Education, Institute of Education Sciences, National
Center for Education Evaluation and Regional Assistance, 2005.
James, D. W. (1997). Some things do make a difference for youth: A compendium of evaluations
of youth programs and practices. Washington, DC: American Youth Policy Forum. (ERIC
Landreneau, K.J. (2011). Sampling strategies. Retrieved January 12, 2011 from
www.natco1.org/research/files/SamplingStrategies.pdf
Running Head: Final Project: Developing a Quantitative Research Plan 21
Little, P. M. D., & Harris, E. (2003, July). A review of out-of-school time program
Snapshot, 1.
No Child Left Behind Act of 2001, Pub L. No. 107-110, 115 Stat. 1425 (2002)
Portes, A. (1998). Social capital: Its origins and applications in modern sociology. Annual
Vanderhaar, J., & Muñoz, M. A. (2006). Educating at-risk African American males: Formative
Voelkl, K. (1997), Identification with school. American Journal of Education, 105, 294-319.