The Impact of Homework Time on Academic Achievement

Steven McMullen May 2010 Working Paper

Abstract This study takes advantage of nationally representative panel data on student behavior and academic performance to test two possible policy reforms. First, I examine a policy that increases the amount of homework that students complete. Second, I examine the impact of increasing the amount of homework assigned. Previous studies have not been able to consistently estimate the impact of homework because of important omitted variables and measurement error, which strongly bias the estimated impact of homework time. This paper, however, uses an instrumental variables approach with student fixed effects to account for both time-varying and time-invariant unobserved characteristics and inputs. This approach produces estimates of the impact of homework time on academic achievement that are much larger than those of previous studies. Additionally, these findings suggest that assigning additional homework primarily improves the achievement of low performing students and students in low performing schools. Thus, assigning more homework could help close the gap in achievement between high and low performing students.

JEL classification: J24; I20; I21 Keywords: homework, academic achievement

Steven McMullen is an assistant professor of economics at Calvin College in Grand Rapids, Michigan. The author would like to thank Tom Mroz, David Blau, Donna Gilleskie, David Guilkey, and Helen Tauchen for their excellent assistance. Steven can be reached at mcmullen@calvin.edu or Department of Economics, Calvin College, 1740 Knollcrest Circle SE, Grand Rapids, MI 49546.

I. Introduction The amount of homework that students should complete has been an issue of debate and scholarship for decades. A number of popular books and articles have been written recently arguing that homework should be limited or prohibited entirely (Bennett and Kalish 2006; Kohn 2006; Kralovec and Buell 2001)1. While many scholars have investigated the impact that homework has on academic achievement, there is no strong consensus in the literature. Moreover, most studies have done little to correct for the biases caused by omitted variables that likely influence students choices regarding study time.2 In this paper, I examine two related issues. The first is the effect of students time spent doing homework on achievement test scores. The second is the impact of assigning additional homework on achievement. The primary econometric challenge that this study overcomes is that there are likely a host of unobserved variables, such as student ability, that influence both how much time a student spends on homework and the students achievement test scores. By combining instrumental variables estimation with an individual-fixed-effects specification, I control for both unobserved heterogeneity that is constant over time and time-varying heterogeneity that might influence the amount of time students spend on homework. This approach yields estimates of the impact of homework that are much larger than those of previous studies. The presence of time-varying unobserved factors is especially important to consider when estimating the impact of education inputs that are under the students control, since
1

Assigning homework seems to be less and less popular. A 2003 Brookings institute report claims that Since 2001, feature stories about onerous homework loads and parents fighting back have appeared in Time, Newsweek, and People magazines; the New York Times, Washington Post, Los Angeles Times, Raleigh News and Observer, and the Tampa Tribune(Loveless 2003). Their research indicates that the homework load for most US students is actually quite low.
2

Recent work done by economists (Aksoy and Link 2000; R. Stinebrickner and T. Stinebrickner 2008; Eren and Henderson 2009) have made progress, but their approaches can only address a subset of the endogeneity problems involved with measuring this effect. The difference between their results and those of previous scholars (Cooper, Robinson, and Patall 2006) indicates that endogeneity problems result in large biases.

students can respond in each period to changing incentives. Moreover in the presence of measurement error, this approach will yield more accurate results. I find that one extra hour of mathematics homework per week improves mathematics achievement by 0.179 standard deviations3. This change is large enough to move a student from the 50th percentile of math achievement to the 57th percentile4 over the course of a school year. Moreover, I find evidence that low-performing students and those in lower-performing schools realize much higher returns to their homework time than other students, and that a policy of assigning more homework disproportionately benefits these students. These findings lead to two important conclusions. First, it is possible for students to overcome the lower achievement that results from attending a low quality school by spending more time doing homework. Second, a policy that increases the amount of homework assigned is likely to reduce the gap between low and high achieving students.

II. Background The limited theoretic literature on homework predicts a positive impact of homework time on academic achievement (Betts 1997; Neilson 2005). Almost all of the empirical studies on this topic find evidence that homework time has a positive impact on academic achievement among high school students, although there is no consensus on the magnitude of the impact. The literature pertaining to this topic will be presented in two sections: first, a review of literature

The impact is reported here in terms of the standard deviation of the mathematics achievement test score used as the measure of academic achievement in this study. The sample used for this study is nationally representative, and thus the impacts can be interpreted as increases in achievement relative to the performance of similar students in the US.
4

The impacts are relative to students who do not increase their homework time.

from outside the economics discipline and second, a review of recent work done by applied economists. Within the fields of education, psychology, and sociology there has been much research on the impact of homework. Cooper et al. (2006) provide a good review of recent work in these fields. All of the published studies that Cooper et al. review that use multivariate regression analysis find similar results: increased homework causes a small increase in academic achievement. These studies, however, are limited to cross section analysis and do not try to take into account omitted inputs or characteristics. A number of scholars have also examined college students (Schuman et al. 1985; Rau and Durand 2000; Hill 1991); all find similar results when trying to document the impact of homework time on the performance. Each study either finds the relationship hard to document or finds a small effect of homework time on college grades. Applied economists have only recently started to pay attention to the impact of homework time on achievement, starting with Julian Betts 1996 analysis of the Longitudinal Study of American Youth. Since then Aksoy and Link (2000), Eren and Henderson (2008, 2009), and Stinebrickner and Stinebrickner (2008) have all examined this question. Each of these studies makes a serious attempt to address unobserved inputs. Eren and Henderson use a value added specification in the parametric portion of their 2008 paper,5 and a student and teacher fixed-effects approach in their 2009 paper. Both Betts and Aksoy and Link use a specification with student fixed effects, and Stinebrickner and Stinebrickner use the students roommates videogame ownership as an instrument, estimating the effect with two stage least squares.

The value added specification is a cross section regression with a lagged test score (or other dependent variable) included on the right hand side of the equation to control for past achievement, inputs, and student characteristics.

This study improves the estimation of the effect of homework on academic achievement in three ways. First, the primary shortcoming in this literature is that previous work fails to adequately account for unobserved inputs that influence both academic achievement and the amount of time spent on homework. None of these studies, with the exception of Stinebrickner and Stinebrickner6 can account for unobserved influences that vary over time or measurement error. To address these potential problems I use the amount of assigned homework and the students locus of control7 as instruments for student homework time, as well as a student fixed effects. Dealing with the endogeneity of students homework time in this manner greatly influences the estimated effects. Second, previous research has documented that the return to

homework time varies based on a students ability (Eren and Henderson 2008). In this study, I document that the return to homework also varies with school quality.

[Table 1] III. Data The National Education Longitudinal Study of 1988 The primary data used for the empirical work come from the National Education Longitudinal Study of 1988, a nationally representative longitudinal survey of students who were in the 8th grade in 1988. Follow up surveys were given in 1990 and 1992, with teacher and school counselor surveys in each wave, and parent surveys in the first and third waves. With each survey the students were given achievement exams in mathematics, science, English, and history. The exams are designed to allow comparison across waves, and to accurately test

Stinebrickner and Stinebrickner (2008) do a good job documenting the relationship between homework and achievement, but do so within a selected non-representative college age population. Their instrumental variable strategy, however, is not reproducible on a large scale.
7

The locus of control measures the extent to which a student believes that she can influence her own future outcomes. For a detailed description, see appendix A.

students at different achievement levels. Exam results are reported as a standardized variable with a t-distribution, with mean 5 and a standard deviation of 1. Of the 17,580 students who have at least two recorded mathematics exam scores, 12,534 students were not included in the sample because they did not have a mathematics instructor supplied data.8 Another 389 students were not included because they did not respond to the questions about homework time or locus of control. Moreover, 597 students were not included because of missing school-wide data on class size and teachers salaries. Finally 82 students were dropped because they did not have enough data on peers to construct peer performance variables. The resulting sample includes 8877 observations on 3978 students. The average student characteristics in this restricted sample do differ from the general sample on some key variables, both because of non-random attrition and non-random survey non-response. However, because the estimation strategy in this study focuses on accounting for unobserved student heterogeneity, many potential sampling-related biases are likely mitigated. Moreover, the results reported in this paper are qualitatively robust to significant changes in the sample. Summary statistics for the variables used in this paper are shown in table 1. For variable definitions, see appendix A.

IV. Estimating the impact of homework on achievement The standard approach to finding the impact of an input into a students academic achievement is to estimate an education production function similar to the following linear econometric model: (1)
8

Each student had one or more teachers surveyed in each wave, but the teacher was often not their mathematics instructor. Only students with a mathematics instructor interviewed in at least two waves can be used for this analysis.

where ATit is an achievement test score, which serves as a measure of human capital. Test scores are assumed to be a function of homework time (Hit), school and district inputs (Sit), teacher inputs (Tit), and individual characteristics (Xit). The alpha parameters represent the impact of the inputs on academic achievement, i is an error term that is constant over time, and is a time-

varying error term. In this specification the parameter of interest is 1, which represents the marginal impact of an additional hour of homework on academic achievement9. There are three reasons why an ordinary least squares estimation of equation 1 will likely return a biased estimate of the impact of homework. First, because the amount of homework that students complete is under the control of students, there may be permanent unobserved (to the econometrician) variables that have a large impact on academic achievement that may also influence a students optimal amount of time to spend on homework. For example, a students innate mathematics ability will certainly impact their mathematics achievement test scores, and will also likely impact the amount of mathematics homework that students complete. Similarly, all education inputs that occur before 8th grade will be constant within each student for the observed period of time, and will likely have a large impact on both achievement and homework time. This type of unobserved heterogeneity will be a source of correlation between the error term and the independent variables in the model, and create bias in the estimates of the associated parameters. A number of different types of possible unobserved heterogeneity are considered in the next section. The second type of omitted variable bias comes from transitory education shocks which will result in time-varying unobserved heterogeneity, such as an illness or other event that only impacts the student for one period but may influence a students academic achievement, their
9

For a more general discussion of the identification of education production function parameters see Todd and Wolpin (2003).

optimal study time, and other education inputs. Stinebrickner and Stinebrickner (2008) document that this type of shock will likely result in a downward bias in estimates of the impact of homework.10 The third source of bias is measurement error in the observed amount of homework that students complete. This variable results from a question asked of the student once per wave, are recorded categorically, and the questions and categories change slightly across waves. Moreover, students may not be able to, or desire to, give accurate replies when asked for the average amount of time that they spent on homework over the past year.11 One method for addressing these estimation problems is to include student fixed effects in the specification (use a within estimator). The individual fixed effect will absorb the timeinvariant individual specific error term (i) as well as any time-invariant observed and unobserved student characteristics. Thus even if there are multiple endogenous inputs other than homework in the education production function, the bias from time-invariant omitted variables such as student ability and education investments from before 8th grade - will be eliminated. Moreover, if the other inputs, such as teacher experience, certification, wages, or class size, are selected by students and parents, this selection is likely based on permanent student characteristics. As such, any bias caused by self selection is likely to be eliminated by using student fixed effects. The fixed effects estimation may produce estimates of the impact of homework ( ) that

are biased downward, however, for two reasons. First, the time varying omitted education shock
10

For example, if a student became seriously ill for one semester, their test scores would likely suffer, but at the same time, they might increase the amount of time spent on homework in an attempt to make up for missed school. This would create a spurious negative correlation between homework time and test scores. A similar process may be likely for other types of time-varying unobserved heterogeneity, such as a poor teacher-student match. For a larger discussion of this effect, see Stinebrickner and Stinebrickner (2008).
11

There is some evidence that students answers are not accurate: the sum of the reported hours spent on each subject is, on average, noticeably higher than the total amount of time students report spending on homework.

may be correlated with the transitory portion of the homework and test score variables. If this is the case, the fixed effects will magnify the impact of transitory changes in homework due to this shock. Second, the measurement error will now account for an even larger portion of the variation left in the observed amount of homework. This means that the bias caused by measurement error will be more serious in a fixed-effects specification than one without fixed effects.12 In order to address all of these problems I use instrumental variables to estimate a specification that includes student fixed effects, an approach recommended for education production function estimation in Todd and Wolpin (2003, 2007). The required instruments must affect student homework and be uncorrelated with the omitted variables. When used in a fixed effects specification, the student fixed effect is included in both the first and second stage equations, and the instruments used must vary over time. The use of instruments to predict homework time isolates the variation in the homework variable that is not correlated with the omitted variables and not due to measurement error. Estimating the effect of this exogenous variation in homework on achievement gives a consistent estimate of the impact of homework.

V. Instrument Validity The two variables used in this study as instruments are the amount of homework assigned by the students teacher and the students locus of control. Exogenous changes in the amount of homework assigned by the students instructor are unlikely to have any direct causal impact on test scores, apart from their effect on the amount of homework completed. The second variable

12

Certainty about the direction of the bias depends on some restrictive assumptions the measurement error will result in a bias towards zero if: i) the measurement error is uncorrelated with the regressor in question, and ii) the measurement error is independent of the measurement error associated with other regressors (Cameron and Trivedi 2005).

that is used to predict students homework is the locus of control. This variable measures the degree to which a student believes that she can impact her own future. A student with an internal locus is more likely to believe that her future depends on choices that she makes, while a student with an external locus is more likely to believe that external forces will dictate the events of her life.13 The first requirement of a good instrument is that it is correlated with the endogenous variable in question. Given the fixed effects specification used here, it must also be true in this case that within-student variation in the instruments predicts within student variation in the amount of homework that students complete. It is plausible that this condition holds for both variables. As teachers assign more homework students will likely spend additional time studying that particular subject. Also, previous studies have found that a strong internal locus of control is associated with positive outcomes in situations that require independent decision-making, such as positive health behaviors (Steptoe and Wardle 2001), lower dropout rates in distance education (Parker 1999), and success in web-based coursework (Wang and Newlin 2000). Thus it not surprising that students in this sample that have an internal locus, on average, spend more time doing homework, since they are more likely to expect the time investment today to pay off in the future. Moreover, the locus of control variables show significant variation within students over time, as might be expected for students at this age. [Table 2] Table 2 shows the first stage equations for the instrumental variables estimates of the education production function. The specification includes both homework and homework squared in the second stage equation, and the square of each instrument in the two first stage

13

The locus of control used here is a combination of students answers to three standard questions. For a detailed description of the construction of the locus of control variable, see appendix A.

10

equations for each specification. These results show that both hours of assigned homework and locus of control are important determinants of student homework time in each specification. Additionally, the last row in Table 3 shows the p-value for an F-test on the joint significance of the excluded instruments. The null hypothesis, that both variables have no impact on students homework, would be rejected with p-values below 0.01 for the first two specifications, and below 0.1 for the third specification which uses a significantly smaller sample. The second requirement for valid instruments is that they do not have an impact on student achievement except through their impact on homework. There are two types of traditional tests for instrument exogeneity, both of which indicate that these instruments are valid. First, the Hansen J-statistic is a chi-squared test for the exogeneity of all the instruments (or a test for overidentifying restrictions), and is appropriate for specifications that are estimated with standard errors robust to heteroskedasticity. The null hypothesis is that the excluded instruments are exogenous and this hypothesis cannot be rejected in either specification. The associated p-values are reported in the second to last row of table 2. As an additional test, I excluded only the assigned homework instruments from the main equation, and then excluded only the locus of control instruments. When included in the main equation, the homework instruments and locus of control instruments were jointly not statistically different from zero (p-values 0.94 and 0.93 respectively). This test, as well as the test of overidentifying restrictions collectively provide strong evidence that these excluded instruments do not predict student achievement independent of their effect on homework completion. Nevertheless, there are three levels of unobserved heterogeneity that could nevertheless result in a correlation between these instruments and student achievement: differences at the

11

student level, the classroom or teacher level, and the school or community level. I will address each in turn. [Table 3] Student level heterogeneity It is possible that the amount of homework assigned might reflect the students past achievement or ability. Much of this effect will likely be eliminated by the inclusion of student fixed effects. To test this, I estimate the impact of a lagged test score on the amount of homework students were assigned, including the teacher and school covariates on the right hand side. For example, if teachers assign more homework to students who have better academic backgrounds or more innate ability, we could expect past test scores to positively predict assigned homework. The results are shown in table 3. The estimates in the first column show that lagged test scores are strong predictors of each instrument when student fixed effects are not included. When the specification includes student fixed effects, however, the coefficient associated with the lagged test scores change sign and cannot be distinguished from zero. This indicates that neither past achievement or permanent ability are correlated with the instruments when fixed effects are included. It is also possible that both of the instruments are correlated with the same unobserved student or school characteristics and contain the same bias. In the last row and last column of table 4 the locus of control is regressed on the amount of assigned homework in specification with student fixed effects and period intercepts. In this specification the students locus of control has no significant effect on the amount of homework that the student is assigned. These variables, therefore, provide independent exogenous variation in homework time.

12

School or community level heterogeneity It is also possible that these instruments would end up being correlated with student achievement because of omitted school characteristics or unobserved community factors. To control for these, I create two school-specific peer variables equal to the average mathematics test score of all of the other observed students in the school, and another equal to the average amount of homework time that other students in the school complete. The averages calculated for these variables are student-specific since they omit the own students achievement and homework time. Also, they are created before limiting the sample for the purposes of the study. Adding these peer variables into the first and second stage equations should control for any unobserved school characteristics that are correlated with both the amount of homework assigned or the locus of control and the students performance.

Classroom or teacher level heterogeneity Another concern is that assigned homework might be biased by teacher-specific unobserved characteristics, and that even within schools, the variation in teacher quality might impact students studying habits and performance. To control for these influences I create peer variables defined at the teacher level instead of the school level. Doing so greatly reduces the usable observations because the amount of clustering within teachers is lower than the clustering within schools in the sample design, especially in the second and third waves. The last column of table 4 shows the results for the specification that includes both the school and teacher level peer variables. Despite the smaller sample, the results are very similar. A related concern is that teachers might assign more homework to some classes than others, based on their appraisal of the students ability and motivation. This could be the case if

13

there was tracking in the students math classes or if some students took advanced placement or remedial courses. To the extent that this homework assignment is based on student and peers achievement, the fixed effects and peer effects should minimize this bias. As an additional control, however, I include a set of indicator variables that tell what achievement level the teacher considers the students in the class to be at, given the following categories: higher level, average, lower level, or widely differing. This control is an important one in Eren and Hendersons analyses (2008, 2009). I include these in every specification, though none of these variables end up being significant in second stage equation, indicating that the fixed effects and peer variables do a good job capturing these achievement level differences.14

[Table 4] VI. Results: The Impact of Mathematics Homework on Mathematics Achievement Table 4 displays the pooled cross section OLS, fixed-effects, and fixed-effects-two-stageleast-squares estimates of the education production function. The empirical model employed includes a quadratic in homework time to capture decreasing returns to studying. The marginal effect of an hour of homework (evaluated at the mean observed weekly mathematics homework time) is reported at the bottom of table 4. Each specification also includes wave indicator variables, the average observed class size and three indicators of teacher quality: whether the students teachers are certified in their subject, an indicator that the teacher has less than 4 years of experience, and the minimum teacher pay in the students school.

14

Because these class level indicators do a good job predicting student homework time, but do not seem to impact achievement in a fixed effects specification, I also estimated a specification where these were added to the set of instruments excluded from the second stage. The results were very similar to those reported in the other specifications in table 4.

14

The ordinary least squares pooled cross section estimate of the impact of an hour of mathematics homework per week, shown in column one, is a 0.049 standard deviation increase in the students mathematics test score, which corresponds to an improvement of about 2 percentile points.15 This estimate may be biased if the students unobserved prior education inputs or innate abilities influence the amount of time spent on homework, or if students select other inputs based on unobservable characteristics. In order to control for these endogeneity issues, a fixed effects specification is employed. If the selection is based on student characteristics that do not change over time, then the studentfixed effects estimate will eliminate the bias not only in the impact of hours of homework, but also in the other inputs. This estimator will also control for the linear effects of any differences in past inputs that are not included in these specifications. The second column in table 4 shows the estimates of the same specification as the one shown in the first column, but with student fixed effects included. These estimates are much smaller. The impact of an hour of homework with student fixed effects is almost 1/5th the size of the OLS estimates presented in the first column. As explained in section four, the fixed effects estimates will likely be biased by presence of measurement error and time-varying unobserved heterogeneity. These problems can be addressed by using instrumental variables in addition to the student fixed effects. The third column in table 4 shows the results of the student fixed effects specification estimated using twostage least squares, where the excluded instruments are the amount of homework assigned by the students teachers and the students locus of control, as well as the square of each instrument. This estimate, while less precisely estimated than the fixed-effects estimate, is much higher, and

15

Starting from the 50th percentile in Mathematics test scores within the unrestricted NELS sample.

15

gives a quantitatively and qualitatively different result. A student doing an average amount of mathematics homework who completes an extra hour of homework each week is estimated to improve her mathematics test score by 0.182 standard deviations, an improvement of 7 percentile points. Even if we accept the lower bound of the 95% confidence interval for this estimate, which is 0.06 standard deviations, the return to homework is still 6 times higher than the fixed effects estimate. This lower bound estimate indicates that an hour of homework per week would move a student from the 50th percentile to the 52nd percentile in mathematics achievement. If the instruments provide exogenous variation in student homework time, the estimates in the third column should not change substantially when other inputs are included in the specification. Column four tests this by adding variables to control for other school-wide inputs and peer group influences. The specification shown in column four includes the average amount of homework done by the students peers across the school,16 and the average mathematics test score of the students peers.17 Though these variables are strong predictors both of students homework time and test scores, the estimate of the impact of homework remains almost exactly the same. Other recent studies that have estimated the impact of homework time on student achievement have not been able to account for both time-varying and invariant unobserved characteristics. These studies found much smaller impacts.18 The results reported here indicate

16 17

The students peers, in this case, are any students with observed test scores from the same school in the same year. (Manski 1993) and (Hanushek et al. 2003) argue that peer effects of this type can introduce a bias due to the reintroduction of a students ability through the students influence on her peers. The estim ated impact of homework does not change when these variables are included, indicating that if these variables are problematic, they do not bias the estimate of the impact of homework. Specifications that instrument for these peer effects using lagged values produce very low precision estimates that cannot be differentiated statistically from the estimates reported here.
18

The estimates presented in this paper are larger in magnitude than any the (Cooper, Robinson, and Patall 2006) review. Aksoy and Link also used the NELS data, and employed student fixed effects. See (Aksoy and Link 2000) table three specification 1. If their estimate is divided by 13, which is the approximate standard deviation reported

16

that the use of student fixed effects to control for omitted inputs and characteristics has consistently produced estimates that are too low. This is likely the case because of a downward bias that results from measurement error, as well as downward bias caused by students responding to negative or positive time varying unobserved heterogeneity. It is also possible that the impact of unobserved past inputs and characteristics biased the result downward, although the direction of bias from this source is uncertain. In light of this, the correction provided by the instrumental variables estimate is especially valuable. It is also worth considering the rate at which the return on homework diminishes. For students currently not spending any time doing mathematics homework, the return on completing one hour of homework per week would be about a 10 percentile point improvement in mathematics test scores. This return decreases as students spend more time studying. These results indicate that, on average, mathematics homework ceases to improve test scores if a student does more than about 11 hours per week. By this standard, the vast majority of students could increase their test scores if they spent more time on their mathematics homework.

VII. The Impact of Assigned Homework on Academic Achievement This strong impact of homework time indicates that any policy that increases the amount of homework students complete could be valuable. One such policy instrument is the amount of homework that teachers assign. Teachers assignments, even if only completed a fraction of the time, can have a strong influence on the amount of homework students complete. To illustrate this, I estimate a reduced form version of the specification displayed in column 4 of table 4, which is similar to the regression in column 2 of table 4, with assigned homework instead of

in table 1, they find that an hour of homework increases mathematics achievement by 0.051 standard deviations. The results of this study indicate that the true effect is much larger.

17

homework completed as the variable of interest. This specification includes student fixed effects, school-wide peer variables, teacher characteristics, class size, class level dummy variables and wave (grade) specific intercepts. The estimated marginal effect of being assigned an additional hour of homework for students currently being assigned the average amount (2.9 hours per week) is 0.015 (standard error 0.004) test score standard deviations. This effect size is notably smaller than the estimated impact of a student completing an hour of homework, as is expected since homework assignments are an imperfect predictor of homework completion. This effect size is also well within the range of similar estimates in other studies (Aksoy and Link 2000; Betts 1996; Eren and Henderson 2008; 2009).

VIII. The Impact of Homework Time by Achievement Level and School Quality In this section, I will try to relax the restriction that students at different levels of achievement or in different types of schools receive the same return on homework time. To do this I estimate the specification from the table 4 column 4 separately for high achieving and low achieving students, as well as high and low performing schools. [table 5] First, students are separated into two groups of equal size based on their 8th grade science test scores. This distinction yields a small difference that is not-statistically significant, but indicates that students at the bottom half of the population in terms of academic performance might see higher returns to homework completion. This evidence does not support the common assumption that more able students receive a higher return to homework time because their time is more productive (Eren and Henderson 2008; Neilson 2005). 19 Second, students were split

19

Eren and Henderson (2008) find higher returns to homework at the top and bottom of the achievement distribution. When I split the sample into more than two groups, I find results very similar to those reported: low

18

into groups of equal size based on the test score performance of their peers, as a general measure of school quality. This distinction yielded a much larger, but still not statistically significant difference in the estimated effect. Students in the lower performing schools experienced a return to homework time that was much larger than those in high performing schools. A student in a low performing school, who currently studies one hour each week that wanted to move from the 50th percentile to the 75th percentile, would have to study about 2 additional hours per week, whereas a student in a high performing school would have to study 4 additional hours to improve the same amount. This may indicate that at low performing schools homework is a stronger determinant of success. One explanation for such a result comes from Bishop (2006), who reports that less material is presented in class time in low performing schools, which may partially explain why studying at home would be more beneficial for these students. At the very least, this evidence does not support the assumption by some education scholars (Kralovec and Buell 2000; Kohn 2006) that homework will only widen achievement gaps. [table 6] Finally, it is worth exploring whether assigning more homework will have a differential impact based on school quality and prior student achievement. Table 6 shows the impact of this policy for the top and bottom halves of the achievement and school quality distributions. The effect is estimated with similar precision for each subgroup, but the effect is much smaller for high performing students and students in high performing schools, though once again, the differences are not statistically significant. Here also there is a noticeable difference in the impact between high and low achieving students, based on their science test scores when entering the sample. The relative effectiveness of assigned homework to low achieving students

achieving students and those in low performing schools have higher returns, moreover, the instrumental variables estimates are not precise enough for the small samples required to investigate a more detailed distributional impact.

19

relative to high achieving students is partially due to the fact that low achieving students are somewhat more likely to increase their homework time if assigned additional homework.

IX. Conclusions The results in this paper strongly support the argument that increasing the time that students spend on their homework will have visible impacts on academic achievement in mathematics. Furthermore, there is some evidence that increasing homework time could improve the performance of low-performing students and those in low-performing schools. Because the sample used is nationally representative, the results should be applicable to a number of school settings, and thus as a policy tool, increasing the amount of homework that students complete shows some promise. Additionally, because much of the benefit of assigning additional homework seems to be realized by low-achieving students and students in lowperforming schools, this policy could be useful for lowering the achievement gap between high achieving and low achieving students.

20

References Aksoy, Tevfik, and Charles Link. 2000. A panel analysis of student mathematics achievement in the US in the 1990s: does increasing the amount of time in learning activities affect math achievement?. Economics of Education Review 19(3): 261-277. Bennett, Sara, and Nancy Kalish. 2006. The Case Against Homework. Crown Publishers. Betts, Julian. 1996. The role of homework in improving school quality. University of California, San Diego, Department of Economics. Bishop, John. 2006. Drinking from the Fountain of Knowledge: Student Incentive to Study and Learn Externalities, Information Problems, and Peer Preasure. In Handbook of the economics of education, eds. Eric Alan Hanushek and Finis Welch. Elsevier. Cameron, A.C., & Trivedi, P. 2005. Microeconometrics: Methods and Applications. Cambridge University Press. Cooper, Harris, Jorgianne Civey Robinson, and Erika Patall. 2006. Does Homework Improve Academic Achievement? A Synthesis of Research, 1987-2003. Review of Educational Research 76(1): 1-62. Eren, Ozkan, and Daniel Henderson. 2008. The Impact of Homework on Student Achievement. Econometrics journal 11(2): 326-348. Eren, Ozkan, and Daniel Henderson. 2009. Are We Wasting Our Childrens time by giving them more homework? Working Paper.

21

Hanushek, Eric. 1999. The Evidence on Class Size. In Earning and learning, eds. Susan E. Mayer and Paul E. Peterson. Brookings Institution Press. Hanushek, Eric, John Kain, Jacob Markman, and Steven Rivkin. 2003. Does Peer Ability Affect Student Achievement?. Journal of Applied Econometrics 18(5): 527-544. Hill, Lester, Jr. 1991. Effort and reward in college: a replication of some puzzling findings. In Replication Research in the Social Science, ed. James W Neuliep. Newbury Park, CA: Sage, p. 139-56. Kohn, Alfie. 2006. The homework myth. Da Capo Press. Kralovec, Etta, and John Buell. 2001. The End of Homework. Beacon Press. Loveless, Tom. 2003. How Well are American Students Learning? Brookings Institution. Manski, Charles F. 1993. Identification of Endogenous Social Effects: The Reflection Problem. The Review of Economic Studies 60(3): 531-542. Neilson, William. 2005. Homework and performance for time-constrained students. Economics Bulletin 9(1): 1-6. Parker, Angie. 1999. A Study of Variables that Predict Dropout from Distance Education.. International Journal of Educational Technology 1(2): 1-10. Rau, William, and Ann Durand. 2000. The Academic Ethic and College Grades: Does Hard Work Help Students to "Make the Grade"?. Sociology of Education 73(1): 19-38.

22

Schuman, Howard, Edward Walsh, Camille Olson and Barbara Etheridge. 1985. Effort and Reward: The Assumption That College Grades are Affected by Quantity of Study. Social Forces 63(4): 945-966. Steptoe, Andrew, and Jane Wardle. 2001. Locus of control and health behaviour revisited: a multivariate analysis of young adults from 18 countries. British Journal of Psychology 92(Pt 4): 659-672. Stinebrickner, Ralph, and Todd Stinebrickner. 2008. The Causal Effect of Studying on Academic Performance. The B.E. Journal of Economic Analysis & Policy 8(1). Todd, Petra, and Kenneth Wolpin. 2003. On the Specification and Estimation of the Production Function for Cognitive Achievement. The Economic Journal 113(485): F3-F33. Todd, Petra, and KennethWolpin. 2007. The Production of Cognitive Achievement in Children: Home, School, and Racial Test Score Gaps. Journal of Human Capital 1(1): 91-136. Wang, Alvin, and Michael Newlin. 2000. Characteristics of Students Who Enroll and Succeed in Psychology Web-based Classes. Journal of Educational Psychology 92(1): 137-43.

23

Tables Table 1 Selected Summary Statistics Wave 1 Mean SD Hours of mathematics homework per week 1.48 1.67 Hours of assigned homework per week 2.45 1.30 Mathematics test score 4.69 0.82 Class size 23.96 5.11 Teacher is certified in mathematics 0.85 0.36 Math teacher has 3 or fewer years of experience 0.09 0.29 Minimum annual teacher wage 28.77 5.13 Locus of control 0.09 0.57 Average of peers' test score 4.52 0.41 Average peers' homework time 5.35 1.54 Number of students 3704

Wave 2 Mean SD 5.29 4.69 3.29 2.20 5.24 0.95 23.48 5.24 0.99 0.07 0.11 0.31 29.49 5.38 0.06 0.61 5.10 0.48 7.28 2.32 3121

Wave 3 Mean SD 5.70 4.19 3.05 1.36 5.89 0.95 27.52 16.34 0.99 0.07 0.04 0.18 31.28 6.94 0.14 0.62 5.51 0.53 13.01 3.32 2052

Note. - This data is a sample of students from the National Education Longitudinal Study of 1988. Wave 1 corresponds to students in the 8th grade, wave 2 to the 10th grade, and wave 3 to the 12th grade. All monetary variables are measured in thousands of dollars, and inflation adjusted to 2005 dollars.

24

Table 2 First Stage Equation Estimation Results: Determinants of Homework Time Corresponding column in table 4 3 4 5 Dependent variable Class size (in units of 10 students) Certified teacher Inexperienced teacher Minimum annual teacher compensation (in thousands) Class level - average Class level - high Class level - differs Homework hours assigned by math teacher* Locus of control* Assigned homework hours squared* Locus of control squared* Observations Student fixed effects School level peer characteristics Teacher level peer characteristics Hansen J statistic (Chi-Sq pvalue) P-value for F-test of the joint impact of excluded instruments HW -0.008 (0.006) 0.202 (0.207) 0.018 (0.175) -0.036 (0.013) 0.407 (0.163) 0.900 (0.195) 0.544 (0.209) 0.163 (0.051) 0.003 (0.115) -0.005 (0.003) 0.226 (0.104) 8770 yes no no HW2 -0.166 (0.110) 2.588 (4.564) 0.151 (3.451) -0.623 (0.266) 4.410 (3.190) 12.970 (3.957) 8.549 (4.204) 2.377 (0.981) -3.513 (2.743) -0.070 (0.056) 5.839 (2.261) 8770 yes no no 0.933 0.003 0.008 0.003 HW -0.008 (0.006) 0.204 (0.207) 0.009 (0.174) -0.025 (0.013) 0.426 (0.162) 0.928 (0.194) 0.574 (0.209) 0.157 (0.051) -0.007 (0.115) -0.005 (0.003) 0.242 (0.104) 8770 yes yes no HW2 -0.178 (0.109) 2.861 (4.614) -0.108 (3.436) -0.465 (0.268) 4.675 (3.170) 13.430 (3.944) 8.770 (4.229) 2.252 (0.982) -3.649 (2.748) -0.063 (0.055) 6.076 (2.273) 8770 yes yes no 0.949 0.009 0.07 HW -0.009 (0.007) 0.155 (0.264) 0.043 (0.224) -0.004 (0.017) 0.712 (0.228) 1.258 (0.262) 0.921 (0.288) 0.082 (0.064) -0.009 (0.153) -0.004 (0.003) 0.346 (0.139) 5453 yes yes yes HW2 -0.198 (0.140) 0.377 (5.750) 0.858 (4.667) -0.153 (0.347) 11.000 (4.523) 21.110 (5.362) 14.760 (5.852) 0.918 (1.268) -4.840 (3.831) -0.055 (0.048) 9.642 (3.195) 5453 yes yes yes 0.88 0.032

Note. - The asterisk denotes instruments excluded from the second stage regression. Standard errors are shown in parentheses. All regressions include wave indicators. The sample includes 8877 observations on 3978 students. The first stage regressions are shown in table 4. 25

Table 3 Testing Instrument Validity Dependent Variable Assigned HW Locus of Control Assigned HW Independent Variable Lagged Test Score Lagged Test Score Locus of Control Without Student Fixed Effects 0.223 (0.043) 0.094 (0.015) With student Fixed Effects 0.052 (0.194) -0.028 (0.050) 0.023 (0.049)

Note. - Standard errors are shown in parentheses. Each cell displays estimates from a separate regression, each of which includes the full set of covariates included in the main specifications. All standard errors are robust to the presence of arbitrary heteroskedasticity, and are clustered at the level of the individual.

26

Table 4 Education Production Function Estimation Results 1 2 3 4 Estimation Method Hours of math homework per week Hours of math homework per week squared and divided by 10 Class size (in units of 10 students) Certified teacher Inexperienced teacher Minimum annual teacher compensation (in thousands) Class level - average Class level - high Class level - differs Marginal Effect of Homework evaluated at the mean Observations Student fixed effects School level peer characteristics Teacher level peer characteristics OLS 0.069 (0.006) -0.003 (0.000) -0.005 (0.001) 0.133 (0.032) -0.075 (0.029) 0.003 (0.002) 0.593 (0.027) 1.348 (0.030) 0.557 (0.035) 0.049 (0.004) 8770 no no no OLS 0.014 (0.003) -0.001 (0.000) -0.001 (0.001) -0.026 (0.023) 0.017 (0.018) 0.002 (0.001) 0.073 (0.018) 0.136 (0.021) 0.095 (0.023) 0.010 (0.002) 8770 yes no no 2SLS 0.259 (0.086) -0.011 (0.004) -0.001 (0.001) -0.047 (0.033) 0.010 (0.026) 0.005 (0.002) 0.013 (0.033) 0.041 (0.048) 0.046 (0.038) 0.182 (0.059) 8770 yes no no 2SLS 0.260 (0.088) -0.012 (0.005) -0.001 (0.001) -0.031 (0.033) 0.004 (0.025) 0.003 (0.002) 0.010 (0.033) 0.046 (0.048) 0.031 (0.038) 0.179 (0.060) 8770 yes yes no

5 2SLS 0.237 (0.107) -0.009 (0.004) -0.001 (0.001) -0.028 (0.041) -0.007 (0.031) 0.000 (0.003) -0.031 (0.055) -0.012 (0.082) -0.018 (0.067) 0.176 (0.082) 5453 yes yes yes

Note. - Standard errors are shown in parentheses. The dependent variable for each regression is the mathematics achievement test score with a standard deviation of one. All regressions include wave dummy variables. Column one is a pooled regression with the standard errors clustered at the level of the individual. All standard errors are robust to the presence of arbitrary heteroskedasticity. The first stage regressions are shown in table 2.

27

Table 5 Marginal Effect of Homework Time by Student Achievement and School Quality Group 8th Grade Science Test Score 0.148 (0.065) 0.182 (0.023) School Quality 0.119 (0.046) 0.250 (0.172)

Top Half Bottom Half

Note. - Standard errors are shown in parentheses. The marginal effects are all evaluated at the mean value of homework for the whole population. Each cell is the instrumental variables estimate of the marginal effect of an hour of homework completed on mathematics test scores. Teacher experience, certification, salary, class size, class level controls, peer effects, and wave indicators were also included in each specification. Prior achievement is measured using the students' 8th grade science test score. School quality is measured using the average mathematics test score of other observed students in the same school. All standard errors are robust to the presence of arbitrary heteroskedasticity.

28

Table 6 Marginal Effect of Assigned Homework on Mathematics Achievement by Prior Achievement and School Quality Group 8th Grade Science Test Score 0.007 (0.005) 0.018 (0.006) School Quality 0.008 (0.006) 0.016 (0.007)

Top Half Bottom Half

Note. - Standard errors are shown in parentheses. For each result, a quadratic in assigned homework is used to predict mathematics achievement in a student fixed effect regression specification. The marginal effects are all evaluated at the mean value of assigned homework for the whole population. Teacher experience, certification, salary, class size, class level controls, peer effects, and wave indicators were also included in each specification. Prior achievement is measured using the students' 8th grade science test score. School quality is measured using the average mathematics test score of other observed students in the same school. All standard errors are robust to the presence of arbitrary heteroskedasticity.

29

Appendix A: Variable Definitions

Mathematics test scores: The cognitive tests were given to students with each survey, raw scores are not used, but instead the scores are normalized across waves before sample restrictions were imposed - with a standard deviation across all waves approximately equal to 1. The tests were created to allow comparison across grades, and to accurately measure student achievement even if students do not complete the entire exam. Hours of mathematics homework: This variable measures the number of hours that the student spends per week on mathematics homework. The NELS data records the hours of homework completed in a series of categories. Each student is assigned the mean value for their category. Class size: This is defined as the average size of the observed classes taken by the student in each wave. Teacher has certification in field: This variable equals one if the mathematics teachers surveyed for this student to teach mathematics, and zero otherwise. Inexperienced teacher: This indicator variable is equal to one if the teacher interviewed has three or less years of experience. The source variables did not allow for separate indicators for one, two, and three years of experience. Minimum teachers wage: This variable comes from the school counselors survey, and records the minimum annual wage for a teacher in the school that the student is attending. This variable is adjusted so that it is measured in units of $1000 inflation-adjusted 2005 dollars. Assigned homework: This is a continuous variable recording the average hours per week that the students interviewed mathematics teacher assigned. The source variables were all continuous, though the base year was a weekly variable and the second and third waves asked for daily amounts. I multiplied the daily homework amounts by 5 to get the weekly amount. 30

Locus of control: This is a composite of three questions, which measures the degree to which the student has an internal locus of control. Students with an internal locus believe that their actions and choices can shape their future, where students with an external locus believe external events will be the primary determinants of what their future is like. A higher number indicates that the students locus is more internal. The questions ask the student to agree or disagree (five point scale) with the following statements: In my life, good luck is more important than hard work for success. Every time I try to get ahead, something or somebody stops me. My plans hardly ever work out, so planning only makes me unhappy. Average of peers Test scores: These variables are the average mathematics test score of all other observed students in the same school (or who have the same teacher) in the same year. This variable was created prior to imposing sample restrictions. Average of peers homework time: These variables are the average total hours of homework reported by all other observed students in the same school (or who have the same teacher) in the same year. This variable was created prior to imposing sample restrictions. Class Level Indicator Variables: The three class level indicators come from a question of each teacher that asks: Which of the following best describes the achievement level of the students in this class compared with the average 8th (10th, 12th) grade student in this school? where the options for response are Higher Levels, Average Levels, Lower Levels, or Widely Differing.

31