Does Homework Improve Academic Achievement? - Semantic Scholar

Does Homework Improve Academic Achievement? A Synthesis of Research, 1987-2003 Cooper, Harris;Jorgianne Civey Robinson;Patall, Erika A Review o f Educational Research; Spring 2006; 76, 1; ProQuest pg. 1

Review of Educational Research Spring 2006, Vol. 76, No. 1, pp. 1-62

Does Homework Improve Academic Achievement? A Synthesis of Research, 1987-2003 Harris Cooper, Jorgianne Civey Robinson, and Erika A. Patall Duke University In this article, research conducted in the United States since 1987 on the effects o f homework is summarized Studies are grouped intofour research designs. The authorsfound that all studies, regardless of type, had designflaws. However, both within and across design types, there was generally consistent evidencefor a positive influence o f homework on achievement. Studies that reported sim ple homework-achievement correlations revealed evidence that a stronger correlation existed (a) in Grades 7-12 than in K-6 and (b) when students rather than parents reported time on homework. No strong evidence wasfound for an association between the homework-achievement link and the outcome measure (grades as opposed to standardized tests) or the subject matter (reading as opposed to math). On the basis o f these results and others, the authors suggest future research. K eywords : homework, meta-analysis.

Homework can be defined as any task assigned by schoolteachers intended for students to carry out during nonschool hours (Cooper, 1989). This definition explic itly excludes (a) in-school guided study; (b) home study courses delivered through the mail, television, audio or videocassette, or the Internet; and (c) extracurricular activities such as sports and participation in clubs. The phrase “intended for stu dents to carry out during nonschool hours” is used because students may com plete homework assignments during study hall, library time, or even during subsequent classes. Variations in homework can be classified according to its (a) amount, (b) skill area, (c) purpose, (d) degree of choice for the student, (e) completion deadline, (f) degree of individualization, and (g) social context. Variations in the amount of homework can appear as differences in both the frequency and length of individual assignments. Assignments can range over all the skill areas taught in school. The purposes of homework assignments can be divided into (a) instructional and (b) noninstructional objectives (cf. Epstein, 1988,2001; Epstein & Van Voorhis, 2001; Lee & Pruitt, 1979). The most common instructional purpose of homework is to provide the student with an opportunity to practice or review material that has already been presented in class (Becker & Epstein, 1982). Preparation assignments introduce material to help students obtain the maximum benefit when the new material is covered in class (Muhlenbruck, Cooper, Nye, & Lindsay, 1999). Exten sion homework involves the transfer of previously learned skills to new situations 1

L Reproduced with perm ission o f the copyright owner. Further reproduction prohibited without permission.

Cooper et al.

(Lee & Pruitt, 1979). Finally, homework can require students to integrate separately learned skills and concepts (Lee & Pruitt, 1979). This might be accomplished using book reports, science projects, or creative writing. Homework has other purposes in addition to enhancing instruction. It can be used to (a) establish communication between parent and child (Acock & Demo, 1994; Balli, Demo, & Wedman, 1998; Epstein, Simon, & Salinas, 1997^}onzalez, Andrade, Civil, & Moll, 2001; Scott-Jones, 1995; Van Voorhis, 2003); (b) fulfill directives from school administrators (Hoover-Dempsey, Bassler, & Burow, 1995); and (c) punish students (Epstein & Van Voorhis, 2001; Xu & Como, 1998). To this list might be added the public relations objective of simply informing parents about what is going on in school (Coleman, Hoffer, & Kilgore, 1982; Como, 1996; Rutter, Maughan, Mortimore, & Ouston, 1979). Homework assignments rarely reflect a single purpose. Rather, most assignments serve several different purposes; some relate to instruction, whereas others may meet the purposes of the teacher, the school administration, or the school district. The degree o f choice afforded a student refers to whether the homework assign ment is compulsory or voluntary. Related to the degree of choice, completion dead lines can vary from short term, meant to be completed overnight or for the next class meeting, to long term, with students given days or weeks to complete the task. The degree o f individualization refers to whether the teacher tailors assignments to meet the needs of each student or whether a single assignment is presented to groups of students or to the class as a whole. Finally, homework assignments can vary accord ing to the social context in which they are carried out. Some assignments are meant for the student to complete independent of other people. Assisted homework explic itly calls for the involvement of another person, a parent or perhaps a sibling or friend. Still other assignments involve groups of students working cooperatively to produce a single product.

Overview The Importance o f Homework and Homework Research Homework is an important part of most school-aged children’s daily routine. According to the National Assessment of Educational Progress (Campbell et al., 1996), over two-thirds of all 9-year-olds and three-quarters of all 13- and 17-yearolds reported doing some homework every day. Sixteen percent of 9-year-olds reported doing more than 1 hour of homework each day, and this figure jumped to 37% for 13-year-olds and 39% for 17-year-olds. More recent surveys support the extensive use of homework, although the amount of homework that students report varies from study to study, depending perhaps on how the question is asked. For example, Gill and Schlossman (2003) reported recent declines in time spent on homework. However, among the youngest students, age 6 to 8 , homework appears to have increased between 1981 (52 minutes weekly) and 1997 (128 minutes weekly; Hofferth & Sandberg, 2000). Homework likely has a significant impact on students’ educational trajectories. Most educators believe that homework can be an important supplement to in-school academic activities (Henderson, 1996). However, it is also clear from the surveys mentioned earlier that not all teachers assign homework and/or not all students com plete the homework they are assigned. This suggests that whatever impact homework 2

Reproduced with perm ission o f the copyright owner. Further reproduction prohibited without permission.

Does Homework Improve Academic Achievement?

might have on achievement varies from student to student, depending on how much each student is assigned or completes. Homework is often a source of friction between home and school. Accounts of con flicts between parents and educators appear often in the popular press (e.g., Ratnesar, 1999; Coutts, 2004; Kralovec & Buell, 2000; Loveless, 2003). Parents protest that assignments are too long or too short, too hard or too easyôr too ambiguous (Baumgartner, Bryan, Donahue, & Nelson, 1993; Kralovec & Buell, 2000; Warton, 1998). Teachers complain about a lack of support from parents, a lack of training in how to construct good assignments, and a lack of time to prepare effective assign ments (Farkas, Johnson, & Duffet, 1999). Students protest about the time that home work takes away from leisure activities (Coutts, 2004; Kralovec & Buell, 2000). Many students consider homework the chief source of stress in their lives (Kouzma & Kennedy, 2002). To date, the role of research in forming homework policies and practices has been minimal. This is because the influences on homework are complex, and no simple, general finding applicable to all students is possible. In addition, research is plentiful enough that a few studies can always be found to buttress whatever position is desired, while the counter-evidence is ignored. Thus advocates for or against homework often cite isolated studies either to support or to refute its value. It is critical that homework policies and practices have as their foundation the best evidence available. Policies and practices that are consistent with a trustworthy synthesis of research will (a) help students to obtain the optimum education benefit from homework, and (b) help parents to find ways to integrate homework into a healthy and well-rounded family life. It is our intention in this article to collect as much of the research as possible on the effects of homework, both positive and negative, conducted since 1987. We will apply narrative and quantitative techniques to integrate the results of studies (see Cooper, 1998; Cooper & Hedges, 1994). While research rarely, if ever, covers the gamut of issues and circumstances confronted by educators, we hope that the results of this research synthesis will be used both to guide future research on homework and to assist in the development of policies and practices consistent with the empirical evidence. A Brief History o f Homework in the United States Public attitudes toward homework have been cyclical (Gill & Schlossman, 1996, 2004). Prior to the 20th century, homework was believed to be an important means for disciplining children’s minds (Reese, 1995). By the 1940s, a reaction against homework had set in (Nash, 1930; Otto, 1941). Developing problem-solving abilities, as opposed to learning through drill, became a central task of education (Lindsay, 1928; Thayer, 1928). Also, the life-adjustment movement viewed home study as an intrusion on other at-home activities (Patri, 1925; San Diego City Schools Research Department, 1936). The trend toward less homework was reversed in the late 1950s after the Russians launched the Sputnik satellite (Gill & Schlossman, 2000; Goldstein, 1960; Epps, 1966). Americans became concerned that a lack of rigor in the educational system was leaving children unprepared to face a complex technological future and to compete against our ideological adversaries. Homework was viewed as a means of accelerating the pace of knowledge acquisition. But in the mid-1960s the cycle again reversed itself (Jones & Colvin, 1964). Homework came to be seen as a .

3 m

{ Reproduced with perm ission o f the copyright owner. Further reproduction prohibited w ithout permission.

Cooper et al.

symptom of excessive pressure on students. Contemporary learning theories again questioned the value of homework and raised its possible detrimental consequences for mental health. By the mid-1980s, views of homework had again shifted toward a more positive assessment (National Commission on Excellence in Education, 1983). In the wake of declining achievement test scores and increased conggm about American’s ability to compete in a global marketplace, homework underwent its third renais sance in 50 years. However, as the century turned, and against the backdrop of con tinued parental support for homework (Public Agenda, 2000), a predicable backlash set in, led by beleaguered parents concerned about the stresses on their children (Winerip, 1999). Past Syntheses o f Homework Research Homework has been an active area of study among American education researchers for the past 70 years. As early as 1927, a study by Hagan (1927) compared the effects of homework with the effects of in-school supervised study on the achievement of 11- and 12-year-olds. However, researchers have been far from unanimous in their assessments of the strengths and weaknesses of homework. For example, more than a dozen reviews of the homework literature were conducted between 1960 and 1987 (see Cooper, 1989, for a detailed description). The conclusions of these reviews varied gready, partly because they covered different literature, used different cri teria for inclusion of studies, and applied different methods for the synthesis of study results. Cooper (1989) conducted a review of nearly 120 empirical studies of homework’s effects and the ingredients of successful homework assignments. Quantitative syn thesis techniques were used to summarize the literature. This review included three types of studies that help answer the general question of whether homework improves students’ achievement. The first type of study compared achievement of students given homework assignments with students given no homework. In 20 studies conducted between 1962 and 1986,14 produced effects favoring homework while 6 favored no homework. Most interesting was the influence of grade level on home work’s relation with achievement. These studies revealed that the average high school student in a class doing homework outperformed 69% of the students in a no-homework class, as measured by standardized tests or grades. In junior high school, the average homework effect was half this magnitude. In elementary school, homework had no association with achievement gains. The next type of evidence compared homework with in-class supervised study. Overall, the positive effect of homework was about half what it was when students doing homework were compared with those not doing homework. Most important was the emergence once again of a strong grade-level effect. When homework and in-class study were compared in elementary schools, in-class study proved superior. Finally, Cooper found 50 studies that correlated the amount of time students spent on homework with a measure of achievement. Many of these correlations came from statewide surveys or national assessments. In all, 43 correlations indicated that students who did more homework had better achievement outcomes, while only 7 indicated negative outcomes. Again, a strong grade-level interaction appeared. For students in elementary school, the average correlation between amount of 4

Reproduced with perm ission o f the copyright owner. Further reproduction prohibited w ithout permission.


homework and achievement was nearly r = 0 ; for students in middle grades it was r = .07; and for high school students it was r = .25. The Need fo r a New Synthesis o f the Homework Literature There are three reasons for conducting a new synthesis of the homework literature: (a) to update the evidence on past conclusions about the effectôf homework and determine if the conclusions from research need modification; (b) to determine whether some of the questions left unanswered by the earlier syntheses can now be answered; and (c) to apply new research synthesis techniques. In the years since the completion of Cooper’s (1989) meta-analysis, a substantial new body o f evidence has been added to the homework literature. For example, a search of ERIC, PsycINFO, Sociological Abstracts, and Dissertation Abstracts between January 1987 (when the search for the earlier synthesis ended) and Decem ber 2003 indicated that over 4,000 documents with homework as a keyword had been added to these reference databases. When we delimited this search to documents that the reference engine cataloged as “empirical,” nearly 900 documents remained. Yet we know of no comprehensive attempt to synthesize this new literature. There fore, a reassessment of the evidence seems timely, both to determine if the earlier conclusions need to be modified and to benefit from the added precision that the new evidence can bring to the current assessment. Cooper’s meta-analysis revealed a consistent influence of grade level on the homework-achievement relationship. However, it produced ambiguous results regarding the possible differential impact of homework on different subject matters and on different measures of achievement. Specifically, research using different comparison groups (i.e., no homework, supervised study, correlations involving different reported amounts of homework) produced different orderings or magnitudes of homework’s relation to achievement for different subject matters and achievement measures. Also, Cooper (1989) found uniformly nonsignificant relationships between the sex of the student and the magnitude of the homework-achievement relationship. However, some recent theoretical perspectives (Covington, 1998; Deslandes & Cloutier, 2002; Harris, Nixon, & Rudduck, 1993; Jackson, 2003) suggest that girls generally hold more positive attitudes than boys toward homework and expend greater effort on it. Emerging evidence from some homework studies (Harris et al., 1993; Hong & Milgram, 1999; Younger & Warrington, 1996) lends empirical sup port to these perspectives. While these theories and results do not directly predict a stronger relationship between homework and achievement for girls than for boys (that is, they predict a main effect of higher levels of achievement among girls than among boys but do not indicate why differences in homework attitude and effort within the sexes would be more closely tied to achievement for one sex than the other, an interaction effect), they do suggest that this remains an important issue. Therefore, exploring these moderating relationships will be a focus of the present synthesis. Also, the Cooper (1989) synthesis paid only passing attention to the ability of the cumulated evidence to establish a causal relationship between homework and achievement. Clearly, the 50 studies that took naturalistic, cross-sectional measures of the amount of time students spent on homework and correlated these with measures of achievement cannot be used to establish causality. About half of the studies that introduced homework as an exogenous intervention and then compared achievement


Cooper etal.

for students who did homework with that of students who did not, or who had in school supervised study, employed random assignment of students to conditions. The other half sometimes did and sometimes did not employ a priori matching or post hoc statistical equating to enhance the similarity of homework and no-homework groups. When homework was compared with no-homework, Cooper reported that studies that used random assignment produced positive effect«j£homework similar to nonrandom assignment studies. However, when compared with in-school super vised study, random-assignment designs revealed no difference between the home work and in-school study students. We will test to determine whether these findings still hold for the new evidence. Also, since the earlier synthesis appeared, numerous studies have employed structural equation modeling to test the fit of complex models of the relationship between various factors and student achievement. Homework has been used as a factor in many of these models. The earlier synthesis did not include these designs, but this synthesis will. Methodologically, the past two decades have introduced new techniques and refinements in the practice of research synthesis. These include, among others, two important advances. First, there is now a greater understanding of meta-analytic error models involving the use of fixed and random-error assumptions that add precision to statements about the generality of findings. Second, new tests have been developed to estimate the impact of data censoring on research synthesis findings. These give us a better sense of the robustness of findings against plausible missing data assumptions. We will use these in the synthesis that follows. Potential Measures o f the Effects o f Homework As might be expected, educators have suggested a long list of both positive and negative consequences of homework (Cooper, 1989; see also Epstein, 1988; Warton, 2001). Table 1 presents a list of potential outcomes that might be the focus of home work research and the potential measures of interest for this synthesis. The positive effects of homework can be grouped into four categories: (a) imme diate achievement and learning; (b) long-term academic; (c) nonacademic; and, (d) parental and family benefits. The immediate effect of homework on learning is its most frequent rationale. Proponents of homework argue that it increases the time students spend on academic tasks (Carroll, 1963; Paschal, Weinstein, & Walberg, 1984; Walberg & Paschal, 1995). Thus the benefits of increased instructional time should accrue to students engaged in home study. The long-term academic benefits of homework are not necessarily enhancements to achievement in particular aca demic domains, but rather the establishment of general practices that facilitate learn ing. Homework is expected to (a) encourage students to learn during their leisure time; (b) improve students’ attitudes toward school; and (c) improve students’ study habits and skills (Alleman & Brophy, 1991; Como & Xu, 1998; Johnson & Pontius, 1989; Warton, 2001). Also, homework has been offered as a means for developing personal attributes in children that can promote positive behaviors that, in addition to being important for academic pursuits, generalize to other life domains. Because homework generally requires students to complete tasks with less supervision and under less severe time constraints than is the case in school, home study is said to promote greater self6


¥

TABLE 1 Potential effects of homework that might serve as outcomes for research Potential positive effects Immediate achievement and learning Better retention of factual knowledge Increased understanding " Better critical thinking, concept formation, information processing Curriculum enrichment Long-term academic benefits More learning during leisure time Improved attitude toward school Better study habits and skills Nonacademic benefits Greater self-direction Greater self-discipline Better time organization More inquisitiveness More independent problem-solving Parental and family benefits Greater parental appreciation of and involvement in schooling Parental demonstrations of interest in child’s academic progress Student awareness of connection between home and school Potential negative effects Satiation Loss of interest in academic material Physical and emotional fatigue Denial of access to leisure time and community activities Parental interference Pressure to complete homework and perform well Confusion of instructional techniques • Cheating . Copying from other students Help beyond tutoring Increased differences between high and low achievers Note. Adapted from Cooper (1989). Copyright 2005 by American Psychological Association. Reprinted with permission.

direction and self-discipline (Como, 1994; Zimmerman, Bonner, & Kovach, 1996), better time organization, more inquisitiveness, and more independent problem solv ing. These skills and attributes apply to the nonacademic spheres of life as well as the academic. Finally, homework may have positive effects on parents and families (HooverDempsey et al., 2001). Teachers can use homework to increase parents’ appreciation of and involvement in schooling (Balli, 1998; Balli, Wedman, & Demo, 1997; Epstein & Dauber, 1991; Van Voorhis, 2003). Parents can demonstrate an interest in the academic progress of their children (Epstein & Van Voorhis, 2001; Balli, Demo,


Cooper et al.

& Wedman, 1998). Students become aware of the connection between home and school. Some negative effects attributed to homework contradict the suggested positive effects. For instance, opponents of homework have argued that it can have a negative influence on attitudes toward school (Chen, & Stevenson, 1989), by satiating stu dents on academic pursuits. They claim any activity remairrw^jyarding for only so long, and children may become overexposed to academic tasks (Bryan, Nelson, & Mathru, 1995). Related to the satiation argument is the notion that homework leads to general physical and emotional fatigue. Homework can also deny children access to leisure time and community activities (Warton, 2001; Coutts, 2004). Proponents of leisure activities point out that homework is not the only circumstance under which after-school learning takes place. Many leisure activities teach important academic and life skills. Involving parents in the schooling process can have negative consequences (Epstein, 1988; Levin, Levy-Shiff, Appelbaum-Peled, Katz, Komar, & Meiran, 1997; Cooper, Lindsay, & Nye, 2000). Parents pressure students to complete homework assignments or to do them with unrealistic rigor. Also, parents may create confusion if they are unfamiliar with the material that is sent home for study or if their approach to teaching differs from that used in school. Parental involvement—indeed the involvement of anyone else in homework—can sometimes go beyond simple tutor ing or assistance. This raises the possibility that homework might promote cheating or excessive reliance on others for help with assignments. Finally, some opponents of homework have argued that home study has increased differences between high- and low-achieving students, especially when the achieve ment difference is associated with economic differences (Scott-Jones, 1984; Odum, 1994; McDermott, Goldman, & Varenne, 1984). They suggest that high achievers from well-to-do homes will have greater parental support for home study, including more appropriate parental assistance. Also, these students are more likely to have access to places conducive to their learning style in which to do assignments and better resources to help them complete a ss ig n m e n ts successfully. With few exceptions, the positive and negative consequences of homework can occur together. For instance, homework can improve study habits at the same time that it denies access to leisure-time activities. Some types of assignments can pro duce positive effects, whereas other assignments produce negative ones. In fact, in light of the host of ways that homework assignments can be construed and carried out, complex patterns of effects ought to be expected. The present synthesis will search for any and all of the above possible effects of homework. However, it is unrealistic to expect that any but a few of these will actually appear in the research literature. We expected the large preponderance of measures to involve achievement test scores, school grades, and unit grades. A few measures of students’ attitudes toward school and subject matters might also appear. Other measures of homework’s effect were expected to be few and far between. One reason for this is because many of the other potential effects are subtle. Therefore, their impact might take a long time to accrue, and few researchers have the resources to mount and sustain long-term longitudinal research. Another reason for the lack of subtle measures of homework’s effect is that the homework variable is often one of many influences on achievement being examined in a study. It is achievement 8


r

Does Homework Improve Academic Achievement ?

.

as the outcome that is the primary focus of investigation with many predictors, rather than homework as the focus with many outcomes measured. Factors That Affect the Utility o f Homework Assignments

■ H

In addition to looking at homework’s effectiveness on different outcomes, re searchers have examined how other variations in assignments mightinfluence their utility. Homework assignments are influenced by more factors than any other instruc tional strategy. Student differences may play a major role because homework allows students considerable discretion about whether, when, and how to complete assign ments. Teachers may structure and monitor homework in a multitude of ways. The home environment may influence the process by creating a positive or nega tive atmosphere for study. And finally, the broader community provides other leisure activities that compete for the student’s time. Table 2 presents a model of the homework process presented by Cooper (1989). The model organizes into a single scheme many of the factors that educators have suggested might influence the success of a homework assignment. The model pro poses that student ability, motivation, and grade level, as well as other individual differences (e.g., sex, economic background), and the subject matter of the homework assignments are exogenous factors, or moderator conditions, that might influence homework’s effect. The model’s endogenous factors, or mediators, divide the home work process into characteristics of the assignment and a home-community phase sandwiched by two classroom phases, each containing additional potential influences on homework’s effects. Finally, Table 2 includes the potential consequences of homework as the outcomes in the process. In this synthesis, the search for factors that might influence the impact of homework will focus only on the exogenous factors and the outcome variables, with the exception of the endogenous factor of amount of homework. Studies of the latter type are included because (a) they would include students who did no homework at all; and (b) achievement variations related to time spent on homework can reasonably be taken to bear on homework’s effectiveness. Our restriction is based on the fact that most studies that look at other variations in endogenous or mediating factors rarely do so in the context of an investigation that also attempts to assess the more general effects of homework. Investigations of mediating factors typically pit one homework strategy against another and do not contain a condition in which students receive no-homework or an alternative treatment. Thus, in an effort to keep our task manageable, we focused here on studies that investigate primarily the general effects of homework, and we excluded studies that exclusively examine variations in homework assignments. (For a review of one such endogenous variable, parent involvement, see Patall, Cooper, & Robinson, 2005).

■

Optimum Amounts o f Homework

■ ■ ■ H H ■ ■

Related to the issue of time spent on homework is the important question conceming the optimum amount of homework. Cooper (1989) found nine studies that allowed for a charting of academic performance as a function of homework time. The line-of-progress was flat in young children. For junior high school students, achievement continued to improve with more homework until assignments lasted between 1 and 2 hours a night. More homework than that was no longer associated with higher achievement. For high school students, the line-of-progress continued to

U

9

.

■

1

l I,1 L t I | B. I I v

I


]


| } ‘ !

go up through die highest point on the measured scales, more than 2 hours. In the present synthesis, we included studies examining time on homework because of their relevance to homework’s general effectiveness; therefore, we also looked for studies that might replicate or extend this finding.

, , , j

Decisions concerning how to search the literature determine the kinds of materials that will form the basis for a synthesis’ conclusions. Identifying the literature is com plicated by the fact that the search has two targets (Cooper, 1998). First, synthesists want to locate all previous literature on the problem. This is especially critical with regard to the retrieval of studies for inclusion in a meta-analysis. Synthesists can exert some control over whether this goal is achieved through their choice of information sources. Second, synthesists hope that the included studies will allow generalizations in the broader topic area. The generalizability of our synthesis was constrained by the students, schools, and communities represented in the literature. We employed several strategies to ensure that our homework synthesis included the most exhaustive set of relevant documents. These strategies included (a) com puterized searches of reference databases; (b) direct contact with active researchers and others who might know of unpublished or “fugitive” homework research; and (c) scrutiny of reference lists of relevant materials. In addition, analyses of the retrieved studies were undertaken to test for indications that the studies in hand might con stitute a biased representation of the population of studies, and if so, to determine the nature of the bias. Avoiding overgeneralization requires recognizing that the students, schools, and communities represented in the retrieved literature may not represent all target pop ulations. For instance, it may be that little or no research has been conducted that examines the effects of homework on first- or second-grade students. A synthesis that qualifies conclusions with information about the kinds of people missing or overrepresented in studies runs less risk of overgeneralization. Such an examination of potential population restrictions will be included in the present work.

Bias and Generalization in Research Synthesis^

Methods for Research Synthesis Literature Search Procedures No matter how thorough the procedures may be, no search of the literature is likely to succeed in retrieving all studies relating homework to achievement. Therefore, systematic data censoring is a concern. That is, the possibility exists that more easily retrievable studies have different results from studies that could not be retrieved. To address this possibility, we collected studies from a wide variety of sources and included search strategies meant to uncover both published and unpublished research. First, we searched the ERIC, PsycINFO, Sociological Abstracts, and Dissertation Abstracts electronic databases for documents cataloged between Tannery 1987, and December 31,2003. The single keyword “homework” was used in these searches! Also, the Science Citation Index Expanded and the Social Sciences Citation Index databases were searched from 1987 to 2004 to identify studies or reviews that had cited Cooper (1989). These searches identified approximately 4,400 nonduplicate potentially relevant studies. Next, we employed three direct-contact strategies to ensure that we tapped sources that might have access to homework-related research that would not be included

11


Cooper et al.

in the reference and citation databases. First, we contacted the dean, associate dean, or chair of 77 colleges, schools, or departments of education at research-intensive institutions of higher education and requested that they ask their faculty to share with us any research they had conducted that related to the practice of assigning homework. Second, we sent similar letters to 21 researchers who, as revealed by our reference database search, had been the first author on two or morenftieles on homework and academic achievement between 1987 and the end of 2003. Finally, we sent similar letters to the directors of research or evaluation in more than a hundred school districts, obtained from the membership list of the National Association of Test Directors. Two researchers in our team then examined each title, abstract, or document. If either of the two felt that the document might contain data relevant to the relation ship between homework and an achievement-related outcome, we obtained the full document (in the case of judgments made on the titles or abstracts). Finally, the reference sections of relevant documents were examined to determine if any cited works had titles that also might be relevant to the topic. Criteria fo r Including Studies For a study to be included in the research synthesis, several criteria had to be met. Most obviously, the study had to have estimated in some way the relationship between a measure of homework activity on the part of a student and a measure of achieve ment or an achievement-related outcome. Two sampling restrictions were placed on included studies. Each study had to assess students in kindergarten through 12th grade. We excluded studies conducted on preschool-aged children or on postsecondary students. It was felt that the purpose and causal structure underlying the homework-achievement relationship would be very different for these populations. For similar reasons, we included only studies conducted in the United States. Finally, the report had to contain enough information to permit the calculation of an estimate of the homework-achievement relationship. Information Retrieved From Evaluations Numerous characteristics of each study were included in the database. These characteristics encompassed six broad distinctions among studies: (a) the research report; (b) the research design; (c) the homework variable; (d) the sample of students; (e) the measure of achievement, and (f) the estimate of the relationship between homework and achievement. Report Characteristics Each database entry began with the name of the author of the study. Then the year of the study was recorded, followed by the type of research report. Each research report was categorized as a journal article, book chapter, book, dissertation, Master’s thesis, private report, government report (state or federal), school or district report, or other type of report. Research Design and Other Study Characteristics The studies in this research synthesis were categorized into three basic design types, some with subtypes.

12


w


First, studies could employ exogenous manipulations of homework. This meant that the presence or absence of homework assignments was manipulated expressly for purposes of the study. Within the exogenous manipulation studies, the experi menters could introduce the manipulation at the student or classrooms level, either by randomly assigning students to homework and no-homework conditions or by some nonrandom process. If a nonrandom process was usedrtheêxperimenter then might or might not employ a priori matching or post hoc statistical procedures to equate the homework and no-homework groups. If procedures were used to equate groups, the variables used to enhance the equivalence of the groups could differ from study to study. Each of these variations in design was recorded for the set of studies that used exogenous homework manipulations. In addition to these design characteristics of exogenous homework manipulation studies and their report information, we recorded (a) the number of students and classrooms included in the homework and no-homework conditions at the begin ning and end of the experiment; (b) the grade level of the students; (c) the subject matter of the homework (reading, other language arts, math, science, social studies, foreign language, other, or multiple subjects); (d) the number of assignments per week and their duration; (e) the measure of achievement (standardized achievement test, teacher-developed unit test, textbook chapter unit test, class grades, overall grade point average, composite achievement score); and (f) the magnitude of the relation ship between homework and achievement. The second type of design included studies that took naturalistic, cross-sectional measures of the amount of time the students spent on homework without any inter vention on the part of the researchers and related these to an achievement-related measure. This second type of design also included an attempt to statistically equate students on other variables that might be confounded with homework and therefore might account for the homework-achievement relationship. For these studies, we also coded the source of the data, that is, whether the data were collected by the researchers or by an independent third party. If data were from an independent source, we coded the source. We coded the analytic strategy used to equate students. Most frequently, this involved conducting multiple regression analysis or the application of a structural equation modeling package. Also, we coded each of the same variables coded for studies that used exogenous manipulations of homework, except for (a) the sample sizes in the homework and no-homework groups (only total sample size in the analysis was recorded); and (b) the number and duration of assignments, which was irrelevant to this design. Instead of the assignment characteristics, we coded the amount of time the student spent doing homework, as measured by student or parent report. The third type of design involved the calculation of a simple bivariate correlation between the time the student spent on homework and the measure of achievement. In these studies, no attempt was made to equate students on other variables that might be confounded with time on homework. For these studies, we also recorded the same variables coded for studies using statistical controls of other variables except, of course, the number and nature of controlled variables. We also coded several addi tional variables related to the sample of students. These included the students’ (a) sex; (b) socioeconomic status (low, low-middle, middle, middle-upper, upper, “mixed,” no SES [socioeconomic status] information given); and (c) whether any of the fol lowing labels were applied to the sample of students (gifted, average, “at risk,” 13


Cooper et al.

underachieving/below grade level, possessing a learning disability, overachieving/ above grade level). Effect Size Estimation For studies with exogenous manipulations of homework, we used the standard ized mean difference to estimate the effect of homework «ajneasures of student achievement. The d-index (Cohen, 1988) is a scale-free measure of the separation between two group means. Calculating the d-index for any comparison involves dividing the difference between the two group means by either their average standard deviation or by the standard deviation of the control group. This calculation results in a measure of the difference between the two group means expressed in terms of their common standard deviation or that of the untreated population. Thus a d-index of .25 indicates that one-quarter standard deviation separates the two means. In the synthesis, we subtracted the no-homework condition mean from the homework condition mean and divided the difference by their average standard deviation. Thus positive effect sizes indicate that the students doing homework had better achievement outcomes. We calculated effect sizes based on the means and standard deviations of students’ achievement indicators, if available. If means and standard deviations were not available, we retrieved the information needed from inferential statistics to calculate ^-indexes (see Rosenthal, 1994). For studies that involved naturalistic, cross-sectional measures of the amount of time spent on homework and related these to achievement but also included an attempt to statistically equate students on other characteristics, our preferred mea sure of relationship strength was the standardized beta-weight, (3. These were derived either from the output of multiple regressions or as path coefficients in structural equation models. The standardized beta-weights indicate what change in the achieve ment measure expressed as a portion of a standard deviation was associated with a one-standard-deviation change in the homework variable. For example, if the standard deviation of the time-spent-on-homework variable equaled 1 hour and the standard deviation of the achievement measure equaled 50 points, then a betaweight of .50 would mean that, on average, students in the sample who were separated by 1 hour of time-spent-on-homework also showed a 25-point separation on the achievement measure. In a few instances, beta-weights could not be obtained from study reports, so the most similar measures of effect (e.g., unstandardized regression weights, b) were retrieved. There were no instances in which we calculated betaweights from other statistics. For studies that involved naturalistic, cross-sectional measures but included no attempt to statistically equate students on third variables, we used simple bivariate correlations as measures of relationship. In some instances these were calculated from other inferential statistics (see Rosenthal, 1994). Using three different measures of association implies that the relationship of homework to achievement cannot be compared across the three different types of design. This is not strictly true. Standardized mean differences and correlation co efficients can be transformed one to the other (see Cohen, 1988). A beta-weight equals a correlation coefficient when no other variables are controlled. However, we chose to present the results using each design’s most natural metric so that the important distinction in their interpretation would not be lost. 14


Coder Reliability Two coders extracted information from all reports selected for inclusion. Dis crepancies were first noted and discussed by the coders, and if agreement was not reached the first author was consulted. Because all studies were independently coded twice and all disagreements resolved by a third independent coder, we did not cal culate a reliability for this process (which would have entailed 4 Qiining three more coders and having them code at least a subset of studies). Methods o f Data Integration Before conducting any statistical integration of the effect sizes, we first counted the number of positive and negative effects. For studies with effect size information, we calculated the median and range of estimated relationships. Also, we examined the distribution of sample sizes and effect sizes to determine if any studies con tained statistical outliers. Grubbs’s (1950) test, also called “the maximum normed residual test,” was applied (see also Barnett & Lewis, 1994). This test identifies outliers in univariate distributions and does so one observation at a time. If outliers were identified, (using p < .05, two-tailed, as the significance level) these values would be set at the value of their next nearest neighbor. Both published and unpublished studies were included in the synthesis. However, there is still the possibility that we did not obtain all studies that have investigated the relationship between homework and achievement. Therefore, we used Duval and Tweedie’s (2000 a, 2000 b) trim-and-fill procedure to test whether the distribution of effect sizes used in the analyses were consistent with variation in effect sizes that would be predicted if the estimates were normally distributed. If the distribution of observed effect sizes was skewed, indicating a possible bias created either by the study retrieval procedures or by data censoring on the part of authors, the trim-andfill method provides a way to estimate the values from missing studies that need to be present to approximate a normal distribution. Then, it imputes these missing values, permitting an examination of an estimate of the impact of data censoring on the observed distribution of effect sizes. Calculating Average Effect Sizes 1 We used both weighted and unweighted procedures to calculate average effect sizes across all comparisons. In the unweighted procedure, each effect size was given equal weight in calculating the average value. In the weighted procedure, each independent effect size was first multiplied by the inverse of its variance. The sum of these products was then divided by the sum of the inverses. Generally speaking, weighted effect sizes are preferred because they give the most precise estimates of the underlying population values (see Shadish & Haddock, 1994). The unweighted effect sizes are also reported because in instances in which these are very different from the weighted estimates, this can give an indication that the magnitude of the effect size and sample size are correlated, sometimes suggesting that publication bias might be a concern. Also, 95% confidence intervals were calculated for weighted average effects. If the confidence interval did not contain zero, then the null hypoth esis of no homework effect can be rejected. Identifying Independent Hypothesis Tests One problem that arises in calculating effect sizes involves deciding what con stitutes an independent estimate of effect. Here, we used a shifting unit of analysis 15


f Cooper et al.

approach (Cooper, 1998). In this procedure, each effect size associated with a study is first coded as if it were an independent estimate of the relationship. For example, if a single sample of students permitted comparisons of homework’s effect on both math and reading scores, two separate effect sizes were calculated. However, for estimating the overall effect of homework, these two effect sizes were averaged prior to entry into the analysis, so that the sample only contribulgdone effect size. To calculate the overall weighted mean and confidence interval, this one effect size would be weighted by the inverse of its variance (based primarily on sample size, which should be about equal for the two component effect sizes). However, in an analy sis that examined the effect of homework on math and reading scores separately, this sample would contribute one effect size to each estimate of a category mean effect size. The shifting unit of analysis approach retains as many data as possible from each study while holding to a minimum any violations of the assumption that data points are independent. Also, because effect sizes are weighted by sample size in the cal culation of averages, a study with many independent samples containing just a few students will not have a larger impact on average effect size values than a study with only a single, or only a few, large independent samples. Tests fo r Moderators o f Effects Possible moderators of homework-achievement relationships were tested by using homogeneity analyses (Cooper & Hedges, 1994; Hedges & Olkin, 1985). Homo geneity analyses compare the amount of variance in an observed set of effect sizes with the amount of variance that would be expected by sampling error alone. The analyses can be carried out to determine whether (a) the variance in a group of indi vidual effect sizes varies more than predicted by sampling error, or (b) a group of average effect sizes varies more than predicted by sampling error. In the latter case, the strategy is analogous to testing for group mean differences in an analysis of variance or linear effects in a multiple regression. Fixed and Random Error When an effect size is said to be “fixed,” the assumption is that sampling error is due solely to differences among participants in the study. However, it is also pos sible to view studies as containing other random influences, including differences in teachers, facilities, community economics, and so on. This view assumes that home work data from classrooms, schools, or even school districts in our meta-analysis also constitute a random sample drawn from a (vaguely defined) population of homework conditions. If it is believed that random variation in interventions is a significant component of error, a random-error model should be used that takes into account this study-level variance in effect sizes (see Hedges & Vevea, 1998, for a discussion of fixed and random effects). Rather than opt for a single model of error, we chose to apply both models to our data. We conducted all our analyses twice, employing fixed-error assumptions once and random-error assumptions once. By employing this sensitivity analysis (Greenhouse & Iyengar, 1994), we could examine the effects of different assump tions on the outcomes of the synthesis. Differences in results based on which set of assumptions was used could then be part of our interpretation of results. For example, 16



if an analysis reveals that a moderator variable is significant under fixed-error assump tions but not under random-error assumptions, this result suggests a limit on the generalizability of inferences about the moderator variable. All statistical analyses were conducted using the Comprehensive Meta-Analysis statistical software package (Borenstein, Hedges, Higgins, & Rothstein, 2005).

Results Studies With Exogenous Introductions o f Homework The literature search located six studies that employed a procedure in which the homework and no-homework conditions were imposed on students explicitly for the purpose of studying homework’s effects. None of these studies was published. Some of the important characteristics and outcomes of each study are presented in Table 3. Apparently, only one study used random assignment of students to conditions. McGrath (1992) looked at the effect of homework on the achievement of 94 high school seniors in three English classes studying the play Macbeth. At one point in the research report, the author states that half of the students “elected to receive no homework” and half “elected to receive homework” (p. 27). However, at another point, the report states that each student was assigned to a condition “by the alpha betic listing of his/her last name” (p. 29). Thus it might be (optimistically) assumed that the students in each of the three classes were haphazardly assigned to homework and no-homework conditions. In the analyses, the student was used as the unit. The experiment lasted 3 weeks and involved 12 homework assignments. Students doing homework did significantly better on a posttest achievement measure, d = .39. A study by Foyle (1990) assigned four whole 5th-grade classrooms (not indi vidual students) to conditions at random, one to a practice homework condition, one to a preparation homework condition, and two to a no-homework control condition. Clearly, assigning only one classroom to each condition, even when done at random, cannot remove confounded classroom differences from the effect of homework. For example, all four classrooms used a cooperative learning approach to teaching social studies, but one classroom (assigned to the practice homework condition) used a different cooperative learning approach from the other three classes. Also, the student, rather than the classroom, was used as the unit for statistical analysis, cre ating the concern that within-class dependencies among students were ignored. Analysis revealed that students differed significantly on a social studies pretest and on a standard measure of intelligence, but it was not reported whether there were preexisting classroom differences on these measures. Students doing homework outperformed no-homework students on unadjusted posttest scores, d = .90, and on posttest scores adjusted for pretest and intelligence differences, d = .99. Foyle (1984) conducted a similar study on six high school classes in American history. Here, the experimenter reported that “the assignment of treatment and control groups was under the experimenter’s control” (p. 90) and two intact classrooms were each assigned randomly to practice homework, preparation homework, and no-homework conditions. However, the student was again used as the unit of analy sis. Analyses of covariance that controlled for pretest scores, aptitude differences, and the students’ sex revealed that students doing homework had higher posttest achievement scores than students who did not. The covariance analysis and post hoc 17



tests revealed a significant positive effect of homework, but an effect size could not be calculated from the adjusted data (because the reported F-test contained two degrees of freedom in the numerator and means and standard deviations were not provided). The approximate, unadjusted homework effect was d = .46. Finstad (1987) studied the effect of homework on mathematics achievement for 39 second-grade students in two intact classrooms. One unit, on-place values to 100, was used, but neither the frequency nor the duration of assignments was reported. One classroom was assigned to do homework and the other not. It was not reported how the classroom assignments were carried out, but it was reported that there were no pretest differences between the classes. Data were analyzed on the student level without adjustment. The students in the classroom doing homework performed sig nificantly better on a posttest measure, d = .97. Meloy (1987) studied the effects of homework on the English skills (sentence components, writing) of third and fourth graders. Eight intact classrooms took part in the study and classes were matched on a shortened version of the Iowa Test of Basic Skills (ITBS) language subtest before entire classes were randomly assigned to homework and no-homework conditions. However, examination of pretest dif ferences on the ITBS language subscale revealed that the students assigned to do homework scored significantly higher than students in no-homework classes. Thus a pretest-posttest design was used to control for the initial group differences, but pretests were used as a within-students factor rather than as a covariate (meaning a significant homework effect would appear as an interaction with time of testing). Also, students who scored above a threshold score on the pretest were excluded from the posttest analysis. Thus only 106 of an original sample consisting of 186 students were used in the analyses, and excluded students were not distributed equally across homework and no-homework conditions. Grade levels were analyzed separately, and classrooms were a factor in the analyses. The class-within-condition effect was not significant, so, again, the student was used as the unit of analysis. Homework was assigned daily for 40 instructional days. This study also monitored the home work completion rates in classrooms and set up reinforcement plans, different for each class, to improve completion rates. The effecfs of homework were gauged by using a researcher-modified version of the ITBS language subtest and a unit mas tery test from the textbook. The complex reporting of statistical analyses made it impossible to retrieve simple effect estimates frorii the data. However, the author reported that the condition-by-time interactions indicated that homework had a sig nificant negative effect on ITBS scores for third graders and a significant positive effect on fourth graders’ unit test scores. Finally, Townsend (1995) examined the effects of homework on the acquisition of vocabulary knowledge and understanding among 40 third-grade students in two classes, both taught by the experimenter. Treatment was given to classes as a whole and it was not stated how each class was assigned to the homework or no-homework condition. The student was used as the unit of analysis. A teacher-prepared posttest measure of vocabulary knowledge suggested that the homework group performed better, d = .71. In sum, the six studies that employed exogenous manipulations all revealed a positive effect of homework on unit tests. One study (Meloy, 1987) revealed a negative effect on a standardized test modified by the experimenter. Four of the six studies employed random assignment, but in three cases assignment to conditions was 19


Cooper et al.

carried out at the classroom level, using a small number of classrooms, and analyses were conducted using the student as the unit of analysis. In the only instance in which random assignment appears to have occurred within classes (McGrath, 1992), students also were used as the unit of analysis. Also, random assignment appears to have failed to produce equivalent groups in one study (Meloy, 1987). While the introduction of homework as an exogenous intervention is a positive feature of these studies, other methodological considerations make it difficult to draw strong causal inferences from their results. Still the results are encouraging because of the consistency of findings. The measurable effects of homework on unit tests varied between d = .39 and d = .97. Also, the three studies that successfully used random assignment, fixed weighted d = .53 (95% Cl = .291.19), random weighted d = .54 (9 5 % Cl = ,26/.82), produced effect sizes that were smaller than those of two studies that used other techniques to produce equivalent groups and for which effect sizes could be calculated, fixed weighted d = .83 (95% Cl = .37/1.30), ran dom weighted d = .83 (95% Cl = .37/1.30); but the difference in mean ^-indexes between these two sets of studies was not significant, fixed Q( 1) = 1.26, ns, ran dom Q(l) = 1.12, ns. Collapsing across the two study designs and using fixed-error assumptions, the weighted mean d-index across the five studies from which effect sizes could be obtained was d = .60 and was significantly different from zero (95% Cl = .38/.82). Using a random-error model, the weighted average rf-index was also .60 (95% Cl = ,38/.82). To take into account the within-class dependencies that were not addressed in the reported data analyses, we recalculated the mean effect sizes and confidence inter vals by using an assumed intraclass correlation of .35 to estimate effective sample sizes. In this analysis, the weighted mean d-index was .63, using both fixed and random-error assumptions, and both were statistically different from zero (95% Cl = .03/1.23, for both). The mean d-index would not have been significant if an intraclass correlation of .4 was assumed. Additionally, the tests of the distribution of ^-indexes revealed that we could not reject the hypothesis that the effects were estimating the same underlying population value when students, were used as the unit of analysis, QjueA5) = 4.09, ns, Qrandom(5) = 4.00, ns, or when effective sample sizes were used as the unit, < 2/^5) = .54, ns, Q w „m(5) = .54, ns. And finally, the trim-and-fill analyses were conducted looking for asymmetry using both fixed and random-error models to impute the mean d-index (see Borenstein et al., 2005). Neither of the analyses produced results different from those described above. There was evidence that two effect sizes might have been missing. Imputing them would lower the mean rf-index to d = .48 (95% Cl = .22/.74) using both fixed and random-error assumptions. The small number of studies and their variety of methods and contexts preclude their use in any formal analyses investigating possible influences on the magnitude of the homework effect, beyond comparing studies that used random assignment versus other means to create equivalent groups. The studies varied not only in research design but also in subject matter, grade level, duration, amount of homework, and the degree of alignment of the outcome measure with the content of assignments. Replications of any important feature that might influence the homework effect are generally confounded with other important features, and no visible pattern connecting effect sizes to study features is evident. 20


Studies Using Cross-Sectional Data and Control o f Third Variables Studies Using the National Education Longitudinal Study (1988,1990, or 1992) The literature search located nine reports that contained multivariate analyses of data collected as part of the National Education Longitudinal Study of 1988 (NELS) or in one of the NELS follow-ups on the same students in 1990, 1992, 1994, or 2000. These studies are described in Table 4. The NELS \9a»>conducted by the National Center for Educational Statistics and involved a nationally representative two-stage stratified probability sample. The final student sample in the first wave included 24,599 eighth-grade students. Each student completed achievement tests in mathematics, reading, science, and social studies in 1988,1990, and 1992, as well as a 45-minute questionnaire that included questions about school, school grades, personal background, and school context. Various waves of the NELS also included surveys of teachers, school administrators, and parents. Student transcripts were collected at the end of their high school careers. Questions on homework were completed by both students and teachers, and they were asked about the total min utes of homework completed or assigned in different subject areas. Several of the studies using the NELS data sampled students from the NELS itself for the purpose of examining questions regarding restricted populations. For example, Peng and Wright (1994) were interested in studying differences in relation ships between predictors of achievement across ethnic groups, with a focus on Asian Americans. Davis and Jordan (1996) focused on African American males, while Roberts (2000) restricted the subsample to students attending urban schools only. Examined as a group, the studies using NELS data use a wide variety of outcome measure configurations and different sets of predictor variables, in addition to home work. Still, every regression coefficient associated with homework was positive, and all but one were statistically different from zero. The exception occurred in the study of African American males on a composite measure of class grades (Davis & Jordan, 1996). The study revealing the smallest beta-weight was a dissertation by Hill (2003). This report presents an unclear description of how the subsample drawn from the NELS was defined. The text reports that students were omitted from the sample if they “attended public schools, live in suburban areas, are neither Black nor Hispanic; and whose teachers are male, not certified in [the subject of the outcome variable], have neither an undergraduate degree in education or in [the subject of the outcome variable], and have neither a graduate degree in education or [the subject of the out come variable]” (pp. 45, 86 , 120). However, the tables in the report suggest that White students were included in the samples. The regression models suggest that students with teachers who had degrees in subjects other than the outcome variable also were included. Thus it is difficult to determine whether sampling restrictions might be the cause of the small regression coefficients associated with homework. The dissertation by Lam (1996) deserves separate mention. In this study using data from 12th graders, the amount of homework students reported doing was entered into the regression equation as four dummy variables. This permitted an examination of possible curvilinear effects of homework. As Table 4 reveals, students who reported doing homework always had higher achievement scores than students who did not do homework (coded as the dummy variable). However, the strongest relationship between homework and achievement was found among students who reported doing 21


Cooper et al.

7 to 12 hours of homework per week, followed by students who reported doing 13-20 hours per week. Students who reported doing more than 20 hours of home work per week revealed a relationship with achievement test scores nearly equal to those reporting between 1-6 hours of homework per week. While this result is sug gestive of a curvilinear relationship between homework and achievement, we must bear in mind that Lam restricted the sample of students to Asi^n Americans and Caucasian Americans. In sum, if we omit (a) the Hill (2003) study (which produced beta-weights of .01 and .02 ), as well as (b) those studies that reported unstandardized regression weights, or (c) those for which coefficients could not be determined, then the reported betaweights for the relation between homework and standardized achievement test scores range from .05 to .28. For composite achievement scores the range is from .05 to .21; for math, it is .09 t o . 16; for reading,. 12 to .28; for science, .09 to .23; and for social studies,. 11 t o . 18. Thus the ranges of estimated regression coefficients appear quite similar across the subject areas. However, we would caution against drawing any conclusions regarding the mediating role of subject matter on the homeworkachievement relationship from these data, because the number and type of predictors in each model are confounded with subject matter. It should also be kept in mind that these estimates refer to high school students only. Studies Using Data Other Than the National Education Longitudinal Study and Performing Multivariate Analyses Table 5 provides information on 12 additional studies that performed multi variate analysis on cross-sectional data in order to examine the relationship between homework and achievement, with other variables controlled. Two of the studies used the High School and Beyond database (Cool & Keith, 1991; Fehrmann, Keith, & Reimers, 1987). The High School and Beyond database drew its 1980 base-year sample of sophomores and seniors from high schools throughout the United States. Probability sampling was used with overrepresentation of special populations. Follow-up surveys were conducted in 1982 and 1^84. Brookhart (1997) used the Longitudinal Study of American Youth database, containing a national probability sample of approximately 6,000 seventh and tenth graders stratified by geographic area and degree of urban development. The rest of the studies used data collected by the researchers for the specific purpose of studying variables related to achievement. Two studies conducted by Smith (1990, 1992), using overlapping data sets of seventh, ninth, and eleventh graders, found some negative relationships between homework and achievement. One of these findings (in Smith, 1992) revealed a small but statistically significant negative relationship between the amount of time spent on homework and language achievement, {3=—.06. However, this study also revealed a significant positive interaction between year in school and time spent on home work. The interaction was not interpreted. This was the only significant negative result obtained in any of the cross-sectional, multivariate studies. The remaining studies that used secondary school students all revealed posi tive and generally significant relationships. The three studies that used elementary school students (Cooper et al., 1998; Olson, 1988; Wynn, 1996) all revealed posi tive relationships between the homework measure and achievement (in Cooper et al., P = .22 for teacher-reported overall grades; in Olsen, P = .10 for math and P = .11 28



for reading; and in Wynn, P = .04 for grade point average). Thus, in addition to using varying predictor variables in the regression models, these studies also included a variety of outcome measures, including not only standardized tests but also teacher-assigned grades. In one instance, (Hendrix, Sederberg, & Miller, 1990) the outcome measure was not achievement but rather an indicator of school commitment/ alienation constructed by the researcher that measured the importance of successful performance on school tasks, effort, and relevance of school work for student’s lives. Thus we would again caution against drawing conclusions about mediating and moderating variables from these studies. It seems safest simply to note that the pos itive relationship between homework and achievement across the set of studies was generally robust across sample types, models, and outcome measures. Structural Equation Modeling Studies Using Data From the National Education Longitudinal Study (1988,1990, or 1992) Table 6 provides information on four studies that tested structural equation models using data from the National Education Longitudinal Study. These analyses all revealed a positive relationship between the amount of time spent on homework and achievement. Not surprisingly, they are also somewhat larger than the relation ships reported in studies that used multiple regression approaches to data analysis. Structural Equation Modeling Studies Using Data From the High School and Beyond (1980,1982,1984) Longitudinal Studies Table 7 provides information on four studies that tested structural equation models using data from the High School and Beyond database. All coefficients but one are positive and statistically significant. Keith and Benson (1992) found anonsignificant negative coefficient for a subsample of Native Americans, P = -.09. The authors caution against strong interpretation of this finding because (a) the sample size was small (n = 147), and (b) Native American students who attended Bureau of Indian Affairs schools were not sampled. Still, it is generally the case that co efficients for the homework-achievement relationship estimated using High School and Beyond data are smaller than those estimate