PART I: TITLE I IN PERSPECTIVE A BRIEF HISTORY OF TITLE I Recognizing the need to help disadvantaged students achieve their full potential, Congress, in 1965, passed the Elementary and Secondary Education Act (ESEA). The ESEA was the first major federal school aid initiative and immediately altered the landscape of American education. A key component of President Johnson's War on Poverty, the ESEA soon became the cornerstone of a federal education enterprise that has, over more than three decades, broadened to encompass programs ranging from special education to educational technology (Congressional Budget Office, 1993; Vanecko & Ames, 1979).
By far the largest program created under this ambitious new legislative initiative was a program originally called Title I: Better Schooling for Educationally Deprived Children.1 Its intent was to:
(PL 89-10, Section 201).provide financial assistance...to local educational agencies serving areas with concentrations of children from low-income families; and to expand and improve their educational programs by various means ... which contribute particularly to meeting the special educational needs of educationally deprived children
Since its inception, Title I's mission has been refined and expanded to focus even more explicitly on ameliorating the impact of poverty and, most recently, to lead states and schools toward more systemic standards-based reform. Today, at an appropriation level of more than $8 billion, Title I dominates a $16 billion federal elementary and secondary education budget. Annually, the program reaches more than 90 percent of school districts across the United States (U.S. Department of Education, 1999). And yet, at its current level of funding, Title I falls short of meeting the needs of many disadvantaged children who could benefit from assistance.
Poverty and School Success
Despite a prolonged period of economic growth in the United States, about one-quarter of children under six are poor, a poverty rate more than twice that for adults (U.S. Department of Health and Human Services, 1999). Research has shown that the conditions of poverty can severely reduce access to the educational supports and experiences that children need to be successful in school (Natriello, McDill, & Pallas, 1987), and that poverty at both the individual and the school level is strongly associated with decreased school performance. Poor children achieve at a lower level, are twice as likely to be retained in grade, and are one-third less likely to attend college than their more advantaged peers (Children's Defense Fund, 1998). The picture for minority children is even worse. Data from the National Assessment of Educational Progress (NAEP) indicate that, at the fourth-grade level, 69 percent of African-American and 64 percent of Hispanic children are reading below the basic level (U.S. Department of Education, 1998a).
There is an equally ominous gap in achievement between students who attend high- and low-poverty schools2 the equivalent of three to four grade levels among fourth-graders (Bryk & Raudenbush, 1992; Hart & Risley, 1995; National Assessment of Educational Progress, 1995, 1997; U.S. Department of Education, 1998a; White, 1982). Indeed, data support the premise that school-level poverty can be an even more important factor in predicting school achievement than a student's individual economic conditions (Puma, Jones, Rock, & Fernandez, 1993). Thus, the 1986 National Assessment of Chapter I (Kennedy, Birman, & Demaline, 1986) concluded that the "achievement scores of all students not just poor students decline as the proportion of poor students in a school increases."
Clearly then, there is a strong educational and public policy rationale for focusing resources on poor children as well as children in high-poverty schools, and this has been the overriding premise of Title I for nearly 35 years. On one hand, the program is designed to funnel cash grants to school districts providing educational services to the poor, allocating the most money to financially-strapped districts burdened by the educational needs of large numbers of disadvantaged children. These grants seek to foster "financial equity" among districts with varying levels of local resources, targeting districts and, under the current version of the law, schools with high concentrations of poor students, regardless of their level of educational achievement. On the other hand, Title I pursues an "educational equity" goal by targeting actual educational services to low-achieving students in Title I schools, regardless of their family's income level. Not surprisingly, these students are disproportionately poor, and the targeting of more funds to higher-poverty schools also means more poor children receive services.
Another important characteristic of Title I is that it provides a funding source that allows a high degree of adaptability to local conditions. Beyond some broad guidelines, local school districts and schools have enormous flexibility to decide where and how to focus the resources they receive. That is, they decide within limits which schools and grades receive funds, how much they receive, the types of services that are provided to students, the content areas targeted for supplemental assistance, and the types of staff used. Consequently, the ultimate success of Title I depends upon the ability of local school administrators to determine how best to use limited program funds to serve the needs of children who are struggling to achieve academic success.
The First 30 Years The Road to Excellence
Title I's goals and administrative focus have evolved substantially over the course of its 35-year history. Although the 1994 reauthorization provided a much-needed focus on standards and accountability, that was not the case in the early years of the program. During the program's first 15 years, it was reauthorized every three years with increasing attention to tightening the rules for resource accountability. As a result, federal rules and regulations proliferated, and sanctions were developed for noncompliance. Strict financial regulation to ensure that funds were spent for services to Title I-eligible students substantially dominated and defined the shape of local Title I programs. Procedural requirements were also expanded to focus funding on low-income schools and the lowest-achieving students, to promote resource parity between Title I and non-Title I schools, to increase the role of parents in program design, and to ensure that Title I funds were used to supplement (not replace) local funds.
One consequence of this emphasis on financial compliance was the widespread adoption of "pull-out" programs by Title I schools, an approach that separated eligible students from their classmates and provided remedial instruction to address their educational needs. But pull-outs came under increasing fire for their lack of coordination with regular classroom instruction and, in 1978, the "schoolwide" option was introduced. The schoolwide approach allowed high-poverty schools (those with 75 percent or more low-income students) to move from assistance targeted to individual students to the use of Title I funds to bring about overall school improvement. Still, requirements for local matching funds precluded almost all eligible schools from implementing schoolwide programs.
The 1980's brought the Reagan Administration's campaign to reduce government regulation and devolve federal control to states and local jurisdictions. In 1981, Congress passed the Education Consolidation and Improvement Act (ECIA), which, while maintaining the essential goals of Title I, reduced 75 pages of federal regulations to just 14. However, like previous Title I revisions, the ECIA focused little attention on instructional issues and lacked incentives to stimulate innovation. Administrative structures and veteran personnel at both the state and district levels were well-established, and traditional Title I instructional practices (e.g., the use of pull-out instruction) continued largely as a matter of custom.
With the publication of A Nation at Risk (National Commission on Excellence in Education) in 1983, the debate concerning Title I shifted from its focus on fiscal compliance issues to a heightened concern for program excellence and raising student achievement. In 1988, the Hawkins-Stafford Elementary and Secondary School Improvement Act played a key role in moving Title I toward fostering overall school improvement. This legislation required greater coordination between Chapter I ¾ as it was renamed in 1981 ¾ and regular classroom instruction, emphasized advanced rather than basic skills, and provided the basis for increased parental involvement. It also introduced the concept of state-supported "program improvement" efforts in those areas where Chapter I students showed insufficient achievement gains. Finally, by dropping local fund-matching requirements for schoolwide programs, the 1988 reauthorization offered increased flexibility in the use of Chapter I funds, giving more high-poverty schools the option to implement schoolwide services.
IASA: Catalyst for School Reform
Unfortunately, the 1988 amendments fell short of fundamentally overhauling the quality of classroom instruction for disadvantaged children. Although some modest program changes were made in response to the new policy direction primarily the expansion of schoolwide programs Chapter I did not become the intended force to drive broader school reform (Millsap, Moss, & Gamse, 1993). The 1993 National Assessment (U.S. Department of Education, 1993) concluded that:
Even the program's most fervent supporters began to openly discuss the need for wholesale changes, as evidenced by a highly influential report by a self-styled "Independent Review Panel" on Chapter I made up of leading policy experts and advocates for poor and minority children (U.S. Department of Education, 1993). The report advocated a greater focus on schoolwide reform, high academic standards for all students, increased accountability for results, and a funding formula that more narrowly targeted higher-poverty schools (replacing traditional requirements that tied funds to program-eligible students).
To a degree, these ideas were reflected in the 1994 Improving America's Schools Act (IASA), which governs the program as it operates today. In particular, the IASA sought to align federal resources and policies with existing state and local school reform efforts to create more comprehensive solutions to improve instruction for all students. There are three broad programmatic themes to the 1994 legislation:
In addition, the 1994 law promotes the philosophy that all children can succeed in mastering higher-level thinking skills; encourages the use of strategies to increase learning time (e.g., before- and after-school, extended-year, and summer programs); provides for increased targeting of program funds within districts; requires professional development that prepares teachers to teach an accelerated, high-quality curriculum; and requires schools receiving Title I funds to involve all families (not just the parents of children targeted for assistance) in ways that help students succeed in school.
Title I Funding
Title I funds are distributed to counties, districts, and schools generally in proportion to the number of poor school-age children in those jurisdictions with a guaranteed minimum allocation for smaller states and adjustments favoring states with higher per-pupil educational expenditures. The formula has changed remarkably little since 1965. The most notable changes, under the current law, are the addition of the "concentration grant" formula, which targets some funds to districts with at least 15 percent (or 6,000) poor children, and a "targeted grants" formula (as yet unfunded), which further extends the new focus on high-poverty districts. In deference to political realities, the most recent reauthorization also includes "hold harmless" provisions, ensuring that districts will receive funding at a level comparable to that of the previous year. Consequently, any shifts in funding (due to formula changes favoring greater targeting) will benefit the neediest districts only on a modest basis.
Although a few districts were eliminated from the program following a 1994 amendment requiring that Title I students make up at least 2 percent of a district's enrollment, program funds remain broadly distributed. In 1997-98, 93 percent of districts received some funding the same overall percentage as in 1987-88 and districts in the highest-poverty quartile continued to receive the same share of funds (49 percent) as they did in 1994 (U.S. Department of Education, 1999).
Some critics argue that the Title I program's impact has been diluted by the political impetus to provide "something for everyone," but there is evidence that the IASA has resulted in greater within-district targeting of funds. Districts have traditionally targeted Title I funds to schools serving children with the highest need first. But many districts defined need in educational terms, selecting those schools with the lowest test scores. The IASA introduced a series of stricter targeting rules designed to ensure that increased funding will go to those schools with the highest levels of poverty (i.e., those with more than 75 percent in poverty). Still, in schools providing targeted assistance, individual students continue to be selected for services based on educational need.
Prior to 1994, as many as 71 percent of public elementary schools received Chapter I funds. However, services were not provided to a substantial number of elementary schools about 14 percent in which 50 percent or more of the students were eligible for subsidized school meals, simply because these schools were located in districts with even more impoverished schools. As a consequence, many low-achieving students did not have access to the supplemental educational services they needed. In fact, in high-poverty schools, about one-third of the children who scored below the 35th percentile on standardized tests were not served by Chapter I (Moskowitz, Stullich, & Deng, 1993).
According to recent data from the U.S. Department of Education (1999), the situation has improved somewhat. The 1994 changes were successful in leading some districts to shift funds from low- to high-poverty schools or to cease funding some lower poverty schools entirely. In 1997-98, the program provided services to 58 percent of all K-12 public schools, a decline from 62 percent in 1993-94. Fully 95 percent of the high-poverty schools were funded, up from 79 percent, and 87 percent of those with at least 50 percent poor students were funded, compared with 78 percent in 1993-94. Conversely, the percentage of low-poverty (i.e., below 35 percent) schools receiving funding dropped from 49 percent to 36 percent during that period.
Because so many districts have traditionally focused their funding on the early grades, the 1994 rules which make it harder to exempt a high-poverty school at any grade level have prompted a precipitous increase in the percentage of high-poverty secondary schools receiving funding (between 1993-94 and 1997-98, from 61 percent to 93 percent). These increases were occasioned by a drop in the number of low-poverty secondary schools served, leading to an overall decline (from 36 percent to 29 percent) in the percentage of secondary schools receiving Title I funding.
While current funding levels are inadequate to support services for all eligible students, there is ample evidence to indicate that, generally, Title I is reaching a diverse population of children who exhibit the greatest need. Participating students tend to be concentrated in higher poverty schools and typically have lower grades and test scores than their peers. (Kennedy, et al., 1986; Puma, et al., 1993; U.S. Department of Education, 1993, 1999). Although white children constitute the largest group of participants in absolute numbers, minority students are disproportionately represented in Title I programs. Data collected by the states in the 1996-97 school year indicate that Hispanic children are the fastest growing group of Title I students, and, for the first time, a higher percentage of Hispanic children than African-American children participated in Title I during the 1996-97 school year (U.S. Department of Education, 1999).
Use of Program Funds
One of the keys to understanding Title I is to recognize that it is not a "program" in the usual sense, but rather, a financial subsidy that targets resources to certain schools and children. The 1994 amendments placed greater emphasis on accountability and the achievement of state standards, but the program does not dictate how schools should achieve these results. Once schools receive their grant, they can choose to spend it with relatively few limitations. Title I funds can be used to hire staff, train teachers, purchase computers and/or software, or run parent programs. Despite this flexibility, most schools out of economic necessity use Title I funds to pay the salaries of teachers and instructional aides, accounting for 70-80 percent of all program expenditures.
The National Assessment of Title I (U.S. Department of Education, 1999) found that, in 1997-98, 84 percent of principals in high-poverty schools reported using aides, as contrasted with 54 percent in low-poverty schools. Moreover, although few aides had the necessary educational background, 98 percent were either teaching or helping to teach students, and more than three-quarters spent at least some of this time teaching without a teacher present. In light of cost considerations, it is not surprising that the number of aides employed by Title I has grown much more quickly than the number of teachers. In 1996-97, the program supported about 74,700 teachers (up 3.75 percent from 72,000 in 1993), while the number of aides rose from 65,000 to 76,900 (up 18 percent over the same period) (U.S. Department of Education, 1993, 1999).
Most students participating in the Title I program receive assistance in reading and language arts. Fewer receive assistance in math, and fewer still receive noninstructional services, such as counseling, nutrition, or transportation services (Puma, et al., 1993). In 1993-94, the last year of the Chapter I law, 72 percent of participating students received instruction in reading, 24 percent in other language arts, 48 percent in math, and 14 percent in other instructional areas.
Delivery of Services
Until recently, the dominant method of providing Title I services has been pull-out programs that deliver supplementary instruction to low-achieving students during the time they would have spent in their regular classes. With educators driven by custom, as well as the desire to comply with financial targeting regulations, this method of instruction remained the dominant mode of service delivery through 1994 (especially in the low- to moderate-poverty schools), despite evidence that pull-outs may not always provide the best means of teaching disadvantaged children (Glass & Smith, 1977; Leinhardt, Bickel, & Palley, 1982; Winfield, 1986, 1991; Winfield & Hawkins, 1993).
Encouraged by evidence from the literature on effective schools (Brookover, Beady, Flood, Schweitzer & Wisenbaker, 1979; Brophy, 1986; Edmonds, 1986; Levine, 1990; MacKenzie, 1983; McDill & Rigsby, 1973; Purkey and Smith, 1983; Rutter, Ouston, & Mortimer, 1979), case studies of disadvantaged schools (Venezky & Winfield, 1979), and recent evaluations of special programs for disadvantaged children (Fashola & Slavin, 1998; Stringfield, Millsap, Herman, Yoder, Brigham, Nesselrodt, Schaffer, Karweit, Levin, & Stevens, 1997), there has been growing interest in alternative service delivery methods in Title I. Most notably, the use of in-class instructional approaches has increased dramatically since the years prior to the 1994 reauthorization, from 58 percent of Title I schools in 1991-92 to 83 percent in 1997-98. Conversely, use of the pull-out model has declined from 74 percent of Title I schools in 1991-92 to 68 percent in 1997-98. But in-class instructional approaches tend to supplement, rather than replace, traditional methods. In 1997-98, over half (57 percent) of Title I schools reported using both modes of instruction (U.S. Department of Education, 1999).
The percentage of schools offering extended learning time has also increased dramatically from 9 to 41 percent since the last reauthorization. In Title I schools offering instructional programs before or after school or on weekends, an average of 12 percent of students participate, while 25 percent participate in summer programs where they are offered (U.S. Department of Education, 1999).
Another important change since passage of the IASA has been the expansion of programs aimed at improving the whole school. Use of this option had been growing steadily since 1988, but accelerated after the 1994 amendments allowed more schools to qualify. According to performance reports submitted by states, there were 14,982 schoolwide programs in 1996-97, up from 3,903 in 1993-94.
By design, the Title I program primarily serves students in schools with the most disadvantaged populations and targets the lowest-achieving students in the schools it serves. It is, therefore, impossible to accurately compare the progress of Title I students with that of disadvantaged nonparticipants using traditional, nonexperimental research methods. Because school districts are obligated to serve the most needy students, potential comparison groups tend to be relatively advantaged. Although sophisticated statistical techniques can be invoked to create a "synthetic control group," such techniques "are only as good as our ability to measure those characteristics that make the two groups of students different" (Puma, Karweit, Price, Ricciuti, Thompson, & Vaden-Kiernan, 1997). Consequently, the findings from Title I evaluations are, by their very nature, inconclusive. Without an experiment in which participants and nonparticipants are randomly assigned, there is simply no way to reliably assess the effect of Title I on student achievement.
The Early Years
These caveats notwithstanding, since the early 1980s, there has been a virtual wave of Title I evaluations. These include: (1) the Sustaining Effects Study (SES), based on data collected from approximately 120,000 students enrolled in over 300 elementary schools (Carter, 1984); (2) a later reanalysis of SES data (Frontera, 1985); (3) an independent replication of the SES (Gabriel, Anderson, Benson, Hill, Pfannensteil, & Stone, 1985); (4) an analysis of Title I program data (Anderson & Stonehill, 1986); (5) analyses of other existing national data by Kennedy, Birman, and Demaline (1986); (6) the Prospects study (Puma, et al., 1997), which monitored the progress of a national sample of some 35,000 students in grades one, three, and seven for up to four years; and, most recently, (7) the national assessment of the post-1994 program (U.S. Department of Education, 1999).
The findings from these studies are mixed and, as one would expect, inconclusive, given the insurmountable methodological obstacles that researchers faced. The SES study, for example, found that the achievement gains in math and reading for Title I participants exceeded those for disadvantaged nonparticipants (but only in grades one through three), while the Prospects study examining another sample of students about a decade later found no discernable differences between the two groups. In other areas, the two studies present consistent findings. Both report evidence of a persistent learning gap between Title I students and their more advantaged peers, and both present evidence that the rate of academic progress for the two cohorts is roughly equivalent. To many, this suggests that, although Title I has not compensated for the early effects of poverty, the program may have prevented disadvantaged students from falling farther behind. Yet, in the absence of a true experiment, even this optimistic conclusion must be viewed as uncertain.
The Post-1994 Program
Because of the programmatic changes mandated by the 1994 reauthorization, including the stepped-up emphasis on standards-based reform/accountability, schoolwide programs, and greater local flexibility, the current program differs substantially from those that preceded it. Unfortunately, relatively little research has been conducted on the post-1994 program, particularly as it relates to the program's impact on students. In fact, because of the IASA-mandated transition to new state-specific assessment systems, there are no comparable data relating to changes in student performance during this period. The recently released National Assessment of Title I (U.S. Department of Education, 1999) acknowledges this, but argues that indirect evidence suggests promising trends in the success of disadvantaged children and high-poverty schools. In particular, the report points to recent National Assessment of Educational Progress (NAEP) data that track changes in academic achievement for national and state samples of students at selected grade levels. These data show that elementary-grade students in high-poverty schools the primary targets of Title I have achieved significant gains in reading and math relative to the national average between 1992 and 1996. Further, the gap between high- and low-poverty schools has been narrowed, though differences remain.
At the time the report was released, only six states had three years of consistent test-score data from new accountability systems. Students in high-poverty schools in five of the states made gains in reading, and schools in four states made gains in math. Moreover, 10 of the 13 large urban districts that reported three years of data showed improvement in at least one of the two subjects, while six reported progress in both. Finally, supporters of the current framework point out that those states quickest to adopt standards-based reforms most notably, Texas and North Carolina have shown the greatest NAEP gains.
Although some interpret these findings as compelling evidence of the positive impact of the post-1994 Title I program, a more cautious approach is subscribed to here. First, there is no way to identify scores for Title I participants, and, even if this were possible, comparisons to nonparticipants would suffer from the same methodological problems that have plagued prior studies. Second, many factors besides Title I contribute to NAEP achievement gains, making it very difficult to conclude that any rise (or fall) in scores is due solely to state or federal efforts to reform education. Thus, while the NAEP gains are consistent with a positive evaluation of Title I's impact on student achievement in the post-1994 period, they do not provide direct support for this interpretation.
Although the available research is equivocal concerning Title I's overall effectiveness in raising student achievement, research findings can still help guide policymakers in making the best use of Title I funds to influence specific school policies and practices. Therefore, in this section, we examine what is known about several key strategies designed to improve student achievement. These observations are not intended as a comprehensive review of the literature or as a formal attempt to statistically link educational inputs and policy options to student outcomes. Nevertheless, they do attempt to place the limited evidence about the effect of Title I on student achievement into a broader context of what we know about how to improve educational outcomes, especially for low-achieving students.
Policymakers and researchers have approached the problem of how to improve school performance from a variety of perspectives, and each has defined the question, the options for action, and the associated research agenda in different ways. We have grouped these studies into four overarching perspectives, according to their principal focus:
Teachers and Classrooms
Schools are complex institutions with a variety of stakeholders, numerous organizational structures and procedures, and a multitude of interactions among them. Given this complexity, it is not surprising that the most common reforms involve curriculum- or instructional-based initiatives that need not be implemented systemwide, can often operate autonomously within a single classroom, and emphasize student learning. As would be expected, the literature on these types of changes is vast, covering at least the last four decades, and has evolved to keep pace with various waves of school reform. The discussion below examines a few of the more prominent themes enriching the current policy debate.
Challenging Instruction for All
Until recently, it has been conventional wisdom that most children should be taught using a hierarchical model of instruction, in which it is assumed that basic skills, such as simple arithmetic computations, must be firmly in place before higher-order skills, such as problem solving, can be taught. As a consequence, most instruction targeting low-achieving students has traditionally emphasized the remediation of basic skills (Allington & McGill-Franzen, 1989). However, this approach has been criticized on the basis of recent research findings indicating that (a) all children can benefit from a range of learning activities, including tasks that focus on problem-solving skills (Knapp & Shields, 1990; Knapp & Turnbull, 1990), and (b) there is an association between consistent higher-order classroom instruction and greater student achievement (Rutter, et al., 1979; Mortimore, Sammons, Stoll, Lewis, & Ecob, 1988; Stringfield & Teddlie, 1988; Teddlie, Kirby, & Stringfield, 1989).
Focusing on low-achieving students, the Prospects study (Puma, et al., 1997) found a significant relationship between higher student achievement and a balanced emphasis on remedial and higher-order skills in classroom instruction. In a classroom observation study of Title I teachers, Crawford (1989) reported that greater achievement for Title I students was associated with the use of task-oriented teaching that avoided classroom disruptions, the use of academically challenging materials, and asking more "opinion" rather than simple factual questions. Another study of 140 classrooms in 15 schools across the country by Knapp, Shields, and Turnbull (1992) found that instruction for disadvantaged children that emphasizes reasoning and problem solving is more effective at teaching advanced skills, at least as effective at teaching basic skills, and better at engaging students in learning. Finally, preliminary data from an ongoing, federally-funded longitudinal study of 71 high-poverty schools, presented in the recent National Assessment of Title I (U.S. Department of Education, 1999), also suggest that certain instructional strategies produce better results in Title I classrooms. Those strategies include more total exposure to reading across content areas, opportunities for discussion in small groups, and an emphasis on understanding and problem solving in math.
Much of this guidance contrasts with the typical pattern in Title I, at least through the 1994 reauthorization, in which services were largely provided to individual students, typically in pull-out mode. This review, although limited, suggests that much can be done to improve student learning by focusing on educational practices within the classroom. That is, improvements to the everyday experiences of students would seem to have a more profound effect on their ability to learn than changes to the small segments of time spent in remedial classes. The 1994 amendments acknowledge this and, thus, promote reform strategies affecting the whole school.
Teachers Matter
Attention is increasingly focused on the quality of the nation's teaching staff (Riley, 1999). In particular, the new drive to raise standards and toughen accountability systems has significantly raised the pressure on teachers. Teachers are being asked to incorporate rapidly developing educational technologies into their classrooms, while, at the same time, they are facing a growing diversity among their students and increasing numbers of students with limited English proficiency (U.S. Bureau of the Census, 1997). Yet new survey data from the National Center for Education Statistics (NCES) indicate that relatively few teachers report feeling well prepared to deal with any of these new challenges (Lewis, Parsad, Carey, Bartfai, Farris, & Smerdon, 1999). Given their key instructional role, it is crucial that policymakers be guided by what researchers have learned about teachers in recent years.
Teacher skills matter. Recent efforts to estimate the "value added" effect of teachers on student test scores in Tennessee have yielded three important conclusions: (1) some teachers are consistently effective in achieving positive gains in student test scores, while others are not; (2) the effect of teachers (both positive and negative) on test scores is cumulative over time; and (3) teacher effects are substantial for all students, especially those achieving at the lowest levels (Sanders & Rivers, 1996; Wright, Horn, & Sanders, 1997). Similar results have been reported by researchers in Dallas and Boston (Haycock, 1998).
Attempts to measure the characteristics of "effective" teachers have been seriously limited by the available data. However, researchers have found positive relationships between students' test scores and teachers' own scores on standardized exams (Ehrenberg & Brewer, 1994, 1995; Ferguson, 1991; Greenwald, Hedges, & Laine, 1996). Other researchers (Ferguson & Brown, 1998) report a strong relationship between students' scores and teachers' literacy skills.
Researchers also report a strong association between student achievement and teachers' training in specific subject areas (Goldhaber & Brewer, 1996). For example, students taught by teachers with an undergraduate or graduate degree in mathematics or science score higher in those subjects on standardized exams (Brewer & Goldhaber, 1996; Goldhaber & Brewer, 1996). Yet, in 1994, over one-third of public elementary school teachers, and nearly half of those in high-poverty schools, were teaching out-of-field (Ravitch, 1999).
Teacher supply is a constraint. Concerns about teacher quality have been further heightened by the realization that, in the next decade, there will be an estimated need for more than two million new teachers (Haselkorn, 1997; National Commission on Teaching and America's Future, 1996), and this estimate may be low if the push to lower class size really takes hold. Two approaches to increasing supply are (a) to change existing pay scales or (b) to alter the requirements for entry into the profession. With regard to the first approach, Murnane and Olsen (1989) found that highly-compensated teachers are more likely to remain in the field, although that is less true of math and science teachers who have private-sector alternatives. There is also some evidence that districts paying higher salaries are more able to recruit higher-quality teachers (Figlio, 1997), but other research suggests that even when they are able, districts may not always select the best teachers (Ballou & Podgursky, 1998). Although there is increasing interest in the use of differential compensation, we know very little about the effect of policies that reward the most effective teachers or those that base teacher pay on knowledge and skills (Drury, 1999; Odden & Kelley, 1997). It is clear, however, that current compensation policies, which reward teachers for years of service and the number of academic credits beyond a bachelor's degree, are poorly aligned with achievement objectives. With regard to the second approach changing the requirements for becoming a teacher several states have already begun to explore alternatives to current licensing procedures. Researchers find little evidence of a relationship between traditional teacher certification policies and productivity (Ballou & Podgursky, 1998), and, though some express concern that alternative certification programs may have adverse consequences for teacher quality, others report that these programs result in greater diversity (Shen, 1998; Villegas & Clewell, 1998).
Professional development is critical. In addition to attracting and retaining the best individuals, schools must confront the need to upgrade and maintain the skills of their existing staff. Unfortunately, professional development has typically been a low priority in most districts, and recent data indicate that teachers, on average, receive one day or less of training per year (Lewis, et al., 1999). Although the literature on professional development offers relatively little information concerning its effect on student achievement, there are some indications that high-quality training where activities are intensive and extend over long periods of time can positively affect instructional practice (Ball & Rundquist, 1993; Corcoran, Shields, & Zucker, 1998; Heaton & Lampert, 1993; McCarthy & Peterson, 1993; Wiley & Yoon, 1995; Wilson, Miller, & Yerkes, 1993). Most recently, a study by Cohen and Hill (1998) in California found that professional development is likely to have the greatest effect on student achievement when it is closely aligned with expectations and standards, curriculum, and student assessment systems. But, research findings also indicate that one-shot or short-term professional development activities are unlikely to have significant lasting effects on classroom behavior. According to Corcoran (1995), high-quality professional development should include: (1) opportunities for "joint work" (such as team teaching and school curriculum committees) that foster greater interdependence among teachers; (2) teacher networks that create professional communities and opportunities to share knowledge; and (3) better collaborations between schools and universities. Collaborative work and greater interdependence may also give rise to professional norms governing individual behavior, thus increasing teacher accountability (Drury, 1999).
There is a strong consensus, then, that efforts to build the capacity of teachers must be the cornerstone of any school reform process (Cohen, 1994). As Elmore and McLaughlin (1988) observed more than a decade ago, "administrative decisions can reflect policy more or less accurately and can set the conditions for effective practice, but [can not] control how teachers will act in the classroom at a given point." Teachers represent the critical link between theory, change in practice, and the impact of educational policies on student learning. Because disadvantaged children are often taught by the least effective teachers (Haycock, 1998),3 the need to build teachers' capacity in high-poverty schools is especially great.
Smaller is Better4
As districts struggle to identify the most appropriate strategies for raising student achievement, class-size reduction (CSR) has emerged as an increasingly popular alternative. The strongest evidence for CSR to date is an experiment commissioned in the late 1980s by the Tennessee legislature, known as the Student-Teacher Achievement Ratio (STAR) study. In the STAR experiment, students and teachers in 79 Tennessee schools were randomly assigned to three types of classes: (1) small classes with 13-17 students; (2) regular classes with 22-25 students; or (3) regular classes with a full-time teacher's aide. The study continued for four years kindergarten through third grade and achievement data on both criterion- and norm-referenced tests were collected each year. Researchers reported significant test-score gains for students enrolled in smaller classes, across all subject areas and for each year of the experiment, but found no effect associated with the addition of a teacher's aide. The observed gains were most pronounced for minority and underprivileged students (Finn & Achilles, 1999).
A recent reanalysis of the STAR data, applying a more sophisticated statistical approach that addresses several design problems in the original study, supports these basic conclusions. However, the reanalysis also suggests that the main benefit of CSR manifests itself by the end of the first year of a child's exposure to small classes. Researchers have interpreted this as evidence that there is a one-time school socialization effect due to small classes that raises the level of a student's achievement in the first year, followed by smaller positive effects in subsequent years (Kruger, 1998).
Although the impact of CSR on achievement seems to decline after a child's first year of exposure to small classes, recent studies demonstrate that the cumulative benefits of small classes are persistent. For example, the latest round of STAR research which follows subjects through secondary school has found that students originally assigned to small K-3 classes are more likely to have college aspirations, as evidenced by their higher rate of participation in college entrance examinations. Consistent with previous STAR findings, this difference is most pronounced for minority students and for those eligible for free or reduced-price lunch (Kruger & Whitmore, 1999). These latest analyses also suggest that, compared with their peers assigned to regular-size classes, students exposed to small K-3 classes complete more advanced coursework in secondary school, have lower dropout rates, are more likely to graduate on schedule, and are more likely to graduate in the top tenth of their classes (Pate-Bain, Fulton, & Boyd-Zaharias, 1999).
Other recent studies lend further support to the STAR findings. Examining fourth- and eighth-graders' performance on the National Assessment of Educational Progress (NAEP), one investigation concludes that students in small classes defined as fewer than 20 students perform better than those assigned to regular-size classes, even after controlling for other factors that might influence test scores. According to the study, students assigned to smaller classes can expect to progress at a faster rate than those assigned to larger classes 33 percent and 12.5 percent faster for fourth- and eighth-graders, respectively. Even more striking, fourth-graders assigned to smaller classes in inner-city schools can expect to progress 75 percent faster than their peers in larger classes (Wenglinsky, 1997).
New findings from Wisconsin's Student Achievement Guarantee in Education (SAGE) program also attest to the effectiveness of class-size reduction. The SAGE program which targets schools with 50 percent or more of their students in poverty limits class sizes to 15 for grades K-3. A recently published study evaluates data from two years of the program's four-year phased implementation. (The program began in 1996 with kindergarten and has expanded each year to include additional grades.) According to the study, first-grade students in SAGE classrooms significantly outperformed their counterparts in other classrooms. While the advantage associated with smaller classes did not grow in second grade, neither did it decrease. Equally important, because the effect was strongest for African Americans, the black-white achievement gap narrowed in SAGE classrooms, while it widened in those classrooms unaffected by the program (Molnar, Smith, Zahorik, Palmer, Halbach, & Ehrie, 1999).
On balance, the findings from the STAR experiment, as well as those from more recent investigations, suggest that significant class-size reduction is likely to yield positive effects on student achievement, especially in the early elementary grades and for minority students. Yet, it is important to keep in mind that smaller classes can be achieved only at a substantial cost. The principal costs fall into three categories: (1) salaries for additional teachers; (2) the cost of building new or expanding existing facilities; and (3) operational costs, including the cost of classroom equipment and support staff. If, however, smaller classes result in less student retention, fewer children with special education needs, or early detection of learning disabilities, these costs may be offset, at least in part, by reductions in future expenditures.
Large-scale CSR initiatives also raise concerns about the potential difficulty in finding and recruiting qualified teachers to meet the new demand for professional staff, especially in light of our earlier remarks about teacher-skill deficiencies. Indeed, a recent study of Californias class-size reduction program reports significant problems in implementing large-scale initiatives of this type, at least in the short term and particularly in high-poverty districts (Stecher & Bohrnstedt, 1999). Poor and minority districts were the slowest to implement the changes; many schools were forced to use space originally set aside for other functions; some districts (e.g., those serving large numbers of disadvantaged students) had to dip into other resources to obtain funding; and, as more teachers were hired, the overall preparedness of staffs declined, especially in poor and minority schools.
It is important to emphasize that class-size reduction is just one of several approaches to increasing student achievement, not an end in itself. In allocating scarce resources, policymakers should always compare the costs and benefits of alternative reform strategies, and CSR is no exception. Still, based on the evidence presented here, flexible, targeted class-size reduction programs particularly those aimed at disadvantaged children in the early elementary grades seem likely to produce significant achievement gains and may also contribute to a reduction in the performance gap separating advantaged and disadvantaged students.
Educational Technology
The growing global importance of information technology has spurred a rapid increase in the use of computers in American schools, from one for every 125 students in 1983, to a computer for every nine students in 1995 some schools even have one computer for every two students (Glennan & Melmed, 1996). As of 1998, three-fourths of all classrooms had at least one computer designated for instructional use (Technology Counts, 1998). In addition, about 89 percent of schools now have Internet access, up from only 35 percent in 1994, and 51 percent of classrooms have such access, up from 27 percent in 1997 (National Center for Education Statistics, 1999). Moreover, earlier differences in access between high- and low-poverty schools appear to have been eliminated, although differences in classroom-level access still exist. Given these statistics, it is not surprising that, in 1998, 42 states invested in education technology, with funding varying from $500,000 in Vermont to $230 million in California (Technology Counts, 1998). Title I represents an important source of funding for this technological expansion. Indeed, Title I funds have paid for a significant portion of the computers now in use in high-poverty schools. Since the implementation of the E-rate a federal initiative that provides crucial discounts on telecommunications and Internet technologies to disadvantaged elementary and secondary schools the purchasing power of Title I dollars has increased significantly. In its first year, E-rate funding for schools and libraries totaled $1.1 billion and is expected to reach $2.2 billion in 1999-2000.
Those who advocate greater use of technology in the classroom argue that America's schools should be transformed into electronic learning centers, increasing both the efficiency of classroom instruction and student motivation to learn (American Association of School Administrators, 1996; Glennan & Melmed, 1996; Means & Olson, 1995). Others, however, express concern about the unequal educational access to technology, particularly in schools with high concentrations of poor and minority students (Coley, Cradler, & Engel, 1997). For now, anyway, black, poor, urban, and rural students are less likely to have access to a home computer, be exposed to higher-order uses of computers in school, and have teachers who have the necessary training in technology (Wenglinsky, 1998).
Of equal concern is the fact that too many teachers are either unwilling, or untrained, to use the new forms of technology (Becker, 1990; Cuban, 1993; National Academy of Science, 1995; Technology Counts, 1998) and that relatively few teachers use computers for a significant part of their daily instruction. As a consequence, many emphasize the need to build greater capacity in teaching and more fully integrate technology into pedagogy (Brown, 1997; Office of Technology Assessment, 1995; Coley, et al., 1997; National Council for Accreditation of Teacher Education, 1997; Solmon, 1998). Studies indicate that it is not simply access to technology that is important, but rather, how teachers use it as a tool to enhance learning (Thompson, Simonson, & Hargrave, 1996). For example, a recent study on the use of computers for math instruction found that students of teachers who used computers for higher-order teaching in math did better on the NAEP tests, but students whose teachers used the computers for "drill and practice" of basic skills did worse (Wenglinsky, 1998).
Other research on the impact of new technology on student learning suggests additional advantages. The Apple Classrooms for Tomorrow (ACOT) Project (Dwyer, 1996), which has been implemented in hundreds of classrooms, has reported positive impacts on student attitudes, motivation, and learning. Means and Olson (1995) conducted case studies of modern technology in very disadvantaged schools and found higher levels of teacher-reported increases in student motivation and learning. The Center for Applied Special Technology (1996) reported positive effects on student learning from the increased availability and use of the Internet for classroom instruction. Finally, Mann, Shakeshaft, Becker, & Kottlamp (1999) examined West Virginias Basic Skills/Computer Education (BS/CE) program and reported that the effective use of learning technology has led directly to significant gains in math, reading, and language arts skills. The programs 10-year history makes it the nations longest-running state program for the implementation of technology in education. The findings were particularly positive for low-income and rural students and for children without computers at home.
Realizing the need for a greater understanding of how, and under what circumstances, technology can be used to improve student achievement, the President's Committee of Advisors on Science and Technology (PCAST) issued a 1997 report on the use of technology to strengthen K-12 education. As part of that report, the committee recommended a broad research agenda, including empirical studies to determine which approaches to the use of technology are most effective.
Individual Schools
As has been noted previously, the past two reauthorizations have moved the focus of Title I more in the direction of those reforms with the potential to impact the whole school, as opposed to traditional programs, which have specifically targeted Title I students (e.g., pull-out programs). Although such schoolwide programs have been sanctioned under Title I since 1978, they were rarely implemented until the 1988 Hawkins-Stafford Amendments removed the requirement that districts provide matching funds. More recently, the 1994 Improving Americas Schools Act further expanded this Title I option by lowering the poverty threshold for participating schools from 75 percent of enrolled students to 50 percent. In many major urban school districts, this change allowed essentially all Title I schools to implement schoolwide programs.
Schoolwide Programs
Much of the impetus for the idea of schoolwide reform comes from work identifying various characteristics of effective schools, including: strong instructional leadership; a clear academic focus and high student expectations; a dedicated and highly motivated administrative and teaching staff; an orderly and disciplined school environment; and a positive school climate, particularly one that emphasizes a community spirit. Subsequent studies (Bryk, Lee, & Holland, 1993; Coleman & Hoffer, 1987; Coleman, Hoffer, & Kilgore, 1982; and, Puma, et. al., 1997) corroborate these early findings and suggest that effective schools are places in which administrators and staff are actively engaged as a learning community continuously seeking ways to raise student achievement (Drury, 1999; Shields, Anderson, Bamburg, Hawkins, Knapp, Ruskus, Wechsler, & Wilson, 1995). Similarly, research on the attributes of successful high-poverty schools (Ragland, Johnson, & Lien, 1997) indicates that these schools share (a) an unwavering focus on the mission of improving academic achievement that forms the basis for every decision, (b) a "no excuses" attitude and an eagerness to experiment with new approaches, and (c) a strong "sense of ownership" throughout the school community
Despite the broad use of Title I schoolwide programs, there is relatively little information available on the impact of this approach on student achievement, and most of what is known comes from Title I evaluations that predate the 1994 reauthorization (Pechman & Fiester, 1994). A special reanalysis of pre-1994 data, commissioned for this report (Puma & Price, forthcoming), indicates that students in high-poverty Chapter I schools choosing the schoolwide option failed to demonstrate greater achievement gains in reading or math than Chapter I students receiving targeted assistance (e.g., through pull-out instruction). However, before 1994, most schoolwide programs were limited to one or more isolated aspects of participating schools' educational programs, such as the acquisition of new technology or the introduction of math manipulatives, and only rarely emphasized a comprehensive approach involving the articulation of multiple facets of a broader educational program.
Comprehensive School Reform
Passage of the Comprehensive School Reform Demonstration (CSRD) initiative in 1997 has helped spur the rapid growth of more comprehensive approaches to school reform. Commonly referred to as "Obey-Porter" (after its Congressional sponsors), the CSRD program provides $150 million each year to assist schools with the implementation of research-based whole-school reforms (grants are available for up to $50,000 per year for three years). The initial legislation listed 17 programs as examples of effective comprehensive school reform models. The more common models, plus those developed by the New American Schools Design Corporation (NASDC) a nonprofit foundation funded by corporate America to develop and implement "break the mold" school reforms are presented in the box below.5 As of the 1998-99 school year, about 2,500 schools had received CSRD grants.
Thus far, evidence of the effectiveness of these comprehensive school reforms is either unavailable or inconclusive, demonstrating the need for more independent and rigorous evaluations. For example, evaluations of the Comer model by its developers at Yale University have, for the most part, been based on simple comparisons of Comer students with "comparable" students in the same district. Similarly, evaluations of Success for All (SFA) undertaken by staff of Johns Hopkins University have relied almost exclusively upon comparisons with students enrolled in so-called "matched" schools. Although these evaluations have produced evidence of higher achievement among program participants as well as evidence that the lowest-achieving students exhibit the highest gains (Fashola & Slavin, 1998; Madden, Slavin, Karweit, Dolan & Wasik, 1993; Slavin, Madden, Karweit, Dolan & Wasik, 1992; Slavin, Madden, Dolan, Wasik, Ross, Smith & Dianda,1996; Slavin, Madden, Karweit, Livermon, & Dolan, 1996) there is a critical need for additional study, employing more sophisticated research designs. Recent investigations of the effectiveness of the new NASDC initiatives are equally limited. Fashola and Slavin (1998) report some early positive results for Slavin's Roots and Wings program, based on test-score comparisons between students in the demonstration schools and all students in the state. And, Ross, Sanders, and Stringfield (1998) report some preliminary data showing positive increases in the rate of growth in student achievement for students in 25 Memphis schools that are implementing six of the NASDC designs, as well as the Accelerated Schools and Paideia models. However, findings are not disaggregated for individual models, and there is reason to suspect that the self-selection of highly effective teachers into reform schools may have skewed the results. Thus, while comprehensive school reform programs seem to offer a more effective approach than schoolwides that focus on just one or two aspects of schooling, we still have much to learn in this area.
|
Standards and Systemic Reform
The first wave of interest in "systemic" change in American education came after the 1983 release of A Nation at Risk (National Commission on Excellence in Education, 1983), which highlighted the low test scores of American students relative to those of their international counterparts. In part, the report blamed poorly trained teachers and low standards of acceptable student work for the "rising tide of mediocrity" in American schools, and as a result, states began reforming their systems creating tougher graduation requirements, longer school days, and more concentrated teacher training. When test scores did not rise sufficiently to declare and "educational victory," a second wave of reform began that relied on organizational theory to suggest increased attention to decentralization and site-based management. Most recently, these ideas have been supplemented by the concept of "standards-based" reform, which forms the foundation upon which the 1994 Title I reauthorization was built.6 But, as Drury (1999) points out in a recent book on school-based reform, "setting standards without giving schools (and school districts) the resources to become rational, productive organizations is an exercise in futility." What, then, do we currently know about the impact of standards-based reform and tougher accountability systems on student learning? And, what kinds of resources and technical assistance do school districts require in order to achieve the new high standards of learning?
Raising the Bar: Standards-Based Reform
Standards-based reform looks beyond the individual school to change the entire system of education through: (1) development of challenging academic standards and achievement expectations for all students; (2) alignment of policies and practices with these standards (including curriculum, assessment, professional development, instructional materials, and parental involvement); (3) strengthening of governance systems to support greater flexibility and innovation at the school level (e.g., giving schools the capacity and incentives to create effective strategies for preparing their students to learn the new standards); and (4) implementation of accountability systems with appropriate incentives and sanctions tied to the achievement of expected standards of performance (Smith & O'Day, 1990). Although many states had already taken the first steps toward the implementation of these ideas, the IASA sharply accelerated the process by, in effect, requiring states to adopt standards and aligned assessments as a condition for participation in Title I. Most now have in place "content standards" that identify what students should learn in particular subject areas, but fewer have developed "performance standards" that clearly identify what students should know and be able to do (Council of Chief State School Officers, 1998a; McLaughlin, Shepard, & O'Day 1995).
While states, districts, and schools have begun to implement standards-based reform, some observers have questioned its underlying assumptions. First, as Weiss (1999) notes, the "theory [of standards-based reform] is exceedingly thin, specifying overall goals, but providing little guidance on how to go about meeting those goals." Second, there are legitimate concerns that because of the quality of teachers in high-poverty schools, disadvantaged students will not receive adequate support to reach the new standards. Third, such broad policy changes have rarely been found to exert a significant influence on student learning (Wang, Haertel, & Walberg, 1993), and top-down reforms have a poor track record in altering educational practice (Elmore, 1994). Some even suggest that increased centralization will create a greater focus on regulatory compliance, derailing local innovation and reducing sensitivity to local educational needs (Knapp, 1997). Finally, lack of local capacity for reform can be an inhibiting factor (O'Day, Goertz, & Floden, 1995), as suggested by a recent study demonstrating that high-poverty districts face greater impediments to implementing standards-based reform (Hannaway & McKay, 1999). In particular, there is evidence to indicate that broad, integrated reforms are difficult to implement (Zucker, Shields, Adelman, Corcoran & Goertz, 1998), and that as school innovations spread from their initial "laboratory" they tend to "lose their steam" (Elmore, 1994). Reforms are less likely to encounter the initial level of support, enthusiasm, and commitment that made them successful as pilot programs, and later adopters often face substantially more difficult circumstances that can increase the challenge of making an innovation work successfully.7 Collectively, these concerns point to the need for an expanded capacity at the district and school levels, a theme which is developed further below (see "Building Capacity").
Creating Incentives: Accountability Systems
A key component of standards-based reform is the use of student performance assessments and other indicators to evaluate productivity. Proponents of better accountability argue that if schools are focused on results and given incentives to achieve desired levels of performance, the goal of higher student achievement will ultimately be realized (Hanushek, 1996a, b). Indeed, the Title I law explicitly gives states responsibility for providing technical support to struggling schools and districts and gives the federal government the authority to oversee states' implementation of this scheme. However, many observers particularly advocates for disadvantaged children have been highly critical of these efforts, at least in their early stages of implementation (Citizen's Commission on Civil Rights, 1998).
Most traditional state and district accountability systems like the traditional accountability system for Title I have focused on inputs (not outcomes), and have taken a regulatory approach using a centralized system of rules and punishments for noncompliance. But such regulatory models have several drawbacks: (1) they assume that there is a known "best" way to achieve desired goals; (2) they are costly to administer; and (3) if poorly implemented, they can prevent the adoption of effective practices. A good example of this latter type of failure is the widespread adoption of Title I pull-out instruction in response to tightened regulatory compliance. Given the inherent deficiencies of the regulatory approach, many argue that it makes more sense to give local decision makers the freedom to choose their own strategies. Because education is a highly decentralized activity teachers working individually in their classrooms the potential for creating effective prescriptive regulations to improve student achievement is severely limited.
Although new accountability systems are growing in popularity (see the box below), recent studies of their implementation in 10 states (Elmore, Abelmann, & Fuhrman, 1996; Massell, 1998) reveal a number of issues that affect successful implementation: (1) determining how to measure student performance, especially the choice of an appropriate test of achievement; (2) deciding what constitutes "good" and "bad" performance and satisfactory progress; (3) making appropriate adjustments to school scores for differences in the types of students enrolled; (4) developing procedures to avoid perverse incentives (e.g., teaching to the test, exclusion of certain children, etc.); (5) making the accountability system fair, both in appearance and in substance; (6) developing sufficient capacity to implement remedies for poor-performing schools; and (7) finding ways to motivate the schools in the "middle of the distribution," where less attention has traditionally been directed.
The New Accountability Systems
Fuhrman (1999) identifies several characteristics of the new accountability systems springing up under the influence of standards-based reforms:
|
Examining the effectiveness of the new accountability systems, Clotfelter and Ladd (1996) report higher pass rates in reading and math in the Dallas schools, where reward systems have been instituted for higher-performing schools. The authors caution, however, that deciding how much of the observed difference is attributable to the new systems is "hard to assess." Similarly, in a recent study of California schools, Herman (1997) acknowledges that assessment practices are not the sole factor subject to change, making it hard to disentangle the effects of testing from significant investments in teacher capacity building and professional development. An earlier study, by Shepard, Flexer, Hiebert, Marion, Mayfield, and Weston (1995), which examined the Maryland comprehensive performance assessment system, reported no student achievement gains attributable to the testing program in reading and only small gains in mathematics. But these tests were administered before standards and curriculum reforms were implemented. The authors of the Maryland study note that "performance assessments did not automatically improve student learning [and that] when teachers' beliefs and classroom practices diverge from new conceptions of instruction, it may be more effective to provide staff development to address those beliefs and practices directly." Thus, these authors conclude that, while "performance assessments are a key element in instructional reform, they are not by themselves an easy cure-all."
Perhaps the most compelling evidence to date in support of the new accountability systems is contained in a recent report by the National Education Goals Panel (Grissmer & Flanagan, 1998). Annually, the Goals Panel tracks and reports on some 33 indicators linked to the eight National Education Goals. In its 1998 report, two states North Carolina and Texas stood out for realizing positive gains on the greatest number of indicators, including the largest average gains in student scores on the tests of the National Assessment of Educational Progress (NAEP) administered from 1990 to 1997. After discounting various competing explanations, the study concluded that "the most plausible explanation for the test score gains" is found in the educational policy environment of the two states. Especially relevant in this context are three key elements of those policies: (1) accountability systems with consequences for results (test score gains are employed as the primary means of ranking schools and schools are rewarded for improved performance); (2) statewide assessments closely linked to academic standards (assessments are conducted annually in every grade from 3 to 8 in reading and math); and (3) data for continuous improvement (student test score data and other information as provided to students, parents, teachers, and school districts through sophisticated computer-based information systems). While the Goals Panel's findings do not constitute definitive evidence of the impact of the new accountability systems, they are, nonetheless, encouraging.
Building Capacity
If standards-based reforms and the new accountability systems are to prove successful in increasing productivity in America's public schools, districts must develop the capacity to support and nurture school-based innovation and change. Systemic change of this kind implies the fundamental restructuring of the school, the district, and their interrelationships (Fullan, 1991; Marsh, 1994). The role of the school district is especially crucial in this regard, since district action must create and sustain the context for successful reform.
District-level authorities must play a central role in guiding the process to establish a districtwide vision of education. This process encompasses several important dimensions, including the development of district goals, content and performance standards (based on, but not limited by, state standards), indicators of success, and districtwide accountability systems. School districts must also develop the capacity to support new and innovative practices at the school level through the creation of modern information and reporting systems, professional development programs, and systems of rewards aligned with district objectives.
Drury (1999) identifies three key resources that must be developed at both the district and school levels to provide an infrastructure conducive to school-based improvement:
Until school systems develop greater capacity in these critical areas, new accountability systems are likely to fall short of their ultimate educational objectives. Some districts, such as Dallas, Texas, have already implemented modern information systems, and others have adopted data-driven decision making as an integral part of their culture with considerable success (Schmoker & Wilson, 1993), but most districts operate in a virtual information vacuum, like "ships without rudders" (Drury, 1999). Further, while New York City's District 2 and Louisville, Kentucky have made professional development the centerpiece of district reform efforts, most districts across the country have developed little capacity in this critical area. Finally, a few districts have begun to experiment with performance-based reward systems and incentives designed to attract the most effective teachers to struggling schools, but this, too, remains a rarity in American public education.
Beyond the Traditional Classroom
The extent to which children learn and achieve success in school is not simply a reflection of what happens to them while they are in school. Children's development and learning reflect a host of influences from their family and community both before they enter school and during the time they are not in school. There are several ways that Title I funds can be used to reach outside school walls to affect the academic achievement of disadvantaged students. Title I funds can be used to involve and educate parents, to extend the school day, or combined with other funding streams, such as Head Start to improve and expand early intervention services for poor children.
Parental Involvement
Title I has mandated parental involvement in its programs for decades at varying levels of specificity. Every Title I district must use at least 1 percent of its budget for parent activities, which can include formal parent advisory councils, parent centers, social events, and educational or social services. Title I funds can also be used to pay for transportation and child care to facilitate parental involvement in schools. The 1994 changes to Title I strengthened the law's emphasis on school/family community partnerships by: (1) specifying that partnerships with families should be linked to student learning; (2) asking schools to develop, jointly with parents, a "compact" that outlines how parents, school staff, and students will share responsibility for improving student achievement; and (3) allowing funds to be commingled to create unified programs that serve all parents.
This emphasis on parental involvement is supported by research showing that the support of parents at home can have a positive effect on students' achievement, attendance, school adaptability, and classroom behavior, as well as a positive effect on parents themselves by giving them the tools to help their children at home (Epstein & Hollifield, 1996). A recent review by Henderson and Berla (1994) discusses a number of parental activities associated with positive academic outcomes for children, including: (1) establishing daily family routines, such as providing a quiet time and place to study, establishing times for going to and arising from bed, eating dinner together, etc.; (2) monitoring out-of-school activities by, for example, limiting TV, arranging after-school activities and supervised care, etc.; (3) modeling the value of learning and hard work; (4) expressing high, but realistic, expectations for achievement (e.g., setting goals and standards, encouraging special talents, etc.); (5) encouraging children's progress in school (e.g., showing interest in school achievement, helping with homework, staying in touch with teachers, etc.); and (6) reading and engaging in discussions among family members (e.g., reading together, discussing the day's events, etc.). Not surprisingly, Phillips, Crouse, and Ralph (1998) report that as much as half of the twelfth-grade difference in student achievement between white and African-American students may be attributable to differences that existed at initial entry into school, and Hedges and Nowell (1998) speculate that it was an increased emphasis on student achievement by African-American parents that largely explains the narrowing of the black-white test score gap observed during the 1970s.
Obviously, schools cannot hope to alter the complex nature of parenting. But they can bring parents into the educational process as partners with schools and teachers, and this can be accomplished in ways that encourage the types of behaviors and interactions described above. For example, in a study based on Prospects data, D'Agostino, Wong, Hedges, and Borman (1998) found that Title I parent involvement programs that foster strong parent-teacher communications can increase parents' efforts to work with their children at home, which, in turn, can influence student achievement. But, as Epstein and Hollifield (1996) warn, not all school/family/community partnerships lead to higher student achievement their success depends largely on how these programs are structured. Most promising are comprehensive programs of school, family, and community partnerships that foster communications with families and community partners and emphasize the importance of all parents' efforts to work with their children at home (Epstein, 1995).
While the evidence on the impact of the home environment on academic achievement is compelling, there is much to learn about how parenting behavior can be modified to positively influence student development. Teachers and administrators should not expect parental involvement to solve all the problems facing schools with high concentrations of poor children. They should, however, seek to foster communications that create strong parent-teacher partnerships and emphasize the importance of parents' efforts to work with their children at home.
Preschool Interventions
Traditionally, Title I funds have been used to address the remediation of educational deficits, rather than their prevention. Although preschool programs have been authorized under Title I since its inception, few dollars have actually been allocated to prepare students for school. Increasingly, however, policymakers have come to recognize that children who enter school prepared will achieve more than those who receive remediation services after the fact.
High-quality early childhood programs can have large effects on children's cognitive development (Barnett, 1995; Campbell & Ramey 1994; Consortium for Longitudinal Studies, 1983; Layzer, Goodson, & Layzer, 1990; Lee, Brooks-Gunn, & Schnur, 1988; McKey, Condelli, Ganson, Barrett, McConkey, & Plantz, 1985; Reynolds, 1992), and there is evidence to suggest that some programs may affect socioemotional functioning as well (Lee et al., 1988; McKey, et al., 1985). But, in the absence of adequate environmental supports during the early years of schooling, these effects can begin to fade (Barnett, 1995; Castro & Mastropieri, 1986; McKey, et al., 1985; Ramey & Ramey, 1992). Lee and Loeb (1995) suggest that one of the reasons for the fade-out of early preschool gains is that disadvantaged children often go on to elementary schools of lower than average quality. Consequently, researchers agree that preschool programs that include early, intensive intervention, along with continued follow-up as children enter school, have the strongest effects on later achievement (Ramey & Ramey, 1992; Wasik & Karweit, 1994; Yoshikawa, 1994).
Extended Learning Time
Students spend about 70 percent of their waking hours outside of school (Clark, 1993), time that "is seldom spent in activities that reinforce what they are learning in their classes" (Steinberg, 1996). More typically, students' energy is focused on activities that "compete with, rather than complement, their studies." Thus, in an analysis of some 20,000 teenagers, Steinberg found that roughly two-thirds were employed, and about half were working more than 15 hours per week. Harris (1998), in a widely discussed book, also demonstrates the importance the interactions that occur among students, both within and outside of school.
Out-of-school Time Programs. Based on an analysis of how children use their out-of-school time, Chimerine, Panton, and Russo (1993) recommended that, rather than supplant community-based activities for children, Title I should "encourage children to take part in productive out-of-school activities, facilitate coordination among existing programs, raise awareness among parents and community members....and help ensure that students have access to the programs they want and need." Program funds can also be used to offer "instruction before or after school and during school vacations [to] reduce the amount of class time students miss for pull-out programs." The IASA encouraged Title I schools to consider such approaches, and recently, funds have been made available for the creation of 21st Century Community Learning Centers. As a result, the proportion of Title I schools offering extended learning time programs has increased dramatically, from 9 percent to 41 percent since 1994 (U.S. Department of Education, 1999). Still, more than half of all Title I schools offer no programs of this kind.
A number of studies have found that children who attend high-quality after-school programs display better peer relations and emotional adjustment than children lacking this experience (Baker & Witt, 1995; Posner & Vandell, 1994). Other studies have reported improvements in social skills (Carlisi, 1996; Steinberg, 1996), school grades (Brooks, Mojica & Land, 1995; Carlisi, 1996; Gregory, 1996; Mayesky, 1980a, b; Posner & Vandell, 1994; Riley & Steinberg, 1994), and school work habits (Posner & Vandell, 1994; Vandell & Pierce, 1997). Finally, a recent review of the literature by Fashola (1998) identifies a number of after-school programs whose effectiveness is supported by research, including several studies employing rigorous research designs featuring treatment and comparison groups. Nonetheless, while the evidence on out-of-school programs is compelling, few would argue that such programs should compete with in-school programs for funding.
Tutoring Assistance. Tutoring represents another approach involving an extension of learning time that has gained popularity in recent years, especially those programs designed to help children in the early grades learn to read. A meta-analysis of 65 studies (Natriello, McDill, & Pallas, 1990) concluded that same- and cross-age tutoring can positively affect achievement in targeted subject areas. However, the Special Strategies study (Stringfield, et al., 1997), which examined four popular tutoring programs in Chapter I schools, failed to find positive effects on student achievement (although these latter findings were affected by a very small sample size and other methodological constraints). In researching an approach using trained adult volunteers, Wasik (in press) concluded that "there is a surprising lack of evidence about achievement effects of one-to-one tutoring by volunteers," despite the rapid proliferation of these methods and the general belief that they are effective. On balance, the research evidence is inconclusive about the general effectiveness of intensive tutoring programs and casts some doubt on the effectiveness of those programs that utilize volunteers.
Extended School Year. For some time, researchers have recognized that students' performance tends to "slip" after the long summer vacation and that this effect is particularly pronounced for disadvantaged children. A review of 39 studies by Cooper, Nye, Charlton, Lindsay and Greathouse (1996) reported that:
the summer loss equaled about one month on a grade-level equivalent scale, or one-tenth of a standard deviation relative to spring test scores. The effect of summer break was more detrimental for math than for reading and most detrimental for math computation and spelling. Also, middle-class students appeared to gain on grade-level equivalent reading recognition tests over summer while lower-class students lost on them.
Possible explanations for these findings include the lack of adequate environmental supports in the homes of underprivileged students, differences in the opportunity to practice different academic material over the summer (with reading practice more available than math practice), and the greater likelihood of memory decay for fact- and procedure-based knowledge than for conceptual knowledge.
In an earlier study, Alexander and Entwisle (1994) presented dramatic evidence that disadvantaged students have rates of increase in academic achievement that are on a par with other students when they are in school, but, as a result of their impoverished home/community environments, lag far behind in summer growth, leaving them no better off (or worse off) at the beginning of the next school year. Similar findings were reported by Karweit, Ricciuti, and Thompson (1994), based on an analysis of Prospects data. These researchers also found that the decline compounds over time that is, the loss for poor children continues in each year of schooling, keeping them behind their more advantaged classmates despite the gains they achieve while in school.
Interest in extended-year schooling has grown, largely in response to such findings, and also as a consequence of American students' low performance on international tests. Proposed solutions include the addition of more instructional time during the summer months, as well as more extensive revisions to the total school schedule. For example, the National Education Commission on Time and Learning (1994) and others have urged school districts to extend the school year to increase the number of days children spend in school. In terms of Title I, the most common application of these ideas has been summer programs for disadvantaged students.
Research on the effectiveness of such reforms has, however, been limited. In a meta-analysis of the available research, Kneese (1996) found that schools that extended the school year achieved positive gains in student academic achievement. However, others argue that it makes more sense, from both an economic and a pedagogical point of view, to simply "improve the way we use the time students already spend in school" (Karweit, 1985). According to this view, Title I funds are better spent improving the quality of the education that children receive during the time they currently spend in school, before reallocating scarce resources to support expanded learning opportunities.
Title I alone cannot compensate for the substantial educational deprivations associated with child poverty. Even at $8 billion, the program is small relative to the total cost of U.S. elementary and secondary education. Research demonstrates that, although Title I serves those students who are most in need of supplementary assistance, the nature of the help they receive is, by itself, insufficient to close the gap in academic achievement between them and their more advantaged classmates.
Research also shows us that, like an addition to an old house, supplementary funds and programs cannot be added to a weak foundation with any assurance of success. No matter how good the Title I staff or how hard these teachers work with relatively few hours of remedial instruction per week, they sometimes face serious obstacles in overcoming a poor school environment. There is sufficient evidence to indicate that the best way to improve the learning for disadvantaged children is to improve what happens throughout their entire school day, and that means improving the educational environment for all children, rather than targeting a few children at the margin. Only in this way can we hope to overcome the achievement gap faced by poor children at school entry. Further, school reform should be multi-dimensional research suggests that there is no "magic bullet" that will, by itself, raise the level of student achievement, especially for the lowest-performing children. Effective school reform programs should, therefore, invoke a variety of strategies selected on the basis of sound research, guided by the clear goal of improving student learning.
Probably the most compelling evidence we have in this regard is that teachers matter. It is increasingly clear that policymakers need to focus more on the operational core of education systems the classroom itself. Setting high standards, and expecting all children to learn a challenging curriculum, is doomed to failure without the teaching staff who can effectively bring all children to the desired point of learning. This means finding polices to attract and retain the best individuals, eliminating the practice of using aides and the most inexperienced teachers to teach the most challenged students (or to teach in the most impoverished schools), and ensuring that teachers have the content knowledge and teaching skills needed to meet the demands placed upon them. Strong professional development should be a key ingredient of any school reform strategy, as should increased opportunities for professional collaboration, particularly through the extension of teacher working time to cover at least part of the summer months to allow time to work on curriculum, instructional development, and other policy-setting activities.
School administrators must also have the capacity to meet the new challenges and demands facing them and the rest of society at the dawn of the 21st century. The movement to set high standards and expectations and to develop accountability systems that focus attention on teacher and student performance is certainly important. But while these activities create the will to improve our schools, they do not address the skill needed to achieve these ambitious goals. Support from the central administration will be essential in developing modern information and reporting systems, in providing support for professional development, and in creating accountability and reward systems that are aligned with educational objectives.
Although the evidence on standards-based reform is still fragmentary, the main idea behind this movement is in line with a stronger body of research showing the importance of curriculum and instruction to student learning. States must set clear, high standards, and districts and schools must be responsible for ensuring that all students are provided with the curriculum, teaching practices, and assistance they need to attain these standards. Of course, schools need meaningful ways to assess progress against those standards, but such assessments should be used to provide meaningful feedback to students, parents, teachers, and administrators, not just to keep score. The entire school must become a learning community in which all stakeholders are focused on the single goal of improving student learning, and in which data are used in a continuous way to monitor progress and adjust the course when necessary.
Technology can be an important tool for bringing about school improvement, but it should not be viewed as a substitute for good teaching. In fact, we know too little at this point about the most effective ways to use computers to support classroom instruction. There are additional concerns about the lack of high-quality content-based software and adequately trained teachers. In the most disadvantaged schools, where the human resources may be limited, computers may be able to serve as a "default strategy" for instruction, especially in certain subject areas where classroom teachers lack adequate content-area preparation. But, to be successful, such strategies will require investment in the development of high-quality content-based computer applications.
Smaller class size appears to be an important means of boosting student achievement during the early years of schooling, when children are learning fundamental skills and adjusting to new social settings. Although the evidence does not seem to justify broad investments in class size reductions for all students at all grade levels a strategy that could overwhelm other important uses of scarce resources there is ample support for more flexible, targeted class-size reduction initiatives. Strategies such as small-group instruction and cooperative learning may also be beneficial, but only if properly implemented and supported by appropriate instructional materials, and only if teachers are well trained in the necessary techniques.
There is strong and compelling evidence that disadvantaged children start school behind their classmates and never catch up from these early deficits. This suggests that increased investments in early education programs are clearly needed. But these programs must themselves be of high quality and must be tied to subsequent high-quality school instruction for the gains to be sustained. Other nonschool factors such as school/family/community partnerships, out-of-school programs, school accountability and incentive systems, and changes in local school governance also show promise, but suffer from a fragmentary research base. There is a critical need for knowledge in each of these areas that illuminates the linkages between schools' capabilities and increased student performance.
Finally, we must confront the fact that the state of educational research is glaringly poor. Although, there are hundreds of articles published every month, many by very talented scholars, most of what we "know" about education is, at best, based on weak research designs, and, at worst, on unsupported claims or anecdotal evidence. Of all the social policy areas, education is certainly the least well supported by sound research. This is not to say that we have learned nothing from the past 40 years of educational research, but "the amount and quality of systematic development and rigorous program evaluation remains limited" (Vinovskis, 1999). Although we may lack definitive answers about what works best, we know a great deal more about what are the right questions to ask, and where to look for effective solutions. The federal government and national organizations need to take a much greater leadership role in the generation of high-quality information concerning "what works" in education.
We especially need continuous experimentation to provide information that supports data-driven decision making in schools as learning communities. Only in this way can policymakers and schools make rational decisions about which steps to take to improve student learning. In particular, we have little, if any, information on cost-benefit tradeoffs. In a world where resources are limited, we need to know which school changes/reforms offer the greatest "bang for the buck." We also need to foster greater commitment among all stakeholders in the educational process if these reforms are to be given a fair test. Too often, schools and policymakers implement reforms that are unsupported by sound research, leading them to jump indiscriminately from one new fad to another. Because this has become common practice, it is hard to muster the necessary support and commitment to sustain any program, even those with substantial potential for success. Given these circumstances, teachers can hardly be blamed for taking the approach that "this too shall pass!" And, even in those cases where these obstacles are overcome, it is often on small demonstration projects, and rarely extends to more broadly-based programs.
An area of particular need for further research is the movement toward whole-school reform, a strategy encouraged by a recent increase in the availability of federal funding to support such programs. Despite developers' claims of effectiveness, we know far less than we think (or should know) about the relative strengths of many of these new reform models. Schools are spending enormous amounts of their precious resources on programs that promise to raise student achievement without clearly understanding whether the interventions they adopt have the potential to work in their school, or what it takes to effectively implement these broad reform strategies. Too many schools get whipsawed as they seek the latest cure-all, abandoning old ideas when they fail to yield promised academic gains in one or two years. Although schools should be encouraged to experiment with new ideas, the choice of a strategy must be based on reliable information derived from independent and rigorous impact evaluations. Claims of the effectiveness of particular interventions should not be based on simple test-score comparisons whether related to national norms or the performance of students in purportedly similar schools and certainly should not be based on anecdotal evidence of success. Far too much is at stake for Americas schoolchildren to waste limited school resources on unproven reforms, or reforms that are ill suited to local conditions.
1 Title I was renamed "Chapter I" as part of the 1981 reauthorization but then regained its original name in 1994. The term "Title I" has been used throughout this paper, except in discussing research that applies specifically to the version of the program that existed during that 13-year period.
2 Throughout this paper, "high-poverty" schools are defined as those in which 75 percent or more of the students are eligible for free or reduced-price school meals; alternatively, "low-poverty" schools are defined as those in which 25 percent or fewer students are eligible for subsidized school meals.
3 For example, Ferguson and Brown (1998) report that individuals scoring lower on state exams in Texas are more likely to teach in districts with high proportions of black students.
4 This section is adapted from Waymack and Drury (1999).
5 Other comprehensive school reform models are described in Herman, Aladjem, McMahon, Masem, Mulligan, O'Malley, Quinones, Reeve, and Woodruff (1999) and U.S. Department of Education (1998b).
6 Another wave of systemic reform - beyond the scope of this report - aims to replace traditional governance structures with market mechanisms, thereby shifting power from public governmental agencies to parents (Chubb & Moe, 1990; Clune & White, 1990). Advocates of such "school choice" reforms believe that parents, having the ability to "vote with their feet," will provide the necessary incentives to drive school improvement, thus eliminating the need for complex government accountability systems. Several models of choice have been proposed and implemented, including vouchers, magnet schools, within-district choice, charter schools, and privatization. The models differ operationally, but all seek to use parental choice to leverage school reform using a private-sector model of producers (schools) and consumers (parents). Advocates further contend that greater choice would result in a more efficient use of resources and increased student achievement. Unfortunately, studies evaluating choice models have focused almost exclusively on this latter issue and have failed to address the much broader issue concerning the role of choice in reforming the existing system. Even within the narrow range of existing studies, there is little evidence to suggest that choice is associated with higher student achievement. The two major tests of vouchers, in Cleveland and Milwaukee, have revealed only marginal differences in performance between voucher recipients attending private schools and their counterparts in the public schools. Other evidence on the performance of charter schools in California (Walsh, 1998) suggests that these new schools are not being held accountable for student performance. And, finally, the leading examples of privatization - Minneapolis, Minnesota (Public Strategies, Inc.), Wilkinsburg, Pennsylvania (Alternative Public Schools, Inc.), Baltimore, Maryland and Hartford, Connecticut (Education Alternatives, Inc.), and the Edison Project, which now manages nearly 50 schools - have yet to be rigorously evaluated.
7 The evidence on the effectiveness of systemic reforms is limited. However, there are some indications of a possible impact on student achievement, often linked to high-quality professional development. Most recently, Cohen and Hill (1998), in a study of standards-based reform in California, reported that: (1) providing teachers with opportunities to learn about standards-based reform increases their knowledge; (2) when these opportunities are tied to the curriculum that students are expected to learn, teachers change their teaching practice; and (3) when student assessments are consistent with teacher training and the curriculum, student achievement scores increase. Studies of the California eighth-grade writing assessment program also appear to indicate positive improvements, both in what teachers do in their classrooms and how students perform when adequate investment is made in teacher capacity building and professional development (Herman, 1997). Similar, albeit modest, results were reported by Zucker, et al. (1998) in their evaluation of the State Systemic Initiatives (SSI) sponsored by the National Science Foundation, but the effects were uneven across the different SSI locations (Knapp, 1997). Finally, research by Grissmer and Flanagan (1998), suggests that the recent NAEP gains in North Carolina and Texas between 1990 and 1997 may be linked to several changes introduced by both states, including: the alignment of standards, curriculum, and assessment; the existence of school accountability systems; and support from the business community in making systemwide changes. Taken together, these findings demonstrate that standards-based reform can yield higher performance on the material that students are expected to learn, but that the role of teachers and their training are critical components of the process by which such gains are realized.