Newsroom

Database: National Report Card Woes

The Full Picture: NAEP Over Time

NAEP scores are down—is it time to panic?

Patte Barth

Scores on the 2015 national Assessment of Education Progress (NAEP) declined for the first time in two decades. Of course, the pundits and think tank wonks are busily parsing the results for signs of what’s gone wrong in American education.

Some see proof that NCLB and test-based accountability are a disaster; others lay the blame on NCLB waivers for weakening the law’s provisions. Many point to the growth in child poverty; some say poverty alone won’t explain it. And, predictably, the Common Core standards are taking the most hits.

Here’s what we know. Between 2013 and 2015, NAEP scores fell by statistically significant margins for fourth- and eighth-grade students in math, and for eighth-graders in reading. The drop puts an end to a hitting streak that has produced steadily improving scores, especially in math, since the early 1990s. The abrupt reversal is what has so many wringing their hands.

Before we rush to judgment about why this happened, however, we should remind ourselves of a fundamental principle from Statistics 101: One year’s data does not make a trend. That is not to say, however, that we should collectively shrug at the drop in scores.

As the Center for Public Education’s (CPE) Jim Hull wrote, “There is simply no way to sugarcoat [the results].” But it is too soon to assign blame, or for that matter, to even conclude that the decline represents a real setback. At the same time, it is something that we should pay attention to in case it portends something worse to come.

So let’s look at the possible explanations being offered:

Test-Based Accountability

Are, as one popular blogger charged, Race to the Top, NCLB, and so-called “corporate reform” responsible for forcing the dumbing down of instruction through an overemphasis on standardized tests?

The problem with this argument is that NCLB accountability has been in place for 12 years, during which time students’ math performance rose with each NAEP administration except for the most recent. Between 2003 and 2013, the overall eighth-grade math performance increased by 7 points on the NAEP scale.

Over that same period, black eighth-graders improved by 8 points, Hispanics by 11 points and low-income students by 9. While all of these groups experienced a two- to three-point decline between 2013 and 2015, it’s hard to make the case that NCLB suddenly caused a drop in scores after being around for a decade while math performance improved.

Which leads us to:

Poverty

According to the Food Research and Action Center, 3.7 million more children have been added to the rolls of federally subsidized school lunch recipients since 2007, the year right before the recession. The relationship between poverty and academic performance is well documented, prompting many observers to speculate that the effect of having more low-income students in schools is now becoming evident in falling NAEP scores.

This is a compelling argument. Low-income students tend to score lower than their more affluent peers. Therefore, when the share of poor schoolchildren increases, it could drag the overall score down.

A clear case would be made for this explanation if there were no change in the upward trajectory of the performance of middle-class students, who now make up a smaller share of the school population. Yet that’s not the pattern we see. The scores for the group of students who are eligible for subsidized meals tracks below non-eligible students, but the path is the same. Both groups posted gains every year they were tested until 2013. In 2015, both groups’ scores dipped.

Researchers Matthew Chingos and Kristin Blagg have developed a statistical method for adjusting NAEP scores for student demographics to predict state NAEP performance. They took a look into the question of the recent two-year drop. Writing in the October 2015 Education Next, they concluded that “the decreases in eighth-grade math and reading scores are too substantial to be blamed on changes in the characteristics of students taking the tests.”

So that leaves:

The Common Core

Over 40 states have spent the last four years implementing new Common Core standards. With so many states pursuing a change of this magnitude at the same time, it’s not unreasonable to ask whether the Common Core and NAEP are working at cross purposes.

The American Institutes for Research maintains an independent NAEP Validity Studies Panel for the purpose of commissioning and discussing “research addressing validity considerations for NAEP.” The panel has sponsored a series of papers examining the alignment between NAEP and the Common Core.

The most recent paper was published this fall. The author Phil Daro and his team compared NAEP test items to the Common Core and recorded the number of items that corresponded to a relevant grade-level standard in math.

No standardized test will cover 100 percent of any set of standards, and the analysts did not expect to find full agreement. Nonetheless, they documented a 79 percent alignment in fourth grade-math, which they called “reasonable,” and 87 percent alignment at eighth-grade, or “strong.”

Another study—this one done for the Brookings Institution—looked specifically at the question of whether Common Core was associated with the 2015 drop in NAEP scores. Researcher Thomas Kane examined the NAEP scores of states that administered either PARCC or Smarter Balanced tests, the Common Core assessments developed with federal funds.

Kane then compared their performance to that of other states. He found that the PARCC/SBAC states did in fact score lower than others but the difference was less than 1 point on the NAEP scale. According to Kane, because the differences represented “less than one-third of the absolute 2013-2015 decline,” it cannot explain the drop in NAEP math.

I did my own back-of-the-envelope calculations and compared the change in NAEP scores by state between 2009 and 2015. I chose 2009 as my baseline because that was the last NAEP administration before states adopted the Common Core in 2010 and so would capture the time when standards began to affect classroom practice.

I found that over the five-year period, the eighth-grade math performance dropped in 26 Common Core states; increased in 16 states; and was unchanged in four. In the states that did not adopt the Common Core, two had lower scores, two increased, and one experienced no change.

My informal analysis of fourth-grade math showed a decrease in 18 Common Core states; an increase in 20; and no change in eight. In short, there was no discernable pattern related to Common Core adoption.

So we can say the decline in NAEP scores is probably not related to NCLB or test-based accountability. We can also say that there is no evidence at this time to conclude that either childhood poverty or the Common Core are responsible, although those are still issues that bear watching. There is one other possible explanation: The 2013-2015 drop is simply an anomaly. If so, we will know if scores bounce up again in 2017.

I presented this information recently in Colorado. A school board member in attendance put it best, I think: “There is no smoking gun here.” Should we panic? No. Should we be watchful and closely monitor progress in our own states? Absolutely. That is just good, solid practice under any circumstances.


Patte Barth (pbarth@nsba.org) is the director of NSBA’s Center for Public Education.

Go to top