10/14/03 -- Last fall, a mistake in the Massachusetts eighth-grade history test forced state officials to review the test results of 75,000 students -- and boost the scores of 14,000. Officials also gave a passing grade to nearly 700 students who'd been told they'd failed the test.
This summer, citing flaws in the Regents Math A exam that led to an unusually high failure rate, New York state officials threw out the test results.
Such mistakes would have been cause for embarrassment -- but little else -- only a few years ago. In this age of high-stakes testing, however, a single mistake is enough to deny a high school student a diploma and prevent a school from meeting academic standards under the No Child Left Behind (NCLB) Act.
"Tests are the linchpin of this whole accountability movement," says Jack Jennings, director of the Washington, D.C.-based Center on Education Policy. "So much is riding on these tests, so that if a mistake is made, it has big consequences."
Given the scope and complexity of today's testing programs, some mistakes are almost inevitable, say testing experts. The proliferation of testing in recent years also is testing the ability of state departments of education and rapidly expanding commercial testing firms to provide tests faster, cheaper, and more accurately than before.
Signs of this stress are easy to find:
• In Georgia, a series of state tests was suspended this year after education officials discovered that 270 test questions were available for review on an Internet site for students, parents, and teachers. Last year, the test results of 340,000 students were set aside because of scoring problems.
• In Nevada, a mistake in administering statewide exams threw off the scores of nearly 31,000 high school students in 2002.
• An error in calculating the passing score on Virginia's writing exams last year caused 5,625 students to mistakenly fail and another 7,702 to receive lower scores than they deserved.
Such mistakes are just the tip of the iceberg, according to a report by the National Board for Educational Testing and Public Policy, based at Boston College's school of education. Of the 103 errors and disputes over testing results identified since 1976, more than two-thirds occurred in the past four years.
"Human error can be, and often is, present in all phases of the testing process," concludes Errors in Standardized Tests: A Systemic Problem. "Error can creep into the development of items. It can be made in the setting of a passing score. It can occur in the establishment of norming groups, and it is found sometimes in the scoring of questions."
The requirements of NCLB are challenging all companies to keep up with market demand, says Stuart Kahl, president of Measured Progress, a test-publishing firm in Dover, N.H. But, he adds, the overall quality of tests is rising, and the likelihood of future errors will decline as the testing industry catches up with demand.
State officials say they are doing what they can to ensure accurate tests. A system of review boards and committees has been expanded to screen tests before they are distributed.
In Ohio, officials agreed to conduct an extra round of pilot testing of the state's high school exit exam, after previous tests found graduate failure rates were high. In response to complaints that this summer's math exams were too difficult, New York state officials have launched an investigation into problems with the exam.
Meanwhile, the testing industry also is being put on notice. In Georgia, state officials have included new penalties for testing errors in the state's latest contract with testing firms. Earlier this year, Nevada slapped fines of more than $400,000 on its testing service.