Choose a Section National Center Home About Us -Fact Sheet -Board of Directors -Center Staff Center News Reports & Papers National CrossTalk -Current Issue -Back Issues -Download CrossTalk -Subscribe -About CrossTalk Subscribe Contact the Center

1 of 3 Stories

Standards for the Standards Movement Do high school exit exams measure up? By Rebecca Zwick AN EDUCATIONAL STANDARDS FRENZY has overtaken our country during the past decade. Forty-eight states have implemented some kind of “standards-based assessment,” and half the states have high school graduation tests in place or in the works. This is all part of an educational reform movement that calls for the establishment of rigorous academic requirements and the use of student testing to monitor progress. Critics charge that the resulting assessment blitz has caused a narrowing of the curriculum as a result of “teaching to the test,” produced a frantic atmosphere that encourages cheating, and created unfair obstacles for ethnic minorities and special-needs students. Unfortunately, professional testing standards rarely are invoked in the debate about the educational standards movement. A notable ex-ception is a recent evaluation report on Cali-fornia’s brand-new high school graduation test, the centerpiece of Governor Gray Davis’ am-bitious plan to improve K–12 education. The exam is scheduled to be administered to ninth graders on a voluntary basis next spring, with mandatory administration to tenth graders to follow in 2002. Beginning in 2004, students who haven’t passed the test will not get diplomas, if all goes as planned. Evaluators now have recommended postponing implementation by one to two years to allow “time to develop an assessment that fully meets professional and legal standards.” Indeed, high school exit exams are on shaky ground according to standards developed by the testing profession, especially when the initiation of high-stakes assessment outpaces curriculum reform. Let’s see how these exams stack up against four key guidelines from the Talmud of testing standards—the newly updated Standards for Educational and Psychological Testing published by the American Educational Research Association, the American Psy-chological Association and the National Council on Measurement in Education. 1) In educational settings, a decision…that will have major impact on a student should not be made on the basis of a single test score. Do high school graduation exams violate this guideline? In a court challenge to the Texas Assessment of Academic Skills, the judge concluded that test scores alone were not being used as the basis for graduation decisions because students had several chances to pass, and grades and attendance also were taken into consideration. But in Texas, as in most states that use exit exams, test scores are indeed the determining factor in graduation decisions, as a 1999 National Research Council report on high-stakes testing points out. It is not permissible, for example, to compensate for low test scores by having exceptionally high grades or taking extra classes. According to the NRC report, test results “should be buttressed by other relevant information…such as grades, teacher recommendations and extenuating circumstances.” A strict interpretation would say that most high school exit examination programs are in violation of this professional testing standard. 2) When test results substantially contribute to making decisions about…graduation, there should be evidence that the test…covers only the…content and skills that students have had an opportunity to learn. In the field test of California’s graduation test, the average tenth grader correctly answered 59 percent of the English language arts questions and 47 percent of the mathematics questions. No decision has been made about the minimum score required to pass the test, but the evaluators estimated that, if students were required to answer 70 percent of the questions correctly (as in Texas), about two-thirds of tenth graders would fail the English section and 85 percent would flunk the math section. They also noted that “minority students, special needs students and English language learners are particularly at risk of failing.” Why the high failure rates? For the majority of the test questions, said the evaluators, “at least one-fourth of the tenth grade students had not received instruction that would allow them to answer…correctly.” We might assume that if reasonable curriculum standards are set, acceptable pass rates will follow. But the crucial steps in between can take years: The curriculum must be brought in line with the standards, teachers must be trained in the new curriculum, and student instruction must be modified accordingly. A basic problem with the standards movement, said Sandra Feldman, president of the American Federation of Teachers, is that “in some states and districts, tests are just about the only part of the standards process that has been put into place.” 3) Test takers [should] receive comparable and equitable treatment during all phases of the testing or assessment process. States embarking on a high school exit examination program face a mountain of unresolved equity issues. For example, according to a recent report by the President’s Advisory Commission on Educational Excellence for Hispanic Americans, the testing policies of many states do not treat English language learners in an equitable way. In particular, the commission criticized the practice of translating tests from English to another language and then simply assuming the translated versions are equivalent to the original tests. Similarly complex issues arise in testing disabled students: Giving an unmodified test to, say, a student with severely impaired vision is obviously unfair. But would reading the test aloud to the student yield an assessment equivalent to the usual version? Without a lot of research, it is impossible to say. 4) In educational settings, test users should ensure that any test preparation activities and materials provided to students will not adversely affect the validity of test score inferences. At least ten states have reported cheating on standardized tests during the last year— and that’s just the cheating done by principals and teachers! Seven teachers in the Sacramento area photocopied the Stanford Achievement Test-ninth edition, which is used to rate the state’s schools, and then taught its content. An Arizona school employee copied a draft version of the state’s graduation test and gave it to consultants who had been hired to prepare teachers for the test. And in Massachusetts, a teacher sent an e-mail message to colleagues at other schools containing questions from a statewide test. Obviously, this is not what backers of standards-based assessment had in mind. Widespread cheating by school personnel could render the assessments useless and produce a generation of “ethically challenged” students as well. What can be done? How can we uphold testing standards without abandoning educational standards? One approach is to build in an initial “limited-stakes” phase in which exit exams are used solely for monitoring the progress of districts, schools and key groups of students. Because this original assessment would be for accountability purposes only, we wouldn’t have to test each student—we could just take a random sample. And because no scores would be assigned to individuals, we wouldn’t need to give every student the same test questions. Later, when the needed curriculum changes had been fully implemented, the high-stakes phase, in which test scores would affect graduation decisions, would begin. This limited-stakes phase-in plan isn’t pie in the sky; it can be accomplished using a well-established data collection method called “multiple matrix sampling.” As an example of how it works, let’s say we’re interested in measuring the math skills of tenth graders from the Paradise School District. To maximize the breadth of the assessment, we could create several distinct sets of questions, and give each of these test forms to a portion of our random sample, making sure each form is short enough to be completed within a single class period. Assuming we sampled students and assigned test forms correctly, we could legitimately use the test results to draw conclusions about how well the district’s tenth graders had mastered a wide range of math skills. There is no reason that students’ names would need to be associated with their test papers—identification codes could be used for tracking purposes. For California, this method of assessment would be déjà vu. This very approach was used by the now-defunct California Assessment Program, described 20 years ago as “the largest and most highly developed of the state programs.” Granted, this strategy has its downsides. It would defer, rather than eliminate some difficult decisions. And it would require that steps be taken to assure that students were motivated to do well during the limited-stakes phase-in. But, by providing for the monitoring of student progress without imposing negative student consequences, this approach could buy us quite a lot. First of all, it would allow unhurried decisions to be made about crucial issues: How can test scores be combined with non-test information in determining who graduates? What testing accommodations are appropriate for English language learners and disabled students? What kind of help should be provided to students who fail the test? In addition, the limited-stakes phase-in period could minimize the incentive to cheat, reduce the investment of classroom time and money, and help to tame the emerging backlash against the proliferation of standardized tests. Most important, it would bring the assessment system into compliance with the opportunity-to-learn standard, which is central to educational improvement efforts: Diplomas would not be at stake until all pieces of the reform process were in place and all students had been given the chance to learn the required material.

E-Mail this link to a friend. Enter your friend's e-mail address:

PREVIOUS STORY | FRONT PAGE | NEXT STORY

Top © 1998 The National Center for Public Policy and Higher Education