The release of the latest results of the California High School Exit Exam last fall prompted one Sacramento Bee columnist to ask who was flunking-the students, the schools or the state. Only 32 percent of the students who took the test in 2002 passed the math section, and 54 percent passed the English language arts portion. Most disturbing were the math pass rates for students who were African American (20 percent), Latino (22 percent), socioeconomically disadvantaged (22 percent), or English learners (limited-English-proficiency students)(18 percent).
Will these dismal results lead to immediate instructional innovations or to an infusion of resources targeted to these groups? That seems unlikely. Instead, many are predicting that the State Board of Education will simply postpone the requirement that students must pass the test to graduate. The state board has already taken actions to minimize the effects of the test. In December 2000, the board voted to shorten the exit exam and to eliminate some difficult algebra questions, after a field test yielded grim results. And in June 2001, the board set the minimum passing scores lower than recommended by a panel of teachers, parents and community members.
All of which leads to a fundamental question: Why give high school exit examinations in the first place?
High school exit exams have had a resurgence in popularity as part of the "accountability" movement that has swept the country in the last decade. Twenty-four states now require the administration of these exams, and 18 are already using them to regulate graduation. Yet even as exit exams are becoming more prevalent, their role is growing muddier as the policies governing them are recast by state boards of education, politicians, courts and special-interest groups.
Are the exams meant to serve primarily as a quality control mechanism for high schools, or are they intended to determine how well each student's performance stacks up relative to a common set of standards? Or, as a third possibility, is the goal to measure every student's academic performance, even if this requires modifying the test-and implicitly, the standards-for some individuals? All these purposes are legitimate, but each has a different set of implications for test design, administration and interpretation. And without some clear thinking about what exit exams are intended to accomplish, states may well end up with tests that are not ideal for any of these purposes.
Let's examine each of these three possible objectives in turn.
1) Provide school-level quality control.
According to an August 2002 report from the Center on Education Policy, an independent organization based in Washington, DC, a central goal of high school exit exams is "to spur a general improvement of public education." The tests are viewed as "a form of quality assurance, especially when they are tied to challenging state standards for what students should know and be able to do." But, surprising as it may seem, high school exit exams aren't necessarily tied to high-school-level standards. And if the goal is school-level quality control, the way these tests are being administered is extremely inefficient.
First of all, consider the test content. Of the 24 states now requiring exit exams, ten give tests that focus on content below the high school level, according to the Center on Education Policy. And sometimes, the level of the test varies by subject. On California's exit exam, for example, the language arts portion assesses state standards through the tenth-grade level, but the math portion only covers standards for grades six and seven, plus Algebra 1, according to the California Department of Education.
In addition, the degree to which exit exams are truly standards-based is called into question by the actions of states that allow pass rates to affect the test content or the minimum passing scores. In a standards-based test, the necessity of mastering certain material, not the success rate, should determine what score is needed to pass.
And if the primary intention is to evaluate the quality of education at the school level and to ensure that schools are teaching the material included in state standards, then it would be far more efficient to arrange for anonymous random samples of students to take the exam than to test all students. (See "Standards for the Standards Movement: Do high school exit exams measure up?" National CrossTalk, Fall 2000.) Savings would be large, both in terms of dollars and classroom hours.
2) Provide certification that individual students meet specific standards.
Another goal of high school exit exams, according to the Center on Education Policy report, is "to make a diploma 'mean something'-namely, that its holder has the knowledge and skills needed to do well in a job, college, and other aspects of daily life." Under this interpretation, administering a high school graduation test provides for a form of certification. Those who have passed the test can be assumed to have certain specific skills in language, math and sometimes other areas. But tests that focus on sub-high-school content, as the exit exams in ten states do, certainly can't provide an endorsement that students have career or college skills. To serve that function, states would need to develop a test blueprint that spelled out the proficiencies needed for college and career success, and to design a test based on this framework.
Furthermore, if exit exams are to be uniformly regarded as certification tests, they must assess the same skills for all test-takers. And, as we explore in the next section, some of the policies that have evolved for administering these tests to special populations aren't consistent with this criterion.
3) Assess all students, incorporating test modifications as needed.
The Center on Education Policy report says that by 2008, 22 of the 24 states requiring exit exams expect to offer accommodations for students with disabilities, nine will offer alternative assessments, and five will offer exemptions from the exam requirement.
The California Department of Education lists on its website the various alterations of exit exam procedures that are to be permitted for students with appropriately documented disabilities. These alterations are divided into two categories. "Accommodations," which are assumed not to change the test in any fundamental way, are usually uncontroversial. They include the provision of large-print or Braille versions of the test, and audio or oral presentation of the math section. "Modifications," which are acknowledged to change what is being measured, include the "audio or oral presentation of the English language arts part of the test," the use of spellcheckers on the writing items, and the use of calculators on the math section.
In the case of "modified" tests, the State Board of Education has the power to decide whether a passing score warrants a high school diploma. According to an evaluation report prepared by the Human Resources Research Organization, about two percent of the students taking the math portion of the California High School Exit Exam in 2002-about 4,300 students-were permitted to use calculators, including about 900 students who had no documented disability whatever.
The policy on accommodations and modifications in California stems from a lawsuit that was filed in 2001 on behalf of special education students. As a result of the suit, U.S. District Judge Charles R. Breyer issued a preliminary injunction in February 2002 that barred administration of California's exit exam under existing polices. To proceed without a policy change, he said, would "violate rights guaranteed to learning disabled students under federal law" to "participate meaningfully in the assessment." Exclusion from the exam, along with the potential denial of a diploma could cause "harm to a student's dignity and educational prospects."
According to Judge Breyer, "the fact that some of these students are incapable of mastering the content of the exam is of no importance; they are still entitled to a valid assessment of their capabilities." The judge's decision stated that learning-disabled students must be granted the testing accommodations and modifications listed in their existing individual educational plans ("IEP's" or "504 Plans").
So is it a good thing to allow calculators and spellcheckers for students with learning disabilities, even though these devices fundamentally alter the exam? Certainly, a strong case can be made for doing so. First, the chance to be included in the assessment preserves the dignity of test-takers with disabilities, as Judge Breyer asserted. But it is more than a question of dignity. Inclusion in the assessment also allows these students, along with their parents and teachers, to obtain information about their academic performance, regardless of whether their scores are comparable to those of other test-takers and regardless of whether they are likely to pass the test. And if they take the test on more than one occasion under the same conditions, the test results can also provide information about their progress over time.
But if tests are to be individualized in ways that clearly alter the nature of the skills that are being assessed, test score users such as colleges and employers should be made aware of the conditions under which the test was administered. If a student used a calculator to pass the test, the score user shouldn't be misled into regarding the passing score as a certification of the ability to do the math problems without a calculator.
And those monitoring school, district or state progress must also be aware of any substantial alterations of testing procedure. Scores resulting from modified administrations should not be aggregated with scores obtained from standard administrations. To allow the scores from modified administrations to stand without providing information about the testing conditions could render the entire testing effort meaningless, and could create incentives for students to be labeled as having a disability. Illegitimate claims of learning disabilities are particularly problematic because, as a 1997 National Academy of Sciences report noted, one district's "learning disabled" student is another district's "low achiever."
An advantage of notifying all parties of the assessment conditions is that it opens up the possibility of further increasing the flexibility of the testing process. As long as test users are informed of testing conditions, and scores from modified administrations are not aggregated with other scores in reporting results, the testing process can be individualized to meet the needs of each student.
Attempts to shoehorn children with vastly different needs into the same modified testing session can then be abandoned. The price of this increased individualization is that for some students, the test will not serve a certification function because passage of the test will not have the same meaning as it does for the majority.
As states are rapidly discovering, the development of a successful graduation testing program is a complex venture that is fraught with technical and political pitfalls, and is subject to the demands of many competing constituencies. High school exit exams can't simultaneously be unobtrusive school-monitoring mechanisms, certification tests linked to a fixed body of knowledge and skills, and individualized assessment devices for gathering achievement information on all students.
The purpose of a test has implications for its design and administration, and for the interpretation and use of scores. Attempting to serve all possible purposes may result in a test that is optimal for none of them. Without further thought about the goals of these proliferating tests, states could expend thousands of hours and millions of dollars without improving high school education.
Rebecca Zwick is a professor at the Gevirtz Graduate School of Education at the University of California, Santa Barbara. She is the author of "Fair Game? The Use of Standardized Admissions Tests in Higher Education." She also serves as chair of the College Board's SAT committee, but the views she expresses here are her own.