Measuring Up's Six Performance Categories:
A Closer Look

The Measuring Up report card series presents a broad picture of how well each state's higher education system is performing in six essential categories. Before they can act on the information the report cards provide, however, business leaders, educators, and policy makers need to gain a more detailed understanding of each performance category.They must examine the social and economic context that lends each category's data their fullest significance, as well as the data's implications for future workforce development trends.

In some instances, a fuller understanding of this context and these implications might compel the business, education, and policy communities to address gaps in available data. In others, it will help business leaders to use current data in constructing more effective improvement initiatives.

In the interest of fostering such an understanding, the following pages offer a brief discussion of each performance category. While this discussion is by no means exhaustive, it should offer business a preliminary framework for further exploration.

1. Preparation: How adequately are students in each state being prepared for education or training beyond high school?

The Impact of K-12 Preparation on Higher Education

The preparation category measures the performance of each state's K-12 system. The National Center has based each state's preparation grade on a combination of high school completion rates, K-12 course-taking data, and K-12 student achievement data. Such data have a significant impact on every state's higher education system. While high school completion is of course a prerequisite for enrolling in the vast majority of postsecondary institutions, extensive research suggests that success in rigorous high school courses, as well as demonstrated proficiency in national assessments and college entrance examinations, are solid predictors of persistence and success in higher education.

The high proportion of postsecondary students who take remedial courses attests to the failure of the K-12 system to equip many students for the rigors of a college education. According to the National Center for Education Statistics (NCES), for example, a full 30% of first-time freshmen in 1995 required remediation in mathematics, reading, or writing.14 Although such remediation is comparatively inexpensive, it does cost many students time, often slowing or even stalling their progress toward important certificates or degrees.

The Responsibility of Higher Education for K-12 Preparation

In recent years, a growing number of educators, business leaders, and policy makers have insisted that higher education share some of the blame for ill-prepared students. They argue that colleges and universities have a responsibility to help elementary and secondary schools prepare students for postsecondary study. For example, the higher education establishment can play a much larger role in defining what students should know and be able to do to succeed in college.

Higher education can also do more to enhance teacher quality. America's colleges and universities have come under mounting scrutiny for their record in teacher education. According to Vartan Gregorian, president of the Carnegie Corporation of New York, "They demean the teaching profession by not providing prospective teachers with the best knowledge of their individual fields, the latest theories of pedagogy, strong skills in technology, considerable classroom experience, and professional mentors."15

Research commonly cites poor teacher preparation as a significant cause of unacceptably high attrition rates among teachers. If these rates persist, we will likely be unable to meet the projected demand for over two million new public school teachers over the coming decade.16 In light of the looming teacher shortage, the need for colleges and universities to offer high-quality teacher preparation programs has assumed particular importance.

2. Participation: Do state residents have sufficient opportunities to enroll in education or training beyond high school?

The Need to Accommodate a Wider Diversity of Educational Needs

The participation category measures the proportion in each state of both young and working-age adults who take part in postsecondary education and training. As the demand for highly educated workers continues to expand in all states, participation rates will become an increasingly important predictor of a state's ability to attract and retain businesses.

Healthy participation rates depend on the ability of higher education to adapt to increasingly complex educational needs. By assessing participation rates among working-age adults as well as young adults, Measuring Up draws attention to the emerging needs of one particularly significant group: lifelong learners. Confronted with constantly shifting knowledge and skill requirements, older working adults are attending postsecondary institutions in unheard-of numbers. Driven in part by this growth in lifelong learning, the average age of undergraduates rose to 26 by the year 2000, and the proportion of undergraduates ages 24 or older grew to 43%.17 A considerable number of college graduates are returning to postsecondary institutions to update their education. According to the American Association of Community Colleges, 28% of students who are enrolled in noncredit courses at community colleges already hold a bachelor's degree or higher.18

These data may well under-represent the magnitude of adult learning in America, much of which takes place in courses or institutions for which the Department of Education keeps no statistics. Results from the National Household Education Survey suggest that as many as 90 million adults participated in adult education in 1999, and that almost half of those were taking work-related courses. Only one fifth of the 90 million adult learners were enrolled in programs leading to a formal postsecondary credential.19 Whether or not these figures are fully reliable, they indicate that both the typical student and the typical course of study have changed dramatically over the past decade.

Many forms of postsecondary learning that do not appear in official statistics can expand educational opportunity and thus contribute to the nation's intellectual capital. Every state's higher education system should therefore offer people a far more varied educational menu than ever before.

The Need for "User-Friendly" Education

Over the long term, states that promote more "user-friendly" educational options will encourage the broadest participation in higher education. Distance learning technologies offer one means of accommodating students who have been traditionally underserved by colleges and universities. By making education available anywhere and anytime, they can benefit people who cannot travel to physical classrooms, or whose work responsibilities prevent them from attending classes on a fixed schedule. A "one-size-fits-all" approach to higher education will stifle postsecondary participation in an era of increasingly diverse educational needs.

3. Affordability: How affordable is higher education for students and their families?

A Crisis in Affordability

The capacity to expand participation in higher education hinges on the ability to make it affordable for even the lowest-income Americans. Rather than merely measuring state levels of financial aid, therefore, any assessment of affordability must also consider the actual cost of higher education as a share of family income across all socio-economic groups. In assigning grades for affordability, the report cards measure the proportion of family earnings required to pay for expenses at two- and four-year colleges, the average annual loan amount carried by undergraduates, and the amount of state financial aid targeted to low-income families.

The national picture of college affordability is grim. While average college tuition, measured in constant dollars, has risen almost 110% since 1981, the median family income has risen by only 27% over the same period.20 Tuition increases have had the most dramatic effects on lower-income students, who must devote an enormous, and growing, share of family income to college expenses. Financial aid has not kept pace with these costs. Barring significant changes, moreover, enrollment increases projected for the coming decade promise only to exacerbate these problems.

The distribution of grades in Measuring Up 2002 reinforces this bleak picture. While only four states earned A's or B's in affordability, 33 earned D's and F's. According to the report card, "Few states offer both low-priced colleges and significant amounts of financial aid targeted to low-income students and families."21 Even in the best-performing states, the average annual undergraduate debt burden approached $3,000.

Several factors contribute to this situation. Since the early 1980s, average college costs at private and public four-year colleges have risen at a rate more than double that of inflation.22 Tuition increases become especially steep during economic downturns, when college and university administrators attempt to compensate for disproportionately large cuts in state higher education budgets. In addition, the recent shift from need-based grants to merit-based aid and tax incentives does not benefit the lowest-income students. Policies to improve affordability must address all of these factors.

Nontraditional Students Left Out in the Cold

Federal and state financial aid systems do not support "user-friendly" postsecondary education. Originally designed for full-time students earning four-year degrees in bricks-and-mortar facilities, today's financial aid systems do not serve the growing proportion of postsecondary students who follow alternative paths toward a degree or who receive their education through alternative delivery systems. A report on financial aid and working adults released by FutureWorks in April 2002 has determined that less than 30% of low-income working parents who enroll less than half-time in Title IV postsecondary programs receive financial aid from federal, state, or institutional sources. In addition, many adult students seek education and training from institutions that do not fall under Title IV of the Higher Education Act. Since these institutions cannot distribute most forms of state and federal student aid, the actual proportion of adult learners who receive no aid is likely much higher.23

Because many students enrolled in part-time or distance education programs remain ineligible for state or federal aid, this situation is unlikely to improve any time soon. In fact, current student aid structures might actually hamper the creation of postsecondary programs that serve the needs of nontraditional students. "Educational institutions have had little incentive to create and offer the more accessible and modularized programs that working adults need," the FutureWorks report concludes.24

Over the long run, federal and state financial aid strategies that do not acknowledge the mounting need for such educational alternatives will ultimately leave millions of students behind while undermining workforce quality.

4. Completion: Do students make progress toward and complete their certificates and degrees in a timely manner?

The Importance of Time-to-Degree

The completion category measures the percentage of first-year community college and four-year college students returning for their second year; the percentage of first-time, full-time students completing a bachelor's degree in a timely manner; and the number of certificates, degrees, and diplomas awarded per 100 graduate students. Employers should be concerned not only with the numbers of postsecondary credentials conferred, but also with the time it takes students to complete these credentials. Given by the increasing pace of new labor market demands, the need for many students to complete their degrees in a timely fashion will grow stronger, especially in times of greater prosperity. In high-demand fields such as teaching and engineering, excessive time-to-degree can worsen the labor shortages that threaten this prosperity.

The Shortage of High-Demand Degrees

Employers should also look beyond the data in Measuring Up to examine not only the number of credentials awarded but also the kinds of credentials students earn. Long-term trends reveal that the supply of students educated in certain fields is often surprisingly out of step with demand for graduates in those fields. Between 1985 and 2000, for example, the number of engineering bachelor's degrees awarded nationwide dropped by 24%; that of mathematics bachelor's degrees fell by a staggering 30%.25 These declines persisted at a time when demand for technically trained workers approached historical highs.

Fighting a Culture of Exclusion

An academic "culture of exclusion" can discourage students from completing postsecondary programs, often exacerbating skills shortages in areas of especially high demand. In science and engineering programs, for example, student attrition rates frequently approach 50%. University policies and practices often contribute to these high levels of attrition. Some academic departments set quotas that prescribe high failure rates, and many reward top-tier professors by giving them small groups of elite students to teach.

Such tendencies reflect a broader philosophy that negatively influences completion rates. A long-standing desire to identify and promote the very best students often diverts attention from a far more critical goal: namely, to maximize the academic achievement of all students. If business leaders, policy makers, or educators focus their interest too exclusively on high-achieving students at high-profile institutions, they will do little to improve overall completion rates.

5. Benefits: What benefits does a state receive as a result of having a highly educated population?

Economic Benefits

The benefits category measures the percentage of a state's population with a bachelor's degree or higher, the economic and civic benefits associated with that state's college-educated adults, and the percentage of adults demonstrating high levels of quantitative, prose, and document literacy. The relevance to employers of economic benefits and adult skill levels is clear. A well-educated state population offers employers higher skills, enjoys a higher average income, and generates greater tax revenues.

Civic Benefits

The civic benefits of higher education are equally significant. The skills necessary for success in the workplace have converged with those that undergird true civic involvement. Voters need knowledge and judgment: active participation in democracy today requires a more profound grasp of topics as varied as economic policy, scientific innovation, foreign relations, cultural difference, and health care. The benefits of higher education include personal, economic, and social well-being, all of which contribute to an environment in which business can thrive.

6. Learning: What do we know about student learning as a result of education and training beyond high school?

Student Learning Outcomes: What We Don't Know Can Hurt Us

Measuring Up 2000 and Measuring Up 2002 reveal that we in fact know very little about what students learn in their education and training beyond high school. This dearth of information prevented the National Center from establishing measures for comparing performance across the 50 states. As a result, every state received an "Incomplete" in this critical performance category.

The 50 "Incompletes" in learning are especially troubling in light of the category's importance to any coherent assessment of quality in higher education. Quality has long been measured in terms of inputs, such as the number and educational background of faculty, or the number of books in a university library. Unless we evaluate their impact on student learning outcomes, such measures of quality can tell us very little, indeed.

As the ultimate measure of quality, student learning lends the other performance categories their full significance. We cannot truly measure the benefits of higher education without a clear accounting of learning, nor can we determine the ultimate value of high participation rates, high completion rates, or even affordability. High quality and widespread accessibility must become inseparable goals.

Because the new currency of the labor market is demonstrated competencies rather than academic degrees, the deficit of information on what students actually learn in college should worry business leaders, educators, and policy makers alike. We simply cannot gauge how well American higher education is preparing us for the demands of an accelerating knowledge economy.

The Growing Need to Communicate Student Learning Outcomes to Employers

The assessment of learning outcomes not only helps policy makers evaluate the intellectual capital of their state, but also helps individual employers make important hiring decisions. Businesses cannot take educational quality on faith. Rather, they require a reliable sense of the knowledge and skills prospective employees can bring to the job.

Because skills have replaced materials as an employer's most essential resource, the lack of information on learning threatens to weaken the ability of businesses to meet world-class standards in a competitive global economy. In an environment where people must move freely and efficiently between education and jobs, the clear communication of demonstrated learning outcomes is essential to facilitating this movement. The reputation of an institution or degree can no longer serve as a reliable proxy for skills and knowledge. The half-life of a degree's reputation is, in fact, growing shorter as changes in skill requirements accelerate; and the importance of demonstrating actual competency has increased accordingly.

Driven by rising demand for lifelong learning, the explosion of new postsecondary credentials-including degrees, occupational skills certificates, and short-term diplomas-further heightens the need for demonstrated learning outcomes. Confronted with this confusing array of different credentials from an unprecedented variety of education providers, employers require a standard for distinguishing true quality.

14 Quoted in Ronald Phipps, College Remediation: What It Is, What It Costs, What's at Stake (Washington, D.C.: Institute for Higher Education Policy, 1998), vii.
15 Vartan Gregorian, "Teacher Education Must Become Colleges' Central Preoccupation," The Chronicle Review, August 17, 2001.
16 A Back to School Special Report on the Baby Boom Echo: No End in Sight (Washington, D.C.: U.S. Department of Education, 1999).
17 National Center for Education Statistics, National Postsecondary Student Aid Study (Washington, D.C.: U.S. Department of Education, 2000).
18 Faces of the Future (Washington, D.C.: American Association of Community Colleges, 2000).
19 U.S. Department of Education, National Center for Education Statistics, 1999 National Household Education Survey, quoted in Brian Bosworth and Victoria Choitz, Held Back: How Student Aid Programs Fail Working Adults (Belmont, MA: FutureWorks, 2002), 2.
20 Trends in College Pricing, 2001 (New York: College Board, 2001).
21 Measuring Up 2002, 26.
22 Trends in College Pricing, 2001.
23 Bosworth and Choitz, 4.
24 Bosworth and Choitz, 1.
25 National Center for Education Statistics, Digest of Education Statistics, 2000-2001 (Washington, D.C.: U.S. Department of Education, 2001).