Front Page
  Current Issue
  Back Issues
  About National CrossTalk

National CrossTalk Summer 1999
News Editorial Other Voices Interview

3 of 3 Stories

Are Academic Audits the Answer?
"Assessment movement" seeks to measure and improve student learning

By David D. Dill

David D. Dill  
OVER THE LAST TWO DECADES there has been increasing concern about the quality of teaching and learning for undergraduate students in U.S. colleges and universities. In the 1980s this concern, coupled with the emerging accountability developments in K-12 education, led to public legislation fostering the "assessment movement" in many states, in which colleges and universities were encouraged to undertake efforts to measure and improve student learning.

In the 1990s the issue of the quality of student learning in postsecondary education has received increased emphasis. A number of states became disenchanted with the ambiguous results of the assessment movement, and have sought more rigorous forms of accountability. These have included public policies designed to directly regulate college and university teaching loads and, in an increasing number of states, the adoption of "performance funding systems" tied to various indicators of undergraduate education.

Several observers have suggested that while there is cause for concern regarding the quality of teaching and learning at the undergraduate level, the public policies currently being pursued are unlikely to effectively address the problem. That is, there is little evidence that directly regulating the processes of teaching within colleges and universities or implementing incentive systems tied to simplistic measures of academic performance will affect the underlying causes of variation in the quality of teaching and learning. Such policies may in fact have unintended negative effects on academic innovation, effectiveness and efficiency.

Instead, these observers have suggested that a more effective approach would be to seek means for restoring and strengthening the internal web of accountability by which colleges and universities have traditionally assured the quality of their teaching and learning.

A number of experiments emphasizing professional self-regulation of the quality of undergraduate teaching and learning are now underway within the regional and professional accrediting community. Several state systems of higher education also are exploring new procedures for improving institutional responsibility for the quality of teaching and learning.

The interest in developing new mechanisms for improving student learning also has stimulated curiosity with forms of quality assurance that have developed outside the United States. Academic agencies in other countries have been experimenting with some novel forms of academic accountability. The rapid expansion of access to higher education in Europe and Asia over the last two decades (termed "massification" outside the U.S.) stimulated public concern about maintaining academic quality.

Lacking a tradition of voluntary institutional and/or professional accreditation, a number of countries developed new mechanisms of quality assurance, most of which emphasized the norm of professional self-regulation in their design and implementation. In the United Kingdom, for example, questions by the Thatcher government in the early 1980s about the quality of teaching in universities motivated the Committee of Vice Chancellors and Principals of the old university system to develop a system of institutional reviews termed "academic audits."

Academic audits, like U.S. regional accreditation, are external reviews directed at the institution level. But unlike accreditation reviews, academic audits are focused on those processes by which the institution monitors its academic standards and acts to improve the quality of teaching and student learning, particularly at the undergraduate level. As a consequence, academic audits help to clarify the collective responsibility of faculty at both the institutional and unit level for monitoring academic standards and assuring the quality of teaching and student learning.

Similar quality assurance mechanisms soon were adopted in New Zealand, in a number of the Nordic countries, in a Europe-wide system of institutional reviews implemented by the Association of European Rectors, and most recently in Hong Kong.

The contemporaneous development and rapid dissemination among other countries of audit-like quality assurance processes suggests that this model may offer some particularly relevant solutions to those in the U.S. who are seeking answers to the problem of improving the quality of undergraduate teaching and learning through academic self-regulation.

Much of the effort to improve teaching and learning in U.S. colleges and universities focuses on incentives and structures designed to improve the performance of individual instructors. These mechanisms include providing teaching awards to faculty members, developing teaching centers that offer support to concerned faculty members, encouraging faculty members to adopt classroom-based assessment techniques for improving instruction, mandating student evaluation of instructors, and disseminating new instructional technologies to interested faculty members.

In contrast, the clear lesson from the quality movement that swept though industry and education over the last decades is that significant improvement in the quality of programs, services or products results not from changes in the individual practices of workers or professionals, but from changes in the processes by which work is coordinated and integrated. Improving the quality of student learning must therefore be a responsibility of the faculty acting collectively rather than individually.

Similarly, research in the United States on teaching and learning consistently reveals that while students' learning of academic content and their cognitive development are related to the quality of individual teaching they receive, it is also significantly associated with the "academic coherence" of the curriculum. That is, student learning is affected by the pattern and sequence of the courses in which they enroll, by curricula requirements to integrate learning from separate courses, and by the frequency of communication and interaction among faculty members in the curriculum.

Therefore more systematic efforts to improve the quality of learning outcomes will necessarily involve encouraging collective efforts by faculty members to redesign course sequences and requirements in order to achieve greater academic coherence, and to develop consistent, valid means for assessing student achievement.

Unfortunately, there is increasing evidence that the capacity of the academic profession(s) and of colleges and universities to encourage faculty collaboration in improving teaching and student learning is eroding. The first cause of this change appears to be a weakening of the professional norms that supported faculty collaboration in the development of coherent curricula and of the collegial processes and structures that traditionally held faculty members collectively accountable for the quality of student learning. A second and related cause, as George Kuh has suggested, is the measurable shift in faculty time and effort away from teaching and into research and scholarship.

In the case of disciplinary norms and standards, surveys on the views of faculty members in different disciplines suggest declining normative influence from the various academic professions over the curricula of academic programs. In many disciplines faculty members did not easily agree on definitions of curricula content, nor were they in agreement that specified sequences of learning content were appropriate for students.

Even in professional fields such as business and engineering, where faculty members reported the highest perceived consensus on the nature of academic knowledge, they also reported that defining the content of the professional courses was one of their most serious curriculum tensions. In several disciplines, faculty members expressed the belief that the field's diversity precluded achieving a consensus on what students should know.

In the case of institutional-level mechanisms for accountability, recent studies on university-level policies for influencing the structure and coherence of academic curricula reveal a similar loss of consensus. For example, a heavily cited early report on the integrity of the college curriculum argued, "The curriculum has given way to a marketplace philosophy: It is a supermarket where students are shopping and professors are merchants of learning. Fads and fashions, the demands of popularity and success, enter where wisdom and experience should prevail. The marketplace philosophy refuses to establish common expectations and norms…Electives are being used to fatten majors, and diminish breadth. It is as if no one cared, so long as the store stays open."

Empirical studies of the catalogues of selective universities and of the transcripts of enrolled students also suggest a decline in the structure and coherence of the undergraduate general educational curriculum. Massy and Zemsky argued in 1994 that there has been "an incipient destructuring -- or deconstructing -- of the undergraduate curriculum over the last two decades that has resulted in fewer required courses, less emphasis on taking courses in ordered sequence, and greater reliance on students to develop their own sense of how the various bits and pieces of knowledge they acquire in the classroom fit together into a coherent picture."

University policies on senior theses and comprehensive examinations also can influence the educational effectiveness of academic curricula. Such requirements foster coherence in students' learning by requiring the integration or analysis of a body of knowledge broader or deeper than that provided in a single course. A 1996 National Association of Scholars study of the curricula requirements over time of the fifty most selective U.S. universities revealed a continuing decline in mandatory requirements for such comprehensive assessments, from 66 percent of the surveyed universities in 1939, to 56 percent in l964, to 12 percent in 1993.

....With regard to faculty involvement in improving teaching and learning, field research at the departmental level in U.S. universities (Massy, Wilger and Colbeck, 1994) also has revealed a form of academic behavior termed "hollowed collegiality." Departments nominally appear to act collectively, but avoid those specific collaborative activities that might lead to real improvements in curricula and instruction.

....Massy, Wilger and Colbeck concluded that "despite these trappings of collegiality, respondents told us they seldom led to the more substantial discussions necessary to improve undergraduate education, or to the sense of collective responsibility needed to make departmental efforts more effective."

A major contributor to this hollowed collegiality was an observed pattern of fragmented communication within departments in which traditions of individual autonomy and academic specialization have led to curricula atomization and isolation among faculty members. Faculty members in the U.S. not only do much of their teaching alone, but because disciplinary sub-fields are defined quite narrowly, many faculty members find it almost impossible to discuss their teaching with one another.

A second and related negative influence on the improvement of student learning as noted by Kuh is the increasing faculty commitment to research. In an influential series of papers published between 1978 and 1990 the economist Estelle James argued that universities are best understood as multi-product firms, producing both teaching and research. At the departmental level universities operate as labor managed cooperatives in which the faculty have significant control over the factors and consequently the costs of production.

But faculty members have a strong preference for research over teaching, because of its intrinsic interest, because of its clear contribution to unit reputation (which in the U.S. is a major proxy for academic quality), and because in increasingly competitive research and academic labor markets time spent on research can lead to higher grant revenue and future earnings for the individual faculty member. Consequently, faculties may arrange their workloads to limit their time investment in teaching and to maximize their time investment in research.

James' model has been confirmed in a number of recent studies of faculty workload in the U.S. that reveal a consistent pattern of faculty decisions leading to a reduction in time devoted to teaching.

It is important to stress that this reduction in time spent on teaching not only may affect faculty performance in individual classrooms, but also negatively affects those collective activities of curriculum development, teaching evaluation and student assessment upon which both effective student learning and the maintenance of academic standards are critically dependent.

Finally, it must be noted that the existing policies and practices for assuring academic standards and quality within U.S. colleges and universities often focus more on the organization and delivery of academic content than on student learning. Since World War II the academic quality assurance mechanisms in most colleges and universities have come to reflect the research universities' valuing of substantive academic content over instruction.

The purpose of this brief article is neither to impugn the motives of faculty members nor to debase the principal of professional self-regulation. To the contrary, if we are to maintain professional control of academic work in the face of increasingly assertive public calls for external regulation, we must clearly understand the possible causes of current problems in student learning.

The reported decline in learning-related student academic activity may of course be related to changes in the broader culture and to the motivations and values of students themselves. But research also suggests erosion of the culture of professional accountability by which colleges and universities have traditionally assured the quality and standards of their academic programs and degrees.

Many of the changes in academic behavior outlined in the preceding section are the product of a pronounced shift over the last forty years in the academic "rules of the game" -- the policies, norms of behavior, and sanctions that facilitate faculty interaction and coordination.

As Elliot Krause has noted in his comparative study of professional influence, "Death of the Guilds," the legal, medical and academic professions have seen their power over their members' behavior steadily decline in the latter half of the 20th century. In each of these professions the regulatory inroads of the state and the growing influence of corporations and the market have altered the incentives for cooperative behavior.

These larger social forces are increasingly compromising the ability of the traditional norms and sanctions of the academic professions, the policies of individual colleges and universities, and the rules of relevant departments to sustain faculty commitment and cooperation in improving student learning.

In this new context we will need to identify means by which the academic community itself can act to restore and develop the internal web of academic accountability within colleges and universities whereby the quality of student learning and maintenance of academic standards is assured. Responsibility for academic standards and the quality of teaching and learning must lie where the power exists to control or change academic practices -- with the faculty of an institution.

But seeking means to better understand and to strengthen the processes within academic institutions whereby the faculty collectively and within academic units exercises its responsibility for academic quality assurance and improvement is a reasonable and necessary public goal. The process of academic audit might contribute to strengthening collegial accountability for improving the quality of teaching and student learning.

David D. Dill is professor of public policy analysis and education at the University of North Carolina at Chapel Hill.

Photo by Jim Strickland, BlackStar, for CrossTalk

E-Mail this link to a friend.
Enter your friend's e-mail address:



National Center logo
© 1998 The National Center for Public Policy and Higher Education

HOME | about us | center news | reports & papers | national crosstalk | search | links | contact

site managed by NETView Communications