Resize Text: 
The National Center for Public Policy and Higher Education and the Council for Adult and Experiential Learning are pleased to announce that Ralph Wolff, the president and executive director of the Western Association of Schools and Colleges since 1996, is the 2008 recipient of the Virginia B. Smith (VBS) Innovative Leadership Award. The award recognizes his leadership in redesigning regional accreditation.
Virginia B. Smith, after whom the VBS award was named, is president emerita of Vassar College and was founding director of the Fund for the Improvement of Postsecondary Education (FIPSE) and associate director of the Carnegie Commission on Higher Education. Joni Finney is the former vice president of the National Center for Public Policy and Higher Education and a practice professor of education at the University of Pennsylvania. Here, they interview Ralph Wolff about his work.
Finney: The selection committee for the Virginia B. Smith (VBS) Award was very impressed with the work you have done at the Western Association of Schools and Colleges (WASC) to create a new kind of accreditation review for senior colleges and universities. We want to learn more about that and the process you went through to develop it.
Wolff: Thanks so much. I’m very honored to receive the award and very much appreciate having the opportunity to talk about what we are trying to achieve at WASC.
Smith: It must have been a difficult task, and we certainly want to hear what you have done and how it was accomplished. But before we get into that, I’m curious about how you got into accreditation work.
Wolff: It happened quite naturally. I went to George Washington Law School from 1968 to 71, and although I did very well—I was editor of the law review—I didn’t really like it. Frankly, I found it not sufficiently engaging and not relevant to the important values of the day.
In my last year I was working on a law review article with Jean Camper Cahn, who had been a law faculty member at GW and the founding director of the Office of Economic Opportunity Legal Services Program. In one of our conversations, I told Jean that she ought to think about creating a law school that would be concerned with public-service law. And she said, “I’ll do it if you can show me it can be accredited.”
So in my last semester of law school, I took the time to learn about substantive-change rules for the North Central Association (NCA) and the accreditation standards of the American Bar Association (ABA). Shortly thereafter, we received a major grant to found a brand-new law school emphasizing public-service law. At that time Antioch College in Ohio had branch sites all over the country. Since it shared the values of the new law school, we negotiated an affiliation with Antioch and set up the law school in Washington, D.C.
I became the director of planning—at the ripe old age of 24. My responsibility was to navigate the school through the North Central review process and to prepare it for ABA accreditation. In record time, the new school, which looked quite different from other law schools, received both accreditations. It continues to exist, only now it is part of the University of the District of Columbia.
After that, the central administration of Antioch asked me to assist in developing an internal quality-assurance process and to work with state licensure in 28 states in which Antioch had programs, as well as being the liaison with NCA for the reaccreditation of all of the Antioch campuses and branches. Working under the mentorship of Morris Keeton, I helped establish a program-review and quality-assurance process for the entire network of programs and branches.
After six years at Antioch, I left to teach law at the University of Dayton. There, I did a lot of work with NCA, serving as a member or chair of several accreditation teams and consulting with institutions about accreditation. Through all this work I became very interested in the interplay between internal and external quality-assurance systems.
When a position opened at the WASC Senior College Commission, Patsy Thrash, the executive director of NCA at the time, called me and encouraged me to apply. I started to work at WASC in January 1981 for what I thought would be three years at most. As you know, I stayed much longer—27 years and counting. It’s been so interesting and challenging that I’ve enjoyed just about every day.
Smith: It sounds like you found the right niche—luckily for you and for accreditation. That leads me to the next question. We all know how slow organizations are to even want to change, let alone actually change, particularly membership organizations. What compelled you to begin the process of change at all? It would probably have been easier to just go on as you were.
Wolff: WASC had been operating for a long time under a handbook that had nine standards and 268 subcomponents. I believe this led to institutional fatigue and a developing sense on the part of institutions, especially the larger comprehensive ones, that all the investment in the accrediting process resulted in very little return on that investment or meaningful change. It became too often a time- and resource-consuming exercise to see if minimum standards were being met, and it had little lasting value. For smaller institutions, self-reviews often proved quite valuable, but even there the institutional investment was significant, and the question remained whether the value added was worth the investment made.
Smith: Do you work with smaller institutions more often than with larger comprehensive ones?
Wolff: Over the last 40 years, there has been a dramatic shift in higher education: Most students today are in larger institutions. This has significant implications for accreditation and its effectiveness.
When accreditation became coupled with financial-aid eligibility after World War II, most institutions enrolled fewer than 10,000 students. Small colleges dominated; the accrediting process then was based on a group being able to “get its arms around” the institution in a single visit. It also seemed quality was easier to define then, using traditional input and resource measures.
But now, 68 percent of our students are in multi-school (and often multi-campus) institutions with well over 10,000 students. So the challenge was to design a process that served large complex institutions equally well. It was clear that the process we had didn’t do that well enough.
Finney: Had the student population also changed over that period of time?
Wolff: Yes, and this too has presented major challenges to the effectiveness of accreditation. Especially in the WASC region, students are more diverse in terms of age, race, ethnicity, and national origin. Many are inadequately prepared for college-level work. In addition, modes of instructional delivery have been changing. That so many aspects of the educational scene have altered has required a transformation in how we define and evaluate quality.
Another element entered the picture about the time we were thinking about changing our system. Beginning in the late 1990s and continuing today, there has been a national conversation on accountability, particularly centered around student learning outcomes. In 1998 Congress made student achievement the first of nine areas in which we had to have standards. But even though a focus on student learning assessment and outcomes needed to become more of a priority for us and institutions, we found that the ones we reviewed were repeatedly “just getting started.” All of these factors led to greater willingness on the part of the region to discuss fundamental changes to the accrediting process.
Smith: These conditions were true in all the accrediting regions. Was there anything that happened with the Senior College Commission of WASC that led it to be more interested than other regions in redesigning accreditation?
Wolff: I can certainly recall several factors that were specific to WASC. For instance, Pat Callan, president of the National Center for Higher Education and Public Policy, was a public member of the commission. When he left, his parting statement to the commission was that it was very troubling for him to read WASC reports in which time and again, teams called for institutions to spend more money—often for physical facilities, libraries, or faculty—but hardly ever were these recommendations connected to improving the outcomes of the learning experience. We assumed that this result would occur but never verified that the changes had had this effect. That led the commission staff to learn more about the institutions that were addressing learning outcomes—such as Alverno College, Kings College, and Truman State-—and to explore how their models might be useful in a new approach to accreditation.
Smith: Listening to you, Ralph, I get the impression that you personally were not particularly daunted by the difficulties of organizational change but rather looked forward to the possibility of taking a new road in interesting directions.
Wolff: That was definitely the case. Part of the reason was that I had been exploring organizational change for several years. In the late '80s and early '90s, outside of my WASC work, I had created the Institute for Creative Thinking, an organization that held workshops and seminars several times a year to study learning organizations. Most of those attending the meetings were from higher education. Personally I was, and continue to be, convinced that we needed to create more adaptive organizations, and accrediting groups at this time especially needed to learn to be far more adaptive.
The James A. Irvine Foundation gave a small grant to the institute. In 1996, when I became director of WASC, I transferred my work with the institute to WASC, and the Irvine Foundation transferred and greatly increased its grant. Soon after that the Pew Charitable Trusts gave WASC an even larger grant to change its accrediting system. Together, the grants gave us $2.4 million over four years for a major transformation of our standards and processes.
Finney: So by about 1997, you had the desire and means to make the changes. You were working with about 160 colleges and universities and a commission of 21 members, including many college presidents and other administrators as well as some faculty members and a few public members. How did you start the process?
Wolff: We had already started it through the Institute for Creative Thinking. Through that organization, I was able to work with several others in higher education from the WASC region to study the literature on learning organizations and how organizations change and adapt in an evolving environment. One of the things that came out of our reading and talking was that we needed to get much clearer about accreditation’s purpose.
The traditional model of accreditation was to have an institution define its mission and then to evaluate whether it was achieving it. Our early discussions made it evident that in one two-day visit every 10 years, we couldn’t assess all aspects of the institution, particularly large comprehensives and major research universities. We needed to rethink what our primary focus should be and then redesign our standards and processes around that.
We were able to enlist considerable support for the redesign by framing a set of questions for the region. We asked, “What should the WASC process achieve? How well are we accomplishing that?” and “What would be the best model of accreditation for five to 10 years from now?”—not “How can we make our old standards more effective?” This freed up great creativity and new thinking. Institutions wanted a process that would recognize their own context and strategic priorities, build on existing data, and help them learn how to address student learning more effectively.
Ultimately, the process led to what we now call our “core commitments.” We agreed that the accrediting process would concentrate on verifying two core commitments made by institutions to its stakeholders—commitments about institutional capacity and educational effectiveness. This sharpened focus enabled us to develop new approaches to our standards and to revise our site-visit process significantly.
Finney: So this seemed to you an ideal time to redesign the process?
Wolff: Correct. Even though I’d been at WASC a long time, I had just become executive director, and I was very excited about the potential for leading WASC and for the institutions we accredited to become more adaptive learning organizations. And we now had the resources to support our deep inquiry into what a new model might look like.
Our grants led us to be among the first to initiate these major reform efforts, but the Pew Charitable Trusts’ grants to other regional accrediting associations created a larger change environment. Its grant to the North Central Association (now the Higher Learning Commission) enabled NCA to develop another alternative accrediting model based on the Baldridge process—the AQIP (Academic Quality Improvement Program). And Pew then funded the Southern Association (SACS) reform effort, which led to the development of its two-stage approach and the Quality Enhancement Project (QEP). There are many similarities among these models; they all focus on continuous improvement and on what we at WASC call “a culture of evidence.”
The fact that a few other regions were also changing their standards and procedures made our task more possible. It is hard to go through a process like this alone, and I’m delighted to say that we were able to learn from other regions as well.
Finney: What steps did you take in order to make the changes? Did you go through a lot of stages in the change process itself?
Wolff: In 1996, when we started this process, we had the idea (and it was only an idea) that accrediting agencies needed to change significantly to be more relevant, add value to institutions, and be more responsive to the growing calls for external accountability. And I remember that in my talks in various parts of the region, I would often say that we wanted to see if it was possible to have the words “creative” and “exciting” in the same sentence with “accreditation.” At the time people thought it was impossible. But we were determined to transform the process from one in which the object was to show that the institution was meeting minimum standards to one that really generated excitement and the potential for change within institutions.
And then the question was, in what areas would those changes be most important to promote? We chose not to use “assessment” or “outcomes” but increasingly centered on the term (which I think we popularized) “educational effectiveness.” We wanted a term that did not already have the baggage of prior (mis)understandings. This opened us up to working collaboratively with institutions on defining what educational effectiveness should mean. Thus, we started with questions rather than conclusions.
Not having a clear concept of what educational effectiveness should mean, we invited Morris Keeton, Alexander (Sandy) Astin, Peter Ewell, and others to write concept papers that addressed this question: What would a new model of accreditation look like if its primary goal were to evaluate and encourage educational effectiveness? We then collected and published the papers and used them as a basis for discussion during concept-development sessions with institutional representatives, trustees, public members, and policy-makers from inside and outside the region. Based on these papers, we then held “blue-sky sessions” in which we discussed what model of accreditation would look most appropriate for five to 10 years out.
We began doing that in 1997 and here we are, more than 10 years later, with the new model in place and functioning.
We also used some of our grant funds to develop a number of experimental visits that challenged the idea that a single three-to-four-day visit could “do it all” for any institution, regardless of size. We built on ideas that came from the concept-development sessions, our study, and our use in one experimental campus visit of the Baldridge process. We also wanted to learn from some new international quality-assurance approaches, so we sent teams to study the academic-audit process in England and Hong Kong.
Finney: What did you learn from the experimental visits?
Wolff: We learned a lot, and I hope we never lose the capacity to experiment periodically to keep our processes adaptive and effective. We learned first and foremost that calling for more focus on outcomes alone would not change accreditation’s center of gravity. We needed to look at, and drastically revise, the core process of “self-study and single visit” to achieve the change in institutional culture that we wanted to see.
We found that when we conducted single visits that included reviews of both the adequacy of resources and educational effectiveness, our teams tended to gravitate to a “default mode.” They focused on compliance and inputs and resources, because that’s what they were most familiar with, and those things were much easier to review.
And so we decided that if we really wanted to focus on educational effectiveness, engage faculty, and develop learning organizations using the accrediting process, we needed to redesign the entire visit process. Each institution would submit a proposal, which would be peer reviewed, indicating what its outcomes would be for the accrediting review process in return for its investment in it. We then divided the review cycle into two reviews 18 months apart, one focused on institutional capacity and the other on educational effectiveness. We also established page limits for each review in order to focus institutions on data and evidence.
This was, I think, a major breakthrough. But we also realized that capacity played a very important role in ensuring quality—an institution needs resources, structures, and processes to carry out its educational mission. There are still many institutions that are challenged in their financial, technological, and personnel capacities, among others. A visit solely for this purpose has proved very valuable and also helps institutions prepare for their educational effectiveness reviews.
So we settled on two visits, but we still needed to draft a handbook that described the nature of those visits and what principles or standards we would abide by. Doing so was truly a collaborative endeavor, with a genuine sense of openness among the 300-plus people involved in the task.
The commission chair—Marilyn Sutton, a faculty member from California State University, Dominguez Hills—and other members of the commission were very helpful during this period of incubation. We also met regularly with the accrediting liaison officers of our research universities, including the University of California, the University of Southern California, and other major institutions, and we think that their ideas helped make this process much more functional for them as well. Indeed, they were major advocates of these changes.
The process culminated in a series of extensive drafting sessions that included many representatives from the region, and after that comments from many inside and outside of the region. All of this eventually led to the handbook in 2001. Fast-forwarding, we’ve now been implementing the handbook’s guidelines for over six years. And it’s kind of like having a child—you can read all the parenting books you’d like, but when that baby is actually in your arms, you realize you’re just starting out, and you still have so much to learn.
Even with all the experimental visits, consultations, drafting sessions, and comments from the field, when we started doing our reviews it became very clear how much we needed to fill in this new concept of what accreditation could become: a collaborative inquiry, something that might stimulate creativity, that might focus institutions on building and sustaining educational effectiveness.
Over the last six years, we realized that, with regard to educational effectiveness, we needed to focus more on student and organizational learning and on learning results, not just assessment processes and activities. Toward this end, we developed a series of forms and rubrics, available on our website, that clarify our expectations and that assist institutions in reviewing programs and assessing student learning. We’ve also come to realize that we weren’t completely clear about what we wanted to do with the capacity review. We’re still trying to clarify that.
It became clear as well that we needed more effective training of evaluation teams. We discovered early on in the implementation of the new handbook that most of the people appointed to teams, particularly ones who had served before, had imbedded in their minds the idea of accreditation as a compliance mechanism. There were not that many people in the country who really understood learning-outcomes assessment at the course, program, or institutional levels. We also wanted to engage faculty more. So we‘re now running a wonderful series of educational seminars for institutional teams. We hope to set up seminars next year on assessment in the disciplines as well, to engage faculty even more.
In other words, we aren’t standing still but trying to reach out to institutions and to our evaluators to improve the value and effectiveness of each and every review.
Finney: I understand there has been an evaluation of the new handbook and process. What did you learn from that?
Wolff: Yes. Since we were in the business of evaluating others and had been using the new handbook for several years, we thought it was time to evaluate ourselves. The results are available on our website. We developed a data portfolio that’s quite extensive. In addition to the external evaluation, which was undertaken by a team appointed by the commission, we did several things for our own educational-effectiveness review that I think are quite distinctive.
We wanted to be considered a world-class accrediting association, so we had to engage some experts outside as well as inside the United States. We sent out three WASC team reports on the institutional educational effectiveness of three different types of institutions, along with the action letters based on those reports. One went to Christian Thune, executive director of the Danish Quality Assurance Agency and one of the founders of the European Network of Quality Assurance Agencies (ENQA), which has developed European standards for quality-assurance agencies. Another went to Pat Hutchings, vice president of the Carnegie Foundation for the Advancement of Teaching, who is nationally respected for her work with faculty on improving teaching and learning. A third went to Frank Murray, the executive director of the Teacher Education Accrediting Commission (TEAC), which had developed an innovative evidence-based approach to accreditation. We asked them whether our reviews met the goals we had set ourselves for transforming accreditation and provided evidence that this process met the highest standards. We received very thoughtful reviews, each with very constructive critiques and suggestions for making our process more effective.
We also had Mary J. Allen, a national expert and author on assessment, do a content analysis of 12 of our visiting-team reports and the related commission letters of action. The analysis was based on some questions that were submitted by WASC and others that emerged during the planning and conduct of the review. The finding I found most encouraging was that even during the short span of time during which we had introduced and started using the handbook, Allen found a “significant and growing impact of the new WASC standards.”
As a third part of our self-evaluation, WASC did a survey of the concerns of institutions that have gone through the process. Peter Ewell, staff to the evaluation committee, set up a confidential e-mail address that could be used by those wishing to comment on the process or the handbook. And the members of the committee, in addition to meeting with presidents, chief academic officers, and accrediting liaison officers to review our effectiveness, independently called a number of institutions that had been reviewed under the new handbook.
All in all, it was an extensive process, very much like the one the institutions go through. We are greatly encouraged by the findings and the ultimate conclusion that we have succeeded in shifting the process overall toward educational effectiveness, with a primary focus on student-learning outcomes. At the same time, these efforts have challenged the capacity of the WASC staff and commission, and we’re now focused on “clarifying and simplifying” a complex process to make it even more effective.
"2005-2006 Carnegie Classification Summary Statistics”
A Carnegie Foundation internal memo, November 2007
Smith: How would you characterize the changes you’ve made in accreditation?
Wolff: The transition has been nothing less than a complete transformation—from a regulatory, once-a-decade, compliance-oriented process to a reflective, evidence-driven, and learning-outcomes-based one that is adapted to the plans, needs, and priorities of each institution and that provides multiple points of feedback over the course of four years.
We’re convinced that this shift is worth the additional effort that makes it work. You can’t use a regulatory process to adapt to the future. The process has to be open and exploratory in nature, even while it ensures that certain standards are met. In the WASC self-evaluation we also said that the shift in focus, from capacity and inputs to student and organizational learning and outcomes, requires a new vocabulary and mode of communication.
Finney: Do you think we are asking too much of accreditation?
Wolff: The national conversation regarding accreditation and recent efforts at rule-making have attempted to give accreditation major new roles and responsibilities—especially for setting standards, which should be in the hands of institutions. The future challenge is to work with institutions to set high standards, evaluate performance against those standards, and strengthen the peer-review process to improve institutional results. I believe accreditation can and should be at the forefront of these efforts. WASC will continue to innovate and adapt to make the process more effective and responsive.
Smith: I can assure you, Ralph, that we’ll all be watching with great interest to see how this shift in emphasis, conversation, and action play out. We wish you great success in your efforts.
Wolff: Thank you very much.
Examples of the New Criteria
1. All degrees (undergraduate and graduate) awarded by the institution are clearly defined in terms of entry-level requirements and in terms of levels of student achievement necessary for graduation that represent more than simply an accumulation of courses or credits.
2. The institution’s expectations for learning and student attainment are developed and widely shared among its members (including faculty, students, staff, and where appropriate, external stakeholders). The institution's faculty takes collective responsibility for establishing, reviewing, fostering, and demonstrating the attainment of these expectations.
3. The institution demonstrates that its graduates consistently achieve its stated levels of attainment and ensures that its expectations for student learning are embedded in the standards faculty use to evaluate student work.
4. In order to improve program currency and effectiveness, all programs offered by the institution are subject to review, including analyses of the achievement of the programs’ learning objectives and outcomes.
Virginia B. Smith Innovative Leadership Award—Previous Recipients and Their Positions at the Time
2007
Carol Twigg
President
National Center for Academic Transformation
2006
David Spence
President
Southern Regional Education Board
2005
George Kuh
Chancellor’s Professor of Higher Education
Indiana University Bloomington
Director
National Survey of Student Engagement
2003
Barbara Leigh Smith and Jean MacGregor
Co-Directors
National Learning Communities Project
The Evergreen State College
2002
Robert Olin
Dean and Professor of the College of Arts and Sciences
The University of Alabama
2001
Tim Riordan
Professor of Philosophy and Associate Dean for Academic Affairs
Alverno College Institute
2001
Peter Ewell
Vice President
National Center for Higher Education Management Systems
2000
Susana Navarro
Executive Director
The El Paso Collaborative for Academic Excellence
Virginia B. Smith Innovative Leadership Award Selection Committee Members
Charles I. Bunting
Vice President
A.T. Kearney, Inc.
Patrick M. Callan
President
National Center for Public Policy and Higher Education
Austin Doherty
Director
Alverno College Institute
Peter T. Ewell
Vice President
National Center for Higher
Education Management Systems
Joni E. Finney
Vice President, National Center for Public Policy and Higher Education and Practice Professor of Education
University of Pennsylvania
David O. Justice
Vice President for Lifelong Learning & Suburban
Campuses
DePaul University
Margaret A. Miller
Director
Center for the Study of Higher Education
Curry School of Education
University of Virginia
Virginia B. Smith
President Emerita
Vassar College
Carol F. Stoel
Program Director
Division of Graduate Education
National Science Foundation
Pamela Tate
President
Council for Adult
and Experiential Learning