Skip Navigation

Cover

March-April 2010

Print
Email
ResizeResize Text: Original Large XLarge Untitled Document Subscribe

Transforming Science Education at Large Research Universities: A Case Study in Progress

There are countless reports stressing the economic and societal benefits to be gained from improved science, technology, engineering, and math (STEM) education for all students. But although there is extensive research on alternative teaching methods that increase student learning and are practical to implement (Wieman 2007, Redish 2003), the combined efforts of federal agencies, private foundations, and many internal institutional programs have achieved little overall change in STEM teaching at the large research universities. This remains a major problem for improving science education at any level, since these universities largely set the norms for how to teach science and what it means to learn science.

We are currently testing a way to change the departmental culture for undergraduate science and math education at research universities in nine departments at the University of Colorado (CU) and the University of British Columbia (UBC), funded through the Science Education Initiatives (for more information on the programs, go to www.colorado.edu/sei and www.cwsei.ubc.ca). While it is still relatively early in this change process, there are significant indications of progress.

TABLE 1. SEI OUTCOMES IN THE FOUR MOST AFFECTED OF NINE PARTICIPATING DEPARTMENTS

Overall Measure

CU: 3 depts. after 3 years

UBC: 1 dept. after 2.5 years

Undergraduate teaching faculty who have changed their teaching practices

45 of 87

28 of 43

# of courses changed

41 (82% of student credit hours taught)

19 (> 75% of student credit hours taught)

We see an emerging culture in which faculty are adopting effective evidence-based teaching methods, collecting data on the results, and coming to see teaching as a rewarding scholarly activity (the SEIs have produced over a dozen research papers on science education). Discussions of teaching in these departments have both increased in frequency and shifted their focus from topical coverage to student learning, pedagogy, and evidence.

Here we discuss the model and how it is being implemented, along with some lessons learned and our informal observations of factors that facilitate or inhibit educational innovation.

Assumptions of the Science Education Initiatives Change Model

The SEI model is based on a few basic assumptions and was designed to avoid certain common pitfalls in educational change initiatives. Later we discuss how valid our initial ideas turned out to be.

There is now an unprecedented opportunity to improve undergraduate teaching methods

The didactic lecture remains the pervasive mode of teaching in universities, but there are methods for achieving far greater learning, even in large-class settings, using novel pedagogies supported by inexpensive technology. The superiority of these teaching methods is supported both by cognitive psychology's research on learning and expertise and by discipline-based educational research from STEM classrooms (Wieman 2007 and references therein).

Data are necessary to convince science faculty to teach differently

Scientists are inherently and appropriately sceptical, so data demonstrating that teaching differently will result in greater learning must be part of any effort to convince them to change their practices.

The department is the necessary unit of change

The department is the unit at research universities that decides what is taught and how it is taught in that discipline; thus any sustained attempts to change teaching practices must focus on the culture of the department. To change that culture, one must affect most undergraduate courses and involve most faculty members. Science departments at large research universities are substantial entities, with dozens of tenure-track faculty, numerous non-tenure-track instructors, and budgets of up to tens of millions of dollars per year. The scale of the change effort must be consistent with this size.

Reward structures need to align with change initiatives

To undertake the effort required in changing traditional practices, both the department as a whole and the individual faculty members involved must have clear incentives to change.

More effective teaching need not take additional time or money, although the process of change requires additional resources

There are many examples of how to achieve greatly improved learning without spending more money (or equivalently, faculty time) on an ongoing basis, particularly with the appropriate use of technology.

However, there are costs associated with designing and testing new courses and with supporting faculty as they learn to teach in new ways. While varying by local context, a reasonable estimate appears to be approximately 5 percent of a department's annual budget per year for five years. In the long run though, these costs are small compared to the potential (but as yet unrealized) savings associated with improvements in the efficiency with which both faculty and students use their time.

Several common pitfalls must be avoided

The SEI was designed to avoid the following mistakes, which can derail educational innovation: putting the emphasis on what to teach rather than on what is learned, promoting changes that increase the expense of instruction (e.g. smaller classes or modified classrooms), and trying to change the teaching of isolated individuals while ignoring the surrounding culture.

The Process

Competitive proposals and awards

The first step in the SEI was to put out a call for competitive proposals from departments to support widespread improvement in the undergraduate education they provide. The level of support offered was up to $1 million per department over five years at CU (actual funding levels are typically $600–800 thousand) and up to $2 million over six years at UBC (actual funding levels are $1.5–2 million Canadian)—in both cases, sufficient funds to attract serious attention.

There are a variety of benefits to having a competitive process rather than simply providing funds to all departments. Probably the most important is that, ideally, such competition engages the entire department in the process. Because we required that the proposals address all core undergraduate courses, including courses for both majors and non-majors, the proposal preparation usually involved a collective discussion about the undergraduate educational goals and practices of the department. Typically this was the first time such a discussion had ever taken place in the department.

The proposals also addressed sustainability—that is, how the improved techniques, materials, and assessment data would be disseminated and reused. The department-based process of transforming the majority of courses, usually with multiple faculty involved, automatically contributes to sustainability as it shifts the perceived ownership of courses from the individual faculty member to the department. This makes it possible to sustain the course transformations and to create a more consistent and coherent curriculum.

Departments that primarily discussed what they might do were more successful in their future efforts than those that focused on the deficiencies of the students or the educational system.

Although it may appear more logical to start by establishing the desired graduates' capabilities and designing the entire curriculum accordingly, we recommended that departments work course by course to avoid getting overwhelmed or sidetracked by special interests. The individual course is a more manageable and rewarding target for individual faculty members, and its transformation is an effective way to change their thinking about education. Then, as a number of faculty members in a department become explicit about learning goals for multiple individual courses, they begin to recognize the gaps and redundancies in the curriculum.

During the proposal-solicitation period, Carl Wieman met with nearly all of the eligible departments to discuss the initiative and the proposal process, typically as part of a faculty meeting. The proportion of the department in attendance at the meeting and the guidance provided by the chair/head were good predictors of the overall interest of a department and of departmental leadership. These meetings also foreshadowed many of the issues that have played out throughout the SEI work.

Despite some scepticism as to the possibility of making dramatic improvements in education, nearly everyone felt that student learning could and should be improved. But there was wide variation in the ideas about how to achieve that improvement. Departments that primarily discussed what they might do were more successful in their future efforts than those that focused on the deficiencies of the students or the educational system.

Faculty frequently expressed concern that they might lose control of the courses they taught. More surprising was the very vocal opposition of a few faculty members who prided themselves on being good teachers, and were recognized as such, but whose reputations were largely based their ability to give captivating lectures rather than on any evidence of student learning.

All but one of the eligible departments chose to submit proposals. The most successful carried out extensive deliberations, and their proposals reflected considerable planning, consensus, and commitments to carry out specific actions. Proposals from other departments were less specific and/or were written with the faculty's approval but without their broad involvement or commitment. Naturally both types experienced a certain amount of difficulty and had to readjust as the project was ramping up, but the latter continued to struggle during the implementation phase as well.

At CU, funding was provided in 2006 to the departments of Geological Sciences, Molecular-Cellular-Developmental Biology, Integrative Physiology, and Chemistry and Biochemistry. A year later, aided in part by a grant from the NSF, the Department of Physics joined the group.

UBC is ramping up more gradually. Funding was provided in mid-2007 to the Department of Earth and Ocean Sciences and the undergraduate program in Life Sciences. After a second round of proposals, in early 2008 two more departments—Computer Science and Physics and Astronomy—were funded, with others likely in the future.

Implementation

Like an external funding agency but with more oversight, we transfer funds to each department annually to spend as they choose, subject to suitable progress in carrying out the proposed course transformations. The features we look for in a successful transformation are listed in Table 2.

TABLE 2. CENTRAL FEATURES OF COURSE TRANSFORMATION

Outcomes

Process

Course- and topic-level learning goals

Meet in working group

Descriptions of student thinking and difficulties with content

  • Review literature

  • Observe students in the course before and after transformation

  • Interview students

  • Create and administer diagnostic tests

Faculty-valued, student-tested assessments of learning

  • Align exams with learning goals

  • Develop research-based conceptual assessments

  • Administer pre/post tests and surveys

  • Interview students

Improved teaching methods

  • Create course materials and activities consistent with research on learning that address known student difficulties

  • Implement desired teaching practices and course structures

Archived materials

Organize materials locally and online

Plan for sustainability

Establish suitable departmental structure, plan teaching assignments

All departments chose to use most of the money provided to hire “science education specialists” (SESes). The SESes collaborate with individuals or small groups to implement the SEI course-transformation process (Chasteen 2010), helping faculty increase their knowledge of teaching and learning research and supporting the introduction of new educational practices and the assessment of learning. They use a variety of methods (e.g., interviews, classroom observations, and analysis of written work) to gather detailed data on student thinking and learning that is shared with the instructor and guides the course design.

There are currently 22 SESes across the nine departments. Success in this role requires mastery of the discipline (most are new PhDs), knowledge of educational and cognitive psychology research and of proven teaching methods, and (most important) diplomatic skills. A small SEI central staff provides training in a few key areas: learning research and science education, learning-goals development, clicker-question design, interpersonal communication, cognitive interviews of students, and designing and conducting rigorous assessment and research studies.

While the SESes are part of a mutually supportive SEI community, first and foremost they must be seen as valued members and resources by the faculty in their home departments. The capabilities of the SESes in assessing learning, identifying student difficulties, and improving student engagement have all been important in enhancing their perceived value. However, solid expertise in the discipline remains critical to the level of respect and trust they are accorded.

The SES model has generally worked well, sometimes spectacularly well. Some departments find SESes such valuable experts on teaching that they are seeking ways to fund these positions permanently. A critical component in their success is establishing an appropriately structured working relationship with faculty. It does not work to simply hire SESes and tell them to find faculty members who are willing to be told how to teach better. Successful SES-faculty collaborations have had some person of authority in the department first obtain from the faculty member a clear commitment to the process and then establish the respective roles and expectations for both faculty member and SES.

Examples

In four of the nine currently funded programs there has been a great deal of progress, and there have been significant improvements in teaching practices by a number of faculty in all but one of the others. Here are some examples of the highly successful ones.

Roles of the “Science Education Specialist” (SES)

Earth & Ocean Sciences (EOS) at UBC

The department has hired four SESes. There are now 28 faculty (2/3 of the department) involved in course or program transformations, and this number is steadily increasing. The department has vigorous leadership and has created a “teaching initiatives committee” to oversee and plan the SEI efforts. The committee is led by an energetic and well-organized faculty member who monitors the SEI work, supervises the SESes, and regularly meets with the department head to review progress.

There is a schedule for transforming nearly all the core undergraduate courses over five years, with an SES assigned to each course and a list of faculty participants. Faculty receive compensation for their involvement in a course transformation, either small amounts of teaching release or extra TA, RA, or post-doc support. Two and a half years after hiring its first SESes, the department has 12 course transformations in process or completed; seven other courses are being changed in line with the SEI goals, often with casual SES help but not full SEI support.

The department has instituted a TA-training program and designed and implemented a pre-post survey of students' perceptions about the earth sciences. It is also working on establishing overall curriculum goals, for which the department is interviewing its recent graduates, older alumni, and employers. Faculty are now examining the alignment of the earth sciences courses, in the process identifying deficiencies in some areas and extraneous topics in others.

The Fluid Earth: Atmosphere and Ocean

Goals

Evaluation

Teaching Changes

  • Course-level goals

  • Lecture-level goals for all lectures

  • Midterm and end-of-term surveys

  • 16-question pre-post test on areas of student difficulty in oceanography and climate change

  • In-class observations of student engagement over time and activity, with summaries for instructors

  • Estensive use of in-class conceptual questions with clickers (archived in an electronic file of lecture notes)

  • Use of small-group discussion

  • Relevance slide added to each lecture and an increased focus on relevance throughout the course

A variety of instructional materials and methods has been introduced in the transformed courses that entail more active learning in the classroom, greater interactivity between students and instructors and among students (who do some work in groups), and more feedback to the students. There are assessments for all of the learning goals, including pre-post conceptual tests, student perception surveys, homework, exams, projects, etc. The conceptual tests, validated through student interviews and faculty reviews, are given each time the course is offered in order to provide a consistent ongoing measure of student learning.

Many instructors now require pre-reading and use clicker questions and peer discussion in class, emphasizing critical thinking and deemphasizing information delivery. Many courses have homework activities that are either new or better aligned with their learning goals and guided by SES data on student difficulties.

With regard to sustainability, all information and materials, including faculty lecture notes for each course, are being archived online so that they can be easily reused and adapted by new instructors. Each course archive also contains a summary of the course design, implementation, results, and common student difficulties by learning goal and topic. The department has a plan for how course changes will be preserved, including which faculty members will be teaching the course for upcoming terms.

Integrative Physiology (IPHY) at CU

This department began with 1.5 SESes in spring 2006 and now has three. Its transformation followed the same path as EOS's. But this department was relatively new, having emerged after the restructuring of the biology departments at CU, so at the time of the proposal, its short-term agenda already included a significant examination of its courses and curriculum. As a result, faculty engagement was high from the start.

The department began with extensive discussions, involving many faculty, about the learning goals for its key course on human physiology, including rethinking where in the curriculum this course should be taught. In a discipline where courses have historically required significant memorization, these conversations highlighted the discipline's core concepts, prioritized and pruned its terminology, connected the content to everyday applications, and defined the critical-thinking and data-interpretation skills students needed. A major outcome has been a shift in the learning goals and assessments from simple recall to the application, synthesis, and evaluation of course material.

In this department the SEI is highly visible, and the SESes are viewed as members of the faculty – they appear on the faculty photo-board – and a major resource. Of the 24 teaching faculty, 11 have partnered with the SESes on significant teaching reforms and another 10 have used the SESes as a casual resource, asking for advice on their teaching efforts, feedback on their teaching ideas, or information about education research findings. At each departmental faculty meeting, the SESes and departmental director provide a brief progress report on the SEI. Several faculty meetings have been largely devoted to issues such as drafting learning goals, and prominent education speakers have been part of the departmental colloquium series.

This department's course transformations have been sustained through careful planning of teaching assignments and the creation by the SESes of a “course book” for each course. This is a large binder containing extensive documentation of all the course materials, information on students' incoming knowledge, student difficulties and good ways that have been found to address them, and data on student performance in previous terms. These books have been valuable guides for new instructors and have sustained course improvements.

Reflections and Lessons Learned

Only time will tell if the course, faculty, and departmental changes will continue after the SEI funding ends. However there are several encouraging signs. First, well over half the faculty in the four most successful departments have already adopted new teaching methods. Also, many of the faculty who have participated in the SEI efforts have introduced new teaching methods into other courses without any SEI support. Several transformed courses have been successfully passed along to new instructors. Finally, many departments have made permanent changes in their resource allocations to provide ongoing support for innovations such as TA training.

Here we return to the initial assumptions of the SEI-model of change, noting some lessons learned about the realities of implementing the SEI course transformation framework and summarizing informal observations of factors that have facilitated or hindered departmental change.

Validity of our initial assumptions

Three of the five basic assumptions that underlie the SEI model have clearly proven to be valid and central to our change model.

Improvements in learning. As one would expect from past research, implementing these new teaching approaches has increased student learning, although there are only six courses where we have comparable measures of learning pre and post transformation. Two main factors conspire against conducting accurate pre-post transformation measures of learning. First, when faculty think carefully about their learning goals, they usually conclude that their previous tests were deficient—so they change them too much to allow for comparisons. Second, SESes work with faculty who are ready to make a change and therefore often do not have the opportunity to conduct detailed assessments before the transformation occurs.

Departmental focus. Without exception, the more the department as a whole has been involved and seen this as a general departmental priority, the more successful and dramatic have been the improvements in teaching.

Departmental and individual incentives. There has consistently been greater faculty buy-in when SEI efforts have been explicitly recognized and rewarded. Faculty also mention two important implicit rewards: 1) seeing how much more engaged students can become, and 2) being able to think about and discuss teaching with their colleagues as a serious scholarly activity. Faculty who have experienced these rewards and are vocal about their experiences have been a force for change, particularly if their enthusiastic students chime in.

The remaining two assumptions require some modification. While research and data on student learning are important and useful, they were seldom compelling enough by themselves to change faculty members' pedagogy, particularly when that change conflicted with their beliefs about teaching and learning. Faculty are more convinced by research and data on student learning from their own courses than from the research on student learning in their discipline, and they largely dismiss research from outside their discipline.

However, a number of SEI faculty members have now become engaged in applying scientific research techniques to their teaching. Fifteen faculty not previously involved in education research have now, in collaboration with SESes, published or are preparing articles related to their course transformation efforts.

While some gains in efficiency have been accomplished by sharing clicker questions, lecture notes, and other materials, we continue to see many untapped opportunities for improving efficiency. A glaring example of inefficiency is the large multi-section, multi-instructor courses where all the instructors prepare independent lectures and exams. As well as wasting time, the resulting inconsistencies in learning across the sections result in duplication of material in subsequent courses.

The more the department as a whole has been involved and seen this as a general departmental priority, the more successful and dramatic have been the improvements in teaching.

But surprisingly and unfortunately, instructors involved in teaching these large multi-section courses have been among the most resistant to changing course structures or teaching approaches. One concern is that designing a course that can be shared and copied means that it could be done by someone less expert in the subject. In reality, a course based on interactive teaching strategies requires greater subject expertise.

There are a number of other barriers to change. Both pre-tenure and senior faculty have often compelling reasons for not wanting to try new teaching strategies. And while most faculty members clearly care about their students' learning, we find that it is often necessary, but not always sufficient, for them to have repeated exposure to new teaching strategies and research on learning for one to two years before they are comfortable making significant changes in their own teaching. But perhaps the greatest barrier to change is the belief that poor educational outcomes are due to the deficiencies of “students these days.” This perennial complaint frees teachers from responsibility for the outcomes of their teaching and causes any attempt to change it to be seen as a lowering of standards by coddling deficient students.

Other observations

We have found it critical that faculty members view the SEI course transformation framework as just that—a framework—that they can adapt to fit their particular courses and goals. The first step, getting faculty to think about their courses' learning goals (i.e., what students need be able to do in order to demonstrate that they have mastered the intended knowledge, skills, meta-skills, and attitudes), is much more difficult than we initially expected. Developing those goals is also a much larger part of transformation process than we realized. It requires a major reorientation from thinking about education in terms of the content faculty deliver to seeing it in terms of the operational competencies one wants students to acquire. This reorientation does not happen quickly or easily.

In the literature on course transformations, student resistance is frequently cited as one of the barriers to adopting more interactive and effective learning strategies; however, we have not found that to be the case. Comparing student course evaluations before and after the many course transformations at CU, we noted that the scores remained essentially the same for the same instructors independent of the pedagogy used. There have been two exceptions, but lowered scores in these instances appear to be appropriate responses to poor planning and/or technology bugs rather than resistance to the pedagogy. In one of those cases, the instructor taught the course again, having fixed the obvious problems, and the student evaluation scores rebounded.

Educational Transformations

Factors Facilitating Change

Factors Inhibiting Change

Department-level

  • Supportive, respected chair/head with authority

  • Broad faculty support and involvement

  • SESes highly visible, treated as colleagues

  • Reward structure for SEI-related activities and demonstrably superior teaching

  • Senior and junior faculty leaders who promote the project

  • Newly formed department in need of a curriculum

  • Unsupportive, inactive, or powerless chain/head

  • Effort limited to a few core faculty

  • SESes not integrated into the department

  • Last-minute teaching assignments

  • Departmental culture that

    • does not respect education research

    • expects total individual freedom in teaching

Faculty-level

  • Faculty dissatisfied with student learning

  • Prior exposure to alternative teaching approaches

  • Strong beliefs about teaching and learning that are inconsistent with education research

  • Blaming lack of learning on students

  • Teaching is a low priority

Course-level

  • Structures and resources to support active learning and other changes (classroom spaces, recitation time, TA support)

  • Many faculty teaching multiple sections of same course*

* Achieving the necessary commitment and coordination across more than two or three faculty teaching the same course at the same time has been difficult.

The lack of student resistance is not an accident. In giving guidance to SESes and faculty on the course transformation process, we emphasize the importance of making it clear to the students why courses are being taught in a non-standard way and how this benefits them. In addition, the ongoing formal and informal sampling of student thinking and opinions during the transformation includes the students as partners in the course-improvement effort. Finally, the learning-goals-centered approach helps ensure consistency across all of the course elements—in-class activities, homework, exams, etc. Inconsistency is often a source of student unhappiness; a classic example is introducing more active collaborative and conceptual work in the classroom but then giving exams that primarily test on memorized facts.

Conclusion

Fundamentally changing how science is taught at major research universities remains a challenging but critical goal. Carried out successfully, it could lead to better science teaching throughout the educational system and in the process dramatically improve the learning of, and attitudes about, science throughout the population. The model discussed above demonstrates that it is possible to bring about large-scale change. Key elements of the SEI are the focus on the department and a willingness to make a one-time investment to achieve change at scale. When this is done, teaching can be far more effective and the faculty can find it more rewarding—a very encouraging sign for the future of science education.

Resources

1. Chasteen, S. V, Perkins, K. K., Beale, P. D., Pollock, S. J. and Wieman, C. E. (2010) A thoughtful approach to instruction: Course transformation for the rest of us. College Science Teaching, (in press)

2. Redish, E. (2003) Teaching physics with the physics suite, Wiley, Hoboken, NJ.

3. Smith, Perkins (2010, March) “At the end of my course, students should be able to…”: The benefits of creating and using effective learning goals. Microbiology Australia, (forthcoming in March 2010)

4. Wieman, C. E. (2007) Why not try a scientific approach to science education?. Change 39:5,

Carl Wieman, recipient of the Nobel Prize in physics in 2001 and the Carnegie Foundation's US University Professor of the Year in 2004, currently directs the Carl Wieman Science Education Initiative at the University of British Columbia and the Science Education Initiative at the University of Colorado. Prior to joining UBC, he served on the faculty at CU from 1984 to 2006 as a distinguished professor of physics and a presidential teaching scholar.

Katherine Perkins is associate director of CU's Science Education Initiative. She has worked on research-based course transformation, sustainable change, and the design and use of interactive simulations.

Sarah Gilbert has been the associate director of the Science Education Initiative at UBC since 2007, following a lengthy career in physics research and as a leader of an optical physics research group. This work has been funded primarily by the University of British Columbia and the University of Colorado, with partial funding provided by the NSF.

Subscribe Become a Subscriber   |   Access for Current Subscribers Access for Current Subscribers

In this Issue

On this Topic

©2010 Taylor & Francis Group · 325 Chestut Street, Suite 800, Philadelphia, PA · 19106 · heldref@taylorandfrancis.com