Where Dreams Go to Die
Mary Lowry remembers sitting in Nicole Gray's office at Foothill College sobbing because she was convinced that she would never realize her goal of earning a four-year college degree. “I thought something was wrong with me,” she said. “No matter how hard I tried—and I had really tried hard—I could not pass a math class.” After testing into developmental mathematics and failing algebra for the third time, she was ready to give up. “I was embarrassed,” she said.
She had been able to do well in all her classwork in high school except math; the same was proving to be true since she had enrolled in community college. “Math just wouldn't click; I just couldn't get it,” she said. In her early 40s when she entered Foothill, this was going to be her last attempt. Math was standing in the way of her dream of having a career where she could “make a difference.”
Lowry is not the only student whose dreams have been deterred in this way. Community colleges are dedicated to the proposition that students can realize upward mobility through education and that learning is possible at any point in our lives. There, many students find success, but many others find that it eludes them.
Developmental mathematics is one of the most serious barriers to educational and economic achievement. Over 60 percent of all students entering community colleges in the United States are required to complete remedial/developmental courses as a first step towards earning associate's or bachelor's degrees. Then, to earn a degree, certificate, or license, students usually must complete at least one college-level math course. A staggering 70 percent of these students never complete the required mathematics courses, blocking their way to higher education credentials and with them, a wide array of technical and related careers.
It was this reality that prompted the Carnegie Foundation for the Advancement of Teaching to develop a program of work that, after only a year in community-college classrooms, has tripled the success rate for developmental-mathematics students in half the time. Lowry was one of those success stories. At Professor Gray's recommendation, she enrolled in Rachel Mudge's Statway class, one of Carnegie's new math pathways, and she did well.
“I was astonished,” Lowry said. “I not only began to understand math—I understood why I had not been able to figure it out before, and I knew it wasn't my fault.” She is has now been accepted to San Jose State University and is on her way to earning a bachelor's degree in sociology.
The pathway that helped Lowry finally find success in mathematics was not available until recently. In 2010, after a year of fund raising and planning, Carnegie formed a network of community colleges, professional associations, and educational researchers to develop and implement the Community College Pathways (CCP) program. The program is organized around two structured pathways, one in statistics (known as Statway™) and the second in quantitative reasoning (Quantway™).
Statway and Quantway are called pathways because they are complex instructional systems that include a common curriculum, pedagogy, and student supports. They differ from traditional math courses in that their approaches, topics, and contexts enable students to think and reason quantitatively, unencumbered by memories of past failures. Statistics and concepts of quantitative reasoning are in the foreground, with the developmental mathematics concepts required to support statistical and quantitative understanding integrated throughout.
Rather than the traditional student struggle through a required two-year sequence of courses leading to calculus, students and faculty are joined in a common, intensive pursuit of a shared goal—for students to achieve college math credit in one year. Statway is designed as a one-academic-year course that allows students to simultaneously meet their developmental mathematics requirement and receive college-level mathematics credit in statistics. Quantway is two separate semester courses: Quantway 1 fulfills the requirements for students' entire developmental mathematics sequence, and Quantway 2 allows them to receive college mathematics credit.
When Carnegie selected the problem of developmental mathematics to work on, it had much to do with the background of its new president. When Anthony Bryk was selected as Carnegie's ninth president in 2008, he already had a very personal interest in and understanding of mathematics that grew out of his own history.
The son of working-class parents who had high aspirations for their child, Bryk's aptitude for math, especially statistics, became a gateway to opportunity for him. Moreover, prior to coming to the Foundation, he worked for almost two decades at the University of Chicago, focusing on helping students develop the academic knowledge and skills to stay in and graduate from high school and move on to postsecondary education.
Many of these students wound up going to community colleges where, in many urban areas like Chicago, upwards of 80 percent of students are assigned to developmental-mathematics courses. Eighty percent of those students never complete those courses successfully.
“If you do not get out of developmental mathematics,” Bryk said, “you cannot acquire credits to transfer to a four-year institution, and you often cannot get access to vocational and technical training programs. The bumper sticker for this problem is, ‘Developmental mathematics is where aspirations go to die.’”
How the Foundation worked on this high-leverage problem is the focus of this article.
A New Direction for Education Research
Close to the time of the Carnegie presidential search, Bryk and his colleague Louis Gomez had published a paper that outlined a new approach to solving problems of practice in education. Bryk explained this approach in a 2009 article in Kappan magazine:
This activity should be organized around the core problems of practice embedded in the day-to-day work of improving teaching and learning and in the institutions where teaching and learning take place. Making progress in addressing these educational problems requires a commitment to a rapid prototyping process by which researchers and practitioners co-develop innovations, try them in schools and other learning contexts, and then refine and try them again. This new infrastructure demands an engineering orientation in which adaptability to local contexts is a direct object of study. In this regard, knowing that a program can work is not good enough; we need to know how to make it work reliably over many diverse contexts and situations. This means accumulating a rigorous knowledge base on practice improvement where the real test of adequacy is its capacity to advance demonstrable, broad-based improvements in teaching and learning.
Bryk, Gomez, and Alicia Grunow, now a senior managing partner at Carnegie, refined this thinking in an essay, “Getting Ideas into Action: Building Networked Improvement Communities in Education.” In it, they proposed a science of improvement research and introduced the idea of a Networked Improvement Community (NIC) that organizes the collective action needed to solve complex educational problems.
They posited that education does not lack for good people working on education improvement as much as it does smarter systems to support social learning. There are many organizations doing good work and trying to solve the failure-rate problem in developmental math. However, there has been no way to reliably implement these innovations at scale.
Parents, legislators, and even the President of the United States are demanding that something be done to improve K-12 test scores and college completion rates. “We are asking much more of our educational institutions than ever before,” Bryk said.
We want our schools and colleges to be more effective, realizing more ambitious academic outcomes for many more students. We want them to be more engaging and responsive to the varied interests of an increasingly diverse student population. We are also requiring that education institutions be more efficient in their use of resources, as public funding is highly stressed.
Bryk terms these the “triple aims of education improvement,” noting that advancing on any one of these goals alone would be ambitious; expecting to accomplish all three is unprecedented.
A New Kind of Partnership
Putting the day-to-day work of educators at the center of reform efforts is key to this model of quality improvement, which draws on over a half century of work in other industries and sectors, both in the US and abroad. So—in collaboration with faculty, institutional researchers, academic leaders, instructional designers, technologists, and other academic experts—Carnegie formed NICs to engage in improvement research focused on the community colleges' developmental-math problem.
Researchers and educators rarely work across roles or institutional boundaries to solve problems in education, and there are few structures to collect, study, and share information about effective pedagogy and student learning. “We have heard a lot about this notion of bridging research and practice, but normally when you see it happen, research stays firmly on one side of a line, practice stays firmly on the other, and we have a tiny space in which they sometimes talk to each other. In improvement research, we bring people to the middle to work together,” said Grunow.
The $13 million initiative, funded by six foundations, built a network of 27 community colleges and three universities across eight states to develop two new mathematics pathways. Carnegie coordinated the work with programs such as Achieving the Dream and the California community-college system's Basic Skills Initiative and reached out to national organizations such as the American Association of Community Colleges and American Mathematical Association of Two-Year Colleges. As the work developed, Carnegie also assembled nationally recognized leaders from the Mathematical Association of America, the American Mathematics Association of Two-Year Colleges, the American Mathematics Society, the American Statistical Association, and the National Numeracy Network to establish ambitious learning goals.
That community continues to grow; at this writing it includes 43 colleges and universities in 11 states, with another 26 queued up for participation. The pathways reach the students whom community colleges need to serve well. A disproportionate number are minority students, from families whose primary language is not English, and the first in their families to pursue a college degree. Lowry is one of some 4,600 students who have now benefitted from this program.
Professors Mudge and Gray joined an initial cadre of 80 Statway and Quantway faculty, which soon grew to 140 members, using common curricula, common assessments, innovative instructional approaches, and common online platforms. From the beginning, these faculty not only contributed to the design and continuous refinement of the materials but also experienced a sea change in the way they taught because of their involvement in the network.
A critical outcome of the faculty experience is the shift in their professional identity from the traditional one of an individual instructor working privately in his or her own classroom to that of a scholar of teaching and learning engaged in collaborative, network-wide improvement. Faculty members are supported in deepening their own instructional practice as they engage in disciplined inquiries about improving teaching and learning.
Later, both Gray and Mudge took sabbaticals from their teaching and joined Carnegie to provide faculty voices within the development team and to serve as content experts with a special knowledge of the needs of developmental students.
After the first year of implementation, results were very encouraging. As Carnegie's first report by Scott Strother and his colleagues showed, 51 percent of Statway students successfully completed the full pathway, earning college credit within one year. And 56 percent of students successfully completed the first semester of Quantway, fulfilling their developmental-mathematics requirement. The majority of these students would otherwise have had to take two or more mathematics courses to complete their developmental-mathematics requirement before enrolling in a college-level mathematics course.
Bryk explains that guiding the work of the NICs is a highly structured, disciplined form of inquiry. “Carnegie starts with a working theory of the problem and a working theory of what solutions might look like for that problem. Together, these form a theory of practice improvement, which has drivers with common measures that guide the work.” Inquiry protocols structure how to gather evidence and test whether certain change ideas are actually improvements.
All of this exists within what sociologists call a social proof network. The community agrees about what is its best knowledge with regard to some specific problem, recognizing that “best knowledge” is always provisional.
In order for these networks to actually function, though, they need what is termed a hub, a structural element of the NIC that allows this knowledge to be communicated, collected, and tested under various circumstances. “These features make it appropriate to refer to this work as an improvement science, because it has all the basic characteristics and social organization of a scientific community. The main difference is its very practical aim: improving practice against some very well-specified problem.”
Three concerns guide the work of continuous quality improvement science. As Bryk recently explained to a group that came to Carnegie to learn the network approach,
The first concern is: Do I actually know how to execute a promising improvement idea? We have lots of good ideas about how things could be better, but often in education we move to immediately implement reform ideas at a very large scale, even when knowledge about how to execute these ideas is lacking. The second concern relates to available human and social resources: Do we actually have the capacity to execute this at the scale we aspire to? Even if we know how to do this, we need to make sure that we have the people and organizational structure to actually execute at the scale envisioned. The third concern is political in nature: How are practitioners likely to receive what we are bringing to them? Are they ready, indifferent, or resistant?
In a quality-improvement framework, these elements mesh to create meaningful improvement: We have to develop technical knowledge; we have to build the conviction that if we go along this path, we will accomplish something of value; and we have to build the capacity of people to work with that knowledge. Often this means that improvement processes begin with small, rapid tests of change instead of large-scale implementation.
The small tests of change build technical knowledge about how to make something work. They also build will; as people experience success, they become advocates. Finally, they develop human capital; the people who are engaged in these tests are learning how to make this work.
Six core principles guide Carnegie's work. They are:
Make the work problem-specific and user-centered. It starts with a single question: “What specifically is the problem we are trying to solve?” It engages key practitioners early and often as co-developers.
Variation in performance is the core problem to address. The critical issue is not simply what works but rather what works for whom and under what set of conditions. Local context considerations lead to variability in implementation in ways that reduce effectiveness. Aim to advance efficacy reliably and at scale, adapt to local contexts, but test those adaptations to warrant them as improvements.
Observe the system that produces the current outcomes. It is hard to improve what you do not fully understand. See how local conditions shape work processes. Make your hypotheses for change public and clear.
We cannot improve at scale what we cannot measure. Embed measures of key outcomes and processes to track whether changes are improvements. We intervene in complex organizations. Anticipate unintended consequences and measure them too.
Anchor practice improvement in disciplined inquiry. Engage rapid cycles of plan, do, study, act (PDSA) to learn fast, fail fast, and improve quickly. That failures occur is not the problem; that we fail to learn from them is.
Accelerate improvements through networked communities. Embrace the wisdom of crowds.
The most difficult and yet probably the most important principle is that variation in performance is the core problem to address. Carnegie Senior Vice President Paul LeMahieu explains it this way:
The traditional empirical warrant substantiating a program as effective holds only in so far as the program is replicated exactly as tested. But programmatic solutions invariably must be adapted as they move into new contexts. The real challenge of implementation, then, is to figure out how to thoughtfully accommodate local contexts while remaining true to the core ideas to ensure improvements in practice that carry the warrant of effectiveness. Rather than emphasizing fidelity of implementation (do exactly what they say to do), this perspective directs attention toward integrity in implementation. It places a premium on understanding the core principles and routines undergirding an improvement initiative and remaining true to these while accommodating local needs and circumstances.
Specifying the Problem and Its Probable Solutions
How is this improvement approach manifested in practice? It begins with identifying the specific problem a network is trying to solve, then analyzing the system that produces current results, then recognizing the key drivers that the network would have to address in order to solve the problem.
In the case of the CCP initiative, these drivers included the irrelevance of the current mathematics curriculum to student's lives. So the curriculum needed to change. The pedagogy also had to be addressed. There was ample evidence that teaching students the same content over and over again in the same way doesn't work. Students who failed to learn the basics of ratios/fractions and proportions in elementary school and who did not benefit from attempts to re-teach it the same way in middle and high school were unlikely to learn this material presented in the same fashion in a community college.
The network would have to attend to how students learn. Drawing on findings in cognitive science and learning research, pathway lessons are built around rich problems. Students struggle with them and then come to see how basic tools of algebra, statistics, data visualizations, and analysis can help them understand the problems better. Such instruction constantly seeks to make explicit the connections between the big ideas of mathematics and the specific tools that we might apply to different problems.
It is also important to attend closely to who the students are in order to engage and ultimately to educate them. Students placed in developmental math often come to the classroom with what Carnegie Senior Fellow Jim Stigler calls “math scar tissue”—the residue of years of failure in mathematics courses.
But we now know from extensive research that math ability is not a fixed attribute but malleable; with effort and deliberate practice, new skills and understandings can be acquired. Some students stop trying soon after the course begins, but most work hard in developmental-math classes—studying long hours, nights, and weekends—yet do so using ineffective strategies.
Productive persistence—tenacity supported by appropriate skills—is a key driver of success. A number of psychological interventions were integrated into the instruction to make a difference in students' tenacity and their use of good strategies.
Language and literacy barriers that impede instruction and learning were another key impediment. Carnegie examined how this barrier was created in textbooks, curricular materials, homework assignments, lectures, and discussion activities and addressed it in the pathways' design.
The Student Experience
Mary Lowry recognized the uniqueness of her Statway experience from the first day. The focus on conceptual understanding applied to real-world problems was especially important for her. “I never knew what math was for; I thought I was just supposed to memorize a lot of equations and it would someday become clear to me. Working with the Statway materials and having the math embedded in real problems finally turned on that light bulb.”
She also said that the pathways group work was essential. “We worked on problems together and we became like a family; I didn't want to let the others down so I probably worked harder.” Creating a sense of belonging—a key predictor of student success—helped students realize that math class was not a foreign place for them.
And the sooner they learn this, the better. A “starting-strong” package includes a set of initial classroom routines targeted at reducing anxiety, increasing interest in the course, and forming supportive social networks. One key activity is a direct-to-student growth mindset intervention, a reading and writing exercise designed to challenge students' view that being a “math person” is a fixed attribute, delivered either in class or via the Internet during the first week of the course.
Lowry said that when she was told through this intervention that research showed that she could “grow her brain,” she was skeptical. However, the proof was in how well she did in the class. “After being told that I could do this over and over and then truly experiencing it, I became a believer. I had never had that kind of constant support in a class before. And now I know I can do math.”
Lowry's and her classmates' progress and struggle were monitored throughout the course by means of periodic short surveys intended to inform faculty about changes that needed to be made to both the lessons and the pedagogy as the term progressed. They also identified students who needed immediate interventions, so that they didn't fall behind or get lost.
The Faculty Experience
Faculty involvement in the networked community began early in the project. When a first version of Statway materials was ready to be tested, a group of faculty, lessons and modules in hand, spent several intense days at Carnegie in what they termed “the cave.”
After working almost around the clock, they left behind a conference room strewn with half-filled coffee mugs, crumpled soda cans, and smudged pizza boxes. They also walked away with a set of revised materials that were threaded through with needed student supports. As the lessons went live in classrooms, other faculty joined in webinars, conference calls, and on-site meetings to further improve the pathways. That work goes on to this day.
Michelle Brock was one of the “cave dwellers.” Brock said that her participation was part of her commitment to collaborate with other faculty members in the co-development of Statway. “I believed in Carnegie's vision that Statway had the possibility of netting amazing results for student success in developmental mathematics, and those of us involved in this initiative from the beginning were committed to going beyond the ‘what's in it for me’ thinking.”
NIC faculty work together to plan instruction, observe each other teaching, and identify the most difficult obstacles that stand in the way of student success in traversing the pathways. As all of this went live in different classrooms and colleges, Carnegie began to assemble a body of evidence about the variability in student outcomes and how the pathways work in different contexts.
NIC faculty participate in calls; campus, regional, and national meetings; webinars; and one-on-one conversations with Carnegie staff. Some test particularly difficult lessons, identify specific problems, and hypothesize improvements in the materials and/or their implementation. All of this learning is shared throughout the network.
Kristin Spiegelberg, who teaches at Cuyahoga Community College, said that usually a professor's work is done in isolation. “We go into our classrooms by ourselves and figure it out. In a NIC, we are working with other faculty, interacting with other teachers, both in my own college and across the country, and we learn from each other.”
Improvement Science in Action
This past year, subnetworks of faculty are working on specific improvement challenges: productive persistence, further Quantway and Statway development, pathways expansion, and enrollment. Subnetwork work cuts across multiple colleges (and sometimes pathways too) and includes content experts, practitioners, and researchers. Other work involves iterative tests of change (plan-do-study act cycles) that might run over a full academic year. All the participants are using the tools of improvement science to do this work.
For instance, the productive-persistence subnetwork is addressing non-academic drivers that influence whether a student remains in the classroom and is successful. The team used results from previous surveys and the research literature to identify three specific concerns that affect students' social ties in the classroom: a sense of belonging, a sense that professors care about them, and their comfort in asking questions. These drivers were selected because data from pathways students indicated that these were closely related to success (earning a grade of “C” or better) and persistence rates (students enrolling in the next term of Statway).
These drivers focused the work on areas that the NIC determined could be significantly improved. In the process, subnetwork members learned a new way to conduct practice research, to gather information about their students, and to look more deeply at productive persistence. Based on this work, subnet members prototyped “change ideas” related to one or more of the three drivers and then conducted PDSA cycles linked to those changes.
For example, subnetwork faculty have initiated improvement research on three change ideas to address students' sense of not belonging:
When students miss class, faculty rarely have a systematic way of reaching out in order to understand why they have been absent and to encourage them to attend future classes. One faculty member in the productive-persistence subnet has developed routines and scripts for emailing absent students. These scripts change over the course of the semester as the relationship between the faculty member and the student evolves. Characteristics of the emails were tested through PDSAs and revisions were made. The faculty member found that attendance improved and has recommended that this script be further tested.
Another faculty member sought to build a sense of belonging by making students responsible for each other's presence in a group noticing routine, which consists of three stages. In the first stage, the faculty member groups students, who get to know each other outside of the math context. In the next stage, groups are responsible for reporting to the faculty who is absent each day. In the final stage, groups take responsibility for contacting students who are missing, encourage them to attend future classes, and give them any materials or information that they missed. Attendance remained strong across the semester (an 85 percent median attendance rate)—quite different from past experiences with similar student groups.
In the starting-strong package, faculty members are advised to give roles (e.g., monitor, reporter, facilitator) to students in the group. Members of the productive-persistence subnet developed and tested a routine for effective role functioning. During group work, students are given laminated cards that describe the expectations for their assigned roles, which rotate throughout the course. The student acting as the facilitator of the group assesses the performance of each student relative to the role they played on that day. The scores are then given to the faculty and incorporated into participation grades. The two faculty members who tested the strategy found that students worked together more effectively and that attendance was strong (a 92 percent median attendance rate).
Having demonstrated promise in this first-stage test of new classroom routines, these ideas are now candidates for further testing across the NIC. As these routines are taken up by new faculty and in different colleges, Carnegie expects further refinements will occur.
The ultimate goal is to assure efficacy under the broadest possible conditions that confront different faculty and students. If they are successful, these innovations will subsequently take on the status of kernel routines—the core set of materials and practices that have demonstrated widespread efficacy and are now broadly shared and used by NIC participants.
After participating in something that is changing students' lives, many faculty have become champions and promoters of the work. Carnegie staff, too, have been eagerly spreading the good news. Over the course of the next three years, Carnegie staff members hope to quadruple the number of participating campuses and increase the number of faculty and students in the pathways by a factor of ten.
The aim is that by 2016, Statway and Quantway will be taught by nearly 500 faculty in 120 institutions to over 17,000 students annually—that over 25,000 students' lives will have been affected and that the numbers will rapidly grow thereafter. Equally important, Carnegie will have institutionalized the practices of evidence-based quality improvement.
All Improvement is Personal
Even though Carnegie has achieved success through the power of networks and the use of the tools of improvement science, the work is personal. For Mary Lowry, it has been “life changing.” For faculty member Duane Benson at South Georgia College in Douglas, Georgia, who felt isolated in his small community, it has provided both personal and professional growth, in that he has been exposed to experts in his field and others who understand what he is attempting to achieve.
Aaron Altose, another faculty participant, adds that the work he does makes a difference beyond his classroom. “Each one of my students will be able to make a change for the better within their own lives,” he said. “Perhaps the changes won't be that grand, but the small impacts on their friends, family, and people around them contribute to the improvement of my community.
1. Bryk, A. S. (2009) Support a science of performance improvement.. Phi Delta Kappa 90:8, pp. 597-600.
2. Bryk, A. S., Gomez, L. M. and Grunow, A. Hallinan, M. (ed) (2011) Getting ideas into action: Building networked improvement communities in education.. Frontiers in sociology of education, pp. 127-162. Springer, New York, NY.
3. Silva, E. and White, T. (2013) Pathways to improvement: Using psychological strategies to help college students master developmental math, Carnegie Foundation for the Advancement of Teaching, Stanford, CA. Retrieved from http://www.carnegiefoundation.org/sites/default/files/pathways_to_improvement.pd
4. Strother, S., Van Campen, J. and Grunow, A. (2013) Community college pathways: 2011–2012 descriptive report, Carnegie Foundation for the Advancement of Teaching, Stanford, CA. Retrieved from http://www.carnegiefoundation.org/sites/default/files/CCP_Descriptive_Report_Year_1pd
Gay M. Clyburn (email@example.com) is associate vice president for public affairs and continuing programs at the Carnegie Foundation for the Advancement of Teaching. The work described in this article is supported by the Carnegie Corporation of New York, the Bill & Melinda Gates Foundation, the William and Flora Hewlett Foundation, the Kresge Foundation, and the Lumina Foundation, in cooperation with the Carnegie Foundation for the Advancement of Teaching.