There's no news like bad news: It travels fast and far. The publication of Richard Arum and Josipa Roksa's influential new book, Academically Adrift: Limited Learning on College Campuses (2011), the findings of which were summarized in an article by the same authors in the March/April Change, has caused a national furor centering around their multi-institutional findings that the general impact of college on student intellectual development is considerably less than stellar.
The book's core conclusion that postsecondary education has little effect on student learning is based largely on three major outcomes: small average gains during college on a standardized measure of critical thinking and complex reasoning (the Collegiate Learning Assessment, or CLA), a large percentage of students failing to make individually significant gains on the CLA during college, and low levels of engagement in serious academic work such as studying and writing. Whether or not they intended to, Arum and Roksa have thrown down a sizable gauntlet. If their findings are robust or broadly generalizable, American postsecondary education may be facing a period of considerable soul searching.
But just how robust are the findings of Academically Adrift (hereafter AcAd) and the other related reports released by Arum and Roksa? This is what replication studies, a time-honored way of testing the generalizability of important findings, are for. While not unheard of, independent replication of findings is not a particularly common event in research on the effects of college on students (Pascarella & Terenzini, 1991, 2005). But the authors of this article — researchers at the Center of Inquiry (COI) at Wabash College and the Center for Research on Undergraduate Education (CRUE) at the University of Iowa who jointly conducted the quantitative component of the Wabash National Study of Liberal Arts Education (WNS) — think that because of its national visibility, AcAd is undoubtedly worth replicating. And because Arum and Roksa were transparent about their research methods, conducting such analyses with the WNS was reasonably straightforward.
Ernest T. Pascarella (firstname.lastname@example.org) is professor and the Mary Louise Petersen Chair in Higher Education at the University of Iowa, where he also co-directs the Center for Research on Undergraduate Education (CRUE).
Charles Blaich (email@example.com) has been at Wabash College since 1991, where he is the director of the Center of Inquiry in the Liberal Arts; he also directs the Higher Education Data Sharing Consortium.
Georgianna L. Martin is a PhD candidate in the Student Affairs Administration and Research program research assistant in CRUE. The research on which this study was based was supported by a generous grant from the Center of Inquiry at Wabash College to the Center for Research on Undergraduate Education at the University of Iowa.
Jana Hanson is a PhD student in the Higher Education and Student Affairs program at the University of Iowa; research assistant in CRUE. The research on which this study was based was supported by a generous grant from the Center of Inquiry at Wabash College to the Center for Research on Undergraduate Education at the University of Iowa.