The study examines the relationship between college activities tied to social skills development and two measures of initial career success-one related to job attainment and another related to overall job quality-immediately following college graduation. Drawing on a set of data from over 18,000 undergraduate alumni who attended 68 private institutions in the United States, and controlling for a host of individual and institutional factors, findings suggest that social skills activities in college relate to early career job attainment and job quality. Additionally, findings suggest that activities linked to finding a job in the first 12 months after graduation were a subset of those related to increased quality of a graduate's first job. These effects were largely general rather than conditional on sociodemographic characteristics, including gender and race/ethnicity.

Across several decades of studies seeking to examine the effects of college on students’ careers, earnings are the outcome that has captured the most attention, as discussed by Mayhew et al. (2016), and large-scale efforts like the College Scorecard feature median earnings as the only career outcome displayed, presented alongside graduation rates and annual average costs in its college search website, as discussed by USDOE (2023). The study sought to build upon and nuance of prior research by examining the effects of higher education on students’ career outcomes based on a multifaceted conceptualization of job quality. The results reinforce the importance of pre-professional activities such as internships and participating in faculty research, although the frequency of engagement differed by major and student race. Ultimately, the findings provide new evidence in three areas: the associations between undergraduate major and the quality of one’s first job following graduation and the impact of engaging in pre-professional activities during college; the associations between undergraduate major and the quality of one's current job, and the mediated role of engaging in pre-professional activities during college and the quality of one’s first job; and the extent to which these mechanisms differ between alumnx from the global majority and those who identify as White.

Scholars and the public alike have questioned the benefits of obtaining an undergraduate education. Although research has extensively examined short-term outcomes associated with college experiences, relatively few studies have investigated non-economic outcomes beyond graduation. This paper explored the link between college experiences and post-college outcomes among 21,716 bachelor's degree recipients from 68 private institutions. Although some variation across demographics was observed, good teaching, academic challenge, and diversity experiences were consistently-and often strongly-related to alumni's perceptions of intellectual and civic growth.

David Eubanks’s “Grades and Learning” is a tale of two arguments, one explicit and one implicit. Eubanks owns the explicit argument. In “Grades and Learning” Eubanks presents a series of careful arguments and models to show that grades can help us better understand how students learn at our institutions. The unspoken assumption behind Eubanks’s argument is that grades are excluded from our assessment toolkit because the assessment community has focused on their shortcomings as measures of student learning without considering their potential strengths. To this end, Eubanks provides evidence of the potential utility of grades and gently reminds us that established assessment tools, like rubrics, also have problems.Stepping back from Eubanks’s article for the moment, we think there is an implicit reason that policymakers and accreditors have excluded grades from assessment, one that does not focus on the psychometric qualities of grades, but on the integrity of the people who grade and the institutions at which they work. In our view, thoughtful arguments of the kind that Eubanks makes will only have impact if we can have an honest conversation about one reason that grades were expelled from the assessment kingdom in the first place.For us at least, our first read of Eubanks’s article was a reminder that, “oh yeah, the nice output tables we get when we run regression in our statistics software are actually the terms of an equation.” You can tell that Eubanks has a Ph.D. in mathematics and that he thinks about models in terms of equations and random error rather than the text-filled-box-and-arrow models we usually see in conceptual discussions about assessment.After reviewing a few arguments against using “grades as data,” Eubanks describes evidence that grades are not completely random. It’s worth pausing to consider how far around the bend some of our thinking about assessment has gone when a scholar feels compelled to point out that the results of a process that countless students and subject matter experts spend so much time and effort on provides more information than just flipping a coin.Eubanks proceeds to review several obvious points that contradict practices we often use in assessment. For example, Eubanks reminds us that using multiple examples of a student’s work is less noisy and gives more thorough information about what students have learned than reviewing a single assignment with a rubric. Eubanks then rolls out a series of models that use grades to investigate things that faculty think about all the time.Eubanks describes a simple way to use grades to model student ability while taking course difficulty into account. Eubanks shows how to model whether encountering a more difficult first course in a course sequence, such as Calculus I and II, benefits some students. He describes more complicated models for how to think about the difference between grades of A and B compared to the difference between grades of C and D. He concludes by making the case for looking at how grades of F are distributed across different groups of students. At each step along the way, Eubanks shows how these models are consistent with other measures such as student evaluations, students’ ratings of their own ability, completion of AP courses, scores on rubrics, catalog descriptions of courses, and research he’s done with student grades at his institution.In our view, Eubanks’s review of these models is thoughtful and compelling. They’re models, so one can pick at underlying assumptions or decide to use different underlying assumptions. Either way, the issue isn’t whether such models are perfect; it is whether they illuminate information that we could use to improve teaching and learning. In our view, Eubanks successfully demonstrates ways we could use the vast grade data from our institutional databases to ask important questions about how students are doing at our institution.So why are grades persona non grata in assessment? This has always perplexed us. From our perspective, there’s nothing different about grades than any other potential assessment measure. Whether rubrics, grades, standardized tests, surveys, portfolios, focus groups, or narrative responses—no measurement tool is perfect. They all have strengths and weaknesses. They can all be used appropriately to draw reasonable and useful conclusions, and they can all be used inappropriately to do the opposite. Why are we so selectively fussy about grades when, if we’re honest with ourselves, most of what we do in assessment is sufficiently flawed that it would never pass muster as serious educational research? We are not setting up randomized trials with carefully controlled applications of pedagogy and content and deploying psychometrically reliable and valid instruments. Most of us are either doing the best we can to find evidence that rises above the level of intuition to try to improve the impact of our courses, departments, and programs, or we are doing the minimum necessary to satisfy accreditors.What is behind the resistance to grades and even other measures of student experience, such as surveys, that have been part of higher education for decades before assessment heated up? To say the quiet part out loud, we think it is because the forces that created our current “dos and don’ts” of assessment were shaped by widespread panic in the early 2000s that higher education was failing. Not only was it failing in its duty to create a globally competitive, educated workforce, for many it was failing at even the most basic educational tasks.In the early 2000s Secretary of Education Margaret Spellings led a commission that included leaders in industry, higher education, and educational foundations in a study of the then-current state of higher education. Here are a few quotes from the report to give you a sense of the national findings from the commission (Commission on the Future of Higher Education, 2006):The Spellings Commission was not an obscure think-tank panel. The commission’s hearings and report generated massive coverage in the higher education press along with numerous presentations and panels at higher education conferences. It reflected the views of policymakers across the political spectrum. And it was not the only report that made headlines. In 2011, Arum and Roksa’s research on student learning in Academically Adrift both emerged in and accelerated the growing skepticism about higher education. It reviewed the findings of longitudinal research on the growth of critical thinking among college students, and it produced depressing headlines. NPR led with “A Lack of Rigor Leaves Students ‘Adrift’ in College,” followed by the finding from the book that “more than a third of students showed no improvement in critical thinking skills after four years of university” (2011). Inside Higher Education and other papers across the country cited the authors’ summary quote from the book:This is the primordial higher education context from which the rapid expansion of assessment emerged. Given widespread skepticism about whether colleges and universities were providing even a modicum of learning for their students, it is not surprising that the metrics that colleges already used, like grades or graduation rates, would be off limits. If you think a restaurant is serving food that is making people sick, you would not test the restaurant by relying on the owners or the cooks to tell you how sanitary their cooking practices are. You would go into the restaurant and take samples. Likewise, if you do not think that college graduates have really learned all that much, then why would you trust the grades that helped them graduate? You need independent outside measurements to make visible what is happening inside the black box of college. That is especially important if you do not think much of anything is happening at all.In the 2000s the first round of “trustworthy” metrics that could put colleges to the test were standardized measures developed by outside test companies, such as the Collegiate Assessment of Academic Proficiency (CAAP), the Collegiate Learning Assessment (CLA), and the Measure of Academic Proficiency and Progress (MAPP). But as these tests fell out of favor with colleges and universities, we shifted to the current double-entry bookkeeping system where faculty grade student work daily using one set of tools, and then when the assessment bell rings they switch to grading using a different set of tools. Whatever you think standardized tests tell you, they are usually psychometrically well designed, and their use does not increase faculty workload.Those who wrote and embraced the Spellings Commission and other sky-is-falling reports wanted to create public comparative metrics so that governments and the people who pay for college could know which institutions were good and which were not. They hoped that public data about learning would force institutions to get better or face failure as tuition dollars and federal and state aid flowed to the institutions with better learning outcomes.But it turns out that this market-based change mechanism failed. Does anyone remember the web-based Voluntary System of Accountability (VSA) designed to display institutional data on student learning (Shulenberger et al., 2008)? No one talks about it anymore, and it has morphed into a paywalled higher education benchmarking service. As far as we can tell, learning outcomes assessment data plays no real role in the higher education marketplace. But we are left with the residue of the thinking that authored the effort to use comparative assessment data as a market-based cudgel to fix or weed out low-performing institutions. At some point, we must shuck assessment rules that are anchored in this 20-year-old policy conversation and open our minds to using any information, albeit carefully, that might help improve the impact of our assessment efforts. Especially information that we can collect with little extra burden on the dedicated people who are teaching our students.In “Grades and Learning,” Eubanks shows ways of using grade data to better understand how education is working at our institutions. The Wabash Study (Pascarella & Blaich, 2013; Blaich et al., 2016) shows that high-quality teaching, as measured with simple, low-cost survey tools, produces broad growth in outcomes like critical thinking, moral reasoning, and interest in engaging with diverse people and ideas along with other outcomes including GPA. Like grade data, the measures from the Wabash Study do not require the hard labor of developing student learning outcomes, creating and managing assessment committees, adopting standardized tests, developing rubrics, or any other duplicate system designed to validate and test the way that faculty measure learning every day.We are not suggesting that grades and simple survey data alone are good enough. Could the quality of grading be better? Absolutely! As Eubanks points out, we should help faculty improve the quality of their grades. Could the quality of teaching be better? Certainly! But, faculty development experts know how to use simple survey data to create professional development programming for faculty. In the zero-sum-game world of time in which we all work, it is important to take advantage of low-hanging data fruit before we add another layer to the work we are already doing.Finally, if we are going to generate new data on student learning to guide our improvement efforts, then we should get serious about it. As Keston Fulcher and his colleagues have pointed out, assessment efforts typically do not include even the most rudimentary design elements necessary to tell an institution whether some change in teaching, course design, curriculum, or something else in students’ learning environment has had an impact on learning (Fulcher et al., 2014). Indeed, in this article they define assessment as a set of processes that, on their own, are not used to improve student learning:In 2017, Fulcher et al. argued that learning improvement differs from assessment because it includes six qualities that are not part of most assessment efforts. These include a high level of program or department faculty involvement, a multiyear focus on a single student learning outcome, high-quality measures, standardized implementation of changes, and most importantly, measuring student learning before and after any changes in students’ learning environment to see if those changes had impact.Learning improvement, done right, is serious, hard work. It is not something a department can dash off a couple of months before an assessment report is due. On the other hand, serious collaborative work among faculty in a department or program is more likely to benefit students than the assessment Potemkin villages many of us build when we design our assessment programs. Even Potemkin villages take effort that might otherwise go toward improving the quality our educational programs.We believe it is time to use low-cost data to its maximum extent or to design and implement high-effort, high-quality studies to improve learning in our departments or programs. Those feeling ambitious can do both. But let us stop performing high-work, low-gain assessment theater.

This chapter reviews research on the effectiveness of clear and organized teaching and presents evidence that students often perceive clarity and organization differently from instructors, leading to recommendations for instructors and administrators to more fully realize the benefits of this good practice.

Assessment UpdateVolume 32, Issue 2 p. 8-12 Articles The Road to Assessment Heaven Is Paved with Good Intentions Charles Blaich, Charles BlaichSearch for more papers by this authorKathleen Wise, Kathleen WiseSearch for more papers by this author Charles Blaich, Charles BlaichSearch for more papers by this authorKathleen Wise, Kathleen WiseSearch for more papers by this author First published: 30 March 2020 https://doi.org/10.1002/au.30208Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat No abstract is available for this article. Volume32, Issue2March/April 2020Pages 8-12 RelatedInformation

Assessment UpdateVolume 30, Issue 6 p. 4-5 Articles Highlights of the 2018 HEDS Annual Conference Charles Blaich, Charles BlaichSearch for more papers by this authorKathy Wise, Kathy WiseSearch for more papers by this author Charles Blaich, Charles BlaichSearch for more papers by this authorKathy Wise, Kathy WiseSearch for more papers by this author First published: 05 December 2018 https://doi.org/10.1002/au.30151Citations: 1Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat No abstract is available for this article.Citing Literature Volume30, Issue6November/December 2018Pages 4-5 RelatedInformation

Extensive research on college impact has identified a range of practices that enhance students’ academic outcomes. One practice—clear and organized instruction—has received increasing attention in recent research. While a number of studies have shown that clear and organized instruction is related to a range of postsecondary outcomes, researchers have not considered the mechanisms that link this educational practice to student outcomes. In this study, we draw on the constructivist theory of learning to identify potential mechanisms that may explain the relationship between clear and organized instruction and academic performance. Results from the Wabash National Study of Liberal Arts Education, including an analytical sample of 7116 students attending 38 four-year institutions in the USA, indicate that three mechanisms examined—faculty interest in teaching and student development, academic motivation, and academic engagement—explain almost two-thirds of the relationship between clear and organized instruction and first-year GPA. When students experience greater exposure to clear and organized instruction, they perceive their faculty as being more invested in their learning and development, and they report being more academically motivated and engaged in their studies. Moreover, students who enter college less academically prepared benefit more from exposure to clear and organized instruction.

Although a growing body of research has demonstrated the value of interacting with diverse peers, a number of questions remain about the relationship between the quality of students' diversity interactions and their cognitive development. In a longitudinal study following 3 cohorts of students from entry into college through the end of their 4th year, we examined how students' positive and negative diversity interactions were related to 2 different outcomes: need for cognition and critical thinking skills. The results indicated that negative diversity interactions were strongly related to both outcomes, and that was the case for students of color and their white peers. Positive diversity interactions, on the other hand, were related to students' need for cognition but not their critical thinking skills, and these interactions disproportionately benefitted white students. We conclude by considering the implications for understanding students' cognitive development and implementing policies and practices that can facilitate positive outcomes on college campuses.

While racial inequalities in college entry and completion are well documented, much less is known about racial disparities in the development of general collegiate skills, such as critical thinking. Using data from the Wabash National Study of Liberal Arts Education, we find substantial inequality in the development of critical thinking skills over four years of college between African American and White students. The results indicate that these inequities are not related to students’ academic experiences in college but are substantially related to their experiences with diversity. These findings have important implications for understanding racial inequality in higher education and considering strategies for addressing observed disparities.

In higher education, it is sometimes hard to ignore the one-two doom and gloom combination of shrinking budgets and expanding accountability. But we are also in the midst of an exciting conversatio...

This chapter examines the role of systems and consortia in scaling and implementing undergraduate research through a study of the efforts of six systems and consortia working together with the Council on Undergraduate Research.

Using a multi-institutional sample of undergraduate students, this study found that the relationships between engaging in high impact/good practices and liberal arts outcomes differ based on students' precollege and background characteristics. Findings suggest that high impact/good practices are not a panacea and require a greater degree of critical evaluation by higher education scholars.

This study estimates the effects of a deep approaches to learning scale and its subscales on measures of students' critical thinking, need for cognition, and positive attitudes toward literacy, controlling for pre-college scores for the outcomes and other covariates. Results suggest reflection is critical to making gains across the outcomes.

Effects of Diversity Experiences on Critical Thinking Skills Over 4 Years of College Ernest T. Pascarella (bio), Georgianna L. Martin (bio), Jana M. Hanson (bio), Teniell L. Trolian (bio), Benjamin Gillig (bio), and Charles Blaich (bio) The benefits of student engagement in diversity experiences on a range of college outcomes have been well documented (e.g., Chang, Denson, Saenz, & Misa, 2006; Gurin, Dey, Hurtado, & Gurin, 2002; Hurtado, 2001; Jayakumar, 2008; Kuklinski, 2006). However, the potential influence of involvement in diversity experiences during college on the cognitive and intellectual outcomes of post-secondary education is only beginning to be understood (Bowman, 2010). Gurin et al. (2002) made a convincing argument for why exposure to diversity experiences might foster the development of more complex forms of thought, including the ability to think critically. Drawing on research that spoke to the social aspects of cognitive development, they pointed out that students will be more likely to engage in effortful and complex modes of thought when they encounter new or novel situations that challenge current and comfortable modes of thinking. This often can happen in classroom settings, but also can occur in other contexts when students encounter others who are unfamiliar to them, when these encounters challenge students to think or act in new ways, when people and relationships change and produce unpredictability, and when students encounter others who hold different expectations for them. Consistent with the argument by Gurin et al. (2002), a series of studies by Dey (1991), Chang et al. (2006), Gurin (1999), Hurtado (2001), and Kim (1996) have suggested that exposure to racial and cultural diversity during college is significantly linked to such outcomes as student self-reported gains in "problem solving," "critical thinking," "cognitive development," and "complexity of thinking." This important initial work alerted scholars to the possibility that a considerable range of cognitive/intellectual growth during college might be fostered by a student's exposure to diversity experiences. However, although student self-reported gains can be revealing and important outcomes, there are some serious concerns about their actual validity (e.g., Bowman, 2011). Inquiry that attempts to estimate the impact of diversity experiences on the development of cognitive and intellectual skills using more objective standardized measures than student self-reported gains is extremely limited. Two early investigations, analyzing the first year of the nearly 20-year-old National Study of Student Learning longitudinal database, addressed the link [End Page 86] between diversity experiences and critical thinking skills—as measured by the Critical Thinking Test of the Collegiate Assessment of Academic Proficiency (Pascarella, Palmer, Moye, & Pierson, 2001; Terenzini, Springer, Yeager, Pascarella, & Nora, 1994). These early studies indicated that, net of important confounding variables (such as precollege critical thinking skills), individual diversity experiences such as attending a racial—cultural workshop and making friends with someone of a different race were significantly and positively linked to first-year gains in critical thinking scores. The Pascarella et al. (2001) investigation, however, also suggested that the positive efiècts of involvement in such diversity experiences on growth in critical thinking were more pronounced for White students than for students of color. The most recent work estimating the influence of diversity experiences on the development of cognitive skills during college analyzed the first year (2006–2007) of the Wabash National Study of Liberal Arts Education (WNS), a longitudinal study focusing on the effects of liberal arts education (Loes, Pascarella, & Umbach, 2012). Loes et al. (2012) found that, when important confounding experiences (e.g., precollege critical thinking skills and tested academic preparation) were taken into account, students' diversity experiences had no overall significant link with first-year gains on a standardized measure of critical thinking skills. However, consistent with the earlier findings of Pascarella et al. (2001), the positive effect of diversity experiences on critical thinking was significantly more pronounced for White students than for students of color. Loes et al. also reported that the effects of diversity experiences were more important for the students least academically prepared for college, as indicated by relatively low ACT scores. The present study sought to determine if the effects of exposure to diversity experiences on critical thinking skills extended beyond the first year of college and if these...

This study analyzes longitudinal data from 17 four-year institutions in the United States to determine how the distinctive instructional and learning environment of American liberal arts colleges accounts for the positive impact of liberal arts college attendance on four-year growth in critical thinking skills and need for cognition. We find that, net of important confounding influences, attending an American liberal arts college (vs. a research university or a regional institution in the United States) increases one’s overall exposure to clear and organized classroom instruction and enhances one’s use of deep approaches to learning. In turn, clear and organized classroom instruction and deep approaches to learning tend to facilitate growth in both critical thinking and need for cognition—thus indirectly transmitting the impact of attending a liberal arts college.

Since its inception, American postsecondary education has had a core belief in the desirability of a kind of individual development that prepares one for a complete life, and it has placed great fa...

Design and Analysis in College Impact Research:Which Counts More? Ernest T. Pascarella (bio), Mark H. Salisbury (bio), and Charles Blaich (bio) Over the last several decades student affairs and assessment scholars who study college impact have utilized a number of different research design and statistical procedures in an attempt to control for the characteristics and propensities that lead students to self-select themselves into a particular intervention or experience. This is particularly important because such characteristics and propensities may seriously confound any estimate of the effect of the intervention or experience itself. By far the most common method used in the college impact literature to date has been covariate adjustment, based on different multiple regression approaches (Pascarella & Terenzini, 2005). This approach relies on statistical control to remove or partial out the confounding effects of student self-selection. Recently, however, there has been considerable criticism of covariate adjustment based on the argument that its estimate of the effect of an intervention or experience can be biased. Rather than relying on regression-based covariate adjustment techniques, a number of scholars have suggested the use of propensity score matching as a more effective analytical approach for controlling the effects of demographic, attitudinal, or other factors that might increase or decrease students' likelihood of self-selecting into a given treatment of interest and, thereby, isolating the effect of the treatment itself (e.g., Reynolds & DesJardins, 2009; Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). In this study we employ both covariate adjustment and propensity score matching to estimate the causal influence of an example intervention—the first year of attendance at a liberal arts college (as opposed to another type of 4-year institution). Specifically we estimated the effect of liberal arts college attendance on three cognitive outcomes. We examined the estimates yielded by these two analytical approaches under different research design assumptions—with and without a precollege measure of each outcome. Our purposes were to determine the comparability of causal estimates using covariate adjustment and propensity score matching, and to examine how these estimates might be affected when different research designs are employed to study college impact. The focus of the study was not specifically on understanding the effects of liberal arts colleges, rather, estimating of the effects of liberal arts colleges versus other 4-year institutions is used only as an example. The approaches we explored could have relevance to estimating of the effects of a broad range of between-college and within-college interventions or experiences. [End Page 329] Methods Sample and Data Collection We analyzed data from the first year of the Wabash National Study of Liberal Arts Education (WNS), which is a longitudinal pretest-posttest investigation of the effects of liberal arts experiences on a range of college cognitive and noncognitive outcomes thought to be associated with undergraduate liberal arts education. The colleges and universities participating in WNS represent a diverse selection of institutions, varying in institutional characteristics such as type and control, selectivity, enrollment, and location within the United States. For our data analysis sample we chose the 2006 WNS iteration, which collected extensive precollege data on students in early Fall 2006 and again in Spring 2007. Our analyses were based on first-year, full-time undergraduates attending 17 different 4-year institutions (11 liberal arts colleges, 3 research universities, and 3 regional institutions). We estimated the effects of the first year of liberal arts college attendance on three cognitive/learning orientation outcome measures: critical thinking skills, need for cognition (a measure of continuing motivation for learning), and positive attitude toward literacy activities. Because of matrix sampling in part of the WNS design, complete precollege and end-of-first-year data were available for 1,377 students on one dependent measure (critical thinking skills) and 2,872 students on the other two dependent measures (need for cognition and positive attitude toward literacy). Although there are clearly limitations with respect to the external validity or generalizability of results obtained with the 17-institution WNS sample, our concern was with estimating the internal validity of the effects of liberal arts colleges. Moreover, as our results are intended for didactic rather than inferential purposes, concerns with generalizing the results of the analyses...