Papers and publications

change view


CEPA Working Papers  

Essay Content is Strongly Related to Household Income and SAT Scores: Evidence from 60,000 Undergraduate Applications

AJ Alvero, Sonia Giebel, Ben Gebre-Medhin, anthony lising antonio, Mitchell L. Stevens and Benjamin W. Domingue

College application essays in an era of machine reading

We utilize a corpus of 240,000 admissions essays submitted by 60,000 applicants to the University of California in November 2016 to measure the relationship between the content of application essays, reported household income, and standardized test scores (SAT) at scale. We find that essays have a stronger correlation to reported household income than SAT scores.

Read this paper

AERA Open, 2021  

Studying Undergraduate Course Consideration at Scale

Sorathan Chaturapruek, Tobias Dalberg, Rene F. Kizilcec, Marissa E. Thompson, Sonia Giebel, Monique Harrison, Ramesh Johari, and Mitchell L. Stevens

What role does course consideration play in college students' pathways?

Using digital trace data to observe this process at scale at a private research university, together with qualitative student interviews, we provide a novel empirical study of course consideration as part of the course selection process.

Read this paper

Journal of Higher Education, 2021  

Ambiguous Credentials: How Learners Use and Make Sense of Massively Open Online Courses

Krystal Laryea, Kathy Mirzaei, Andreas Paepcke & Mitchell Stevens

What good is a MOOC?

As low-status academic offerings purveyed by high-status institutions, massively open online courses (MOOCs) are ambiguous credentials. In interviews with 60 people who devoted substantial time to at least one MOOC between 2014-2017, we find that people use MOOCs to build skills for application at work and home, build relationships, navigate life transitions, and enhance formal presentations of self.

Read this paper

PNAS, 2020  

The confidence gap predicts the gender pay gap among STEM graduates

Adina D. Sterling, Marissa E. Thompson, Shiya Wang, Abisola Kusimo, Shannon Gilmartin, and Sheri Sheppard

How is self-confidence related to the gender wage gap in STEM?

Is there a gender pay gap among graduates in some science, technology, engineering, and math (STEM) fields? Women and men have near-identical human capital at college exit, but cultural beliefs about men as more fit for STEM professions than women may lead to self-beliefs that affect pay. We hypothesized that women and men would be paid differently upon college exit, and that gender gaps in self-beliefs about one’s abilities, or self-efficacy, would correspond to this difference. Using data from a three-wave longitudinal study of graduates of engineering programs from 2015–2017, we find a confidence gap that aligns with a gender pay gap. Overall, these findings point to the role that cultural beliefs play in creating gender disparities among STEM degree-holders.

Read this paper

Hamilton Project / Brookings, 2020  

Building Tomorrow’s Workforce Today: Twin Proposals for the Future of Learning, Opportunity and Work

Richard Arum & Mitchell L. Stevens

A federal policy proposal to assist adult learners in the wake of the pandemic

The US federal government has serially called upon colleges and universities to assist the nation in moments of national crisis. This brief outlines an ambitious plan to enlist the postsecondary sector in helping millions of Americans get back to work in the wake of the pandemic and equip them for ongoing prosperity in a highly dynamic economy.

Read this paper

Teaching Online in 2020: Experiments, Empathy, Discovery

Maxwell Bigman and John C. Mitchell

What happened in college classrooms in the wake of COVID?

The authors attended discussions and interviewed instructors in Stanford’s Computer Science Department to identify successful approaches and problem areas in the rapid transition to online learning.

Read this paper

Application Essays and the Performance of Merit in US Selective Admissions

Ben Gebre-Medhin, Sonia Giebel, AJ Alvero, anthony lising antonio, Benjamin W. Domingue and Mitchell L. Stevens

What is merit?

The authors combine qualitative and quantitative techniques to observe how 55,016 applicants to a highly selective public university narrate their cases for admission.

Read this paper

Course Reviews Reveal Gender Differences and other Scientific Insight about the Students who Submit Them

David Lang, Youjie Chin, Andreas Paepcke, and Mitchell L. Stevens

What's in a course review?

College and university students submit millions of course reviews each year, yet these instruments are only rarely leveraged for scientific inquiry. This paper examines 11,255 reviews submitted to computer science courses to illustrate how such inquiry might proceed.

Read this paper

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020  

Reinforcement Learning for the Adaptive Scheduling of Educational Activities

Jonathan Bassen, Bharathan Balaji, Michael Schaarschmidt, Candace Thille, Jay Painter, Dawn Zimmaro, Alex Games, Ethan Fast, and John C. Mitchell

Can reinforcement models improve learning?

We show that a Reinforcement Learning (RL) model produces better learning gains using fewer educational activities than a linear assignment condition, and produces similar learning gains to a self-directed condition using fewer educational activities and with lower dropout rates.

Read this paper

Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2020  

AI and Holistic Review: Informing Human Reading in College Admissions

AJ Alvero, Noah Arthurs, anthony lising antonio, Benjamin W. Domingue, Ben Gebre-Medhin, Sonia Giebel, and Mitchell L. Stevens

Can AI improve holistic review?

We use a variety of text classification algorithms on a large corpus of college admissions essays to model the extent and ways that AI might inform human evaluation.

Read this paper

Learning Analytics and Knowledge, 2020  

Is Faster Better? A Study of Video Playback Speed

David Lang, Guanling Chen, Kathy Mirzaei, and Andreas Paepcke

Fast forward?

Intriguing relationships between video pacing, course persistence and academic achievement.

Read this paper

Sociology Compass, 2020  

Universities as Peculiar Organizations

Charlie Eaton and Mitchell L. Stevens

What is a university?

An appraisal of universities as distinctive institutions, on three dimensions.

Read this paper

The New York Times 2020  

What is a College Education in the Time of Coronavirus?

an editorial by Richard Arum and Mitchell L. Stevens

Read this paper

Proceedings of The 13th International Conference on Educational Data Mining (EDM), 2020  

Whose Truth is the “Ground Truth”? College Admissions Essays and Bias in Word Vector Evaluation Methods

Noah Arthurs and AJ Alvero

Are word vector evaluation methods biased?

Widely used word vector evaluation methods are biased towards the language of high income students. This problematically suggests that some students’ language is closer to the “ground truth” than others.

Read this paper

Educational Data Mining, 2019  

Using Latent Variable Models to Observe Academic Pathways

Nate Gruver, Ali Malik, Brahm Capoor, Chris Piech, Mitchell L. Stevens, and Andreas Paepcke

How are student enrollment decisions best modeled with large-scale datasets?

Using ten years of anonymized transcript data, the authors use a probabilistic modeling approach to model and predict student choices. This allows us to capture the complex relationships between courses, such as the tendency of some courses to serve as prerequisites for others without a formal requirement.

Read this paper

Proceedings of the Sixth ACM Conference on Learning at Scale (L@S), 2019  

Via: Illuminating Undergraduate Academic Pathways at Scale

Geoffrey Angus, Richard Diehl Martinez, Mitchell L. Stevens, and Andreas Paepcke

Which courses, and why?

We offer an analytic toolkit, called Via, which transforms commonly available enrollment data into formal graphs that are amenable to interactive visualizations and computational exploration. The tool is intended for a variety of stakeholders: college students, instructors, and administrators.

Read this paper

Annual Meeting of the American Sociological Association (ASA), 2018  

Choices, Identities, Paths: Understanding College Students’ Academic Decisions

Mitchell L. Stevens, Monique H. Harrison, Marissa E. Thompson, Arik Lifschitz, and Sorathan Chaturapruek

How do students choose their courses?

Drawing on preliminary empirical research involving a web-based course exploration and planning tool in use at a private U.S. research university, we develop a conceptual framework for studying college students’ academic choices.

Read this paper

Proceedings of the Fifth ACM Conference on Learning at Scale (L@S), 2018  

How a Data-Driven Course Planning Tool affects College Students’ GPA: Evidence from Two Field Experiments

Sorathan Chaturapruek, Thomas S. Dee, Ramesh Johari, René F. Kizilcec and Mitchell L. Stevens

How does a course planning tool affect college students’ grades?

We conducted a large-scale field experiment in which all undergraduates were randomly encouraged to use Carta, a web-based course planning tool. We found that use of the platform lowered students’ GPA by 0.28 standard deviations on average.

Read this paper

Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 2018  

OARS: Exploring Instructor Analytics for Online Learning

Jonathan Bassen, Iris Howley, Ethan Fast, John Mitchell, and Candace Thille

How can learning analytics systems improve online education?

Our study suggests new design goals for learning analytics systems, the importance of real-time analytics to many instructors, and the value of flexibility in data selection and aggregation for an instructor when working with an analytics system.

Read this paper

Issues in Science and Technology, 2018  

Research Universities and the Future of Work

Mitchell L. Stevens

What do universities owe the future?

Read this paper

EDUCAUSE Review, 2018  

Setting the Table: Responsible Use of Student Data in Higher Education

Martin Kurzweil and Mitchell L. Stevens

What does it mean to use student data responsibly?

The higher education community must set the table and invite others to help us define ethical practice and responsible use of student data in the rapidly changing digital world of the academic enterprise.

Read this paper

Educational Data Mining, 2017  

Making the Grade: How Learner Engagement Changes after Passing a Course

Ben Domingue, Alex Kindel, and Andreas Paepcke

How does online learners' course engagement evolve?

Read this paper

InsideHigherEd, 2015  

What We’ve Learned from MOOCs

editorial by Candace Thille, John Mitchell and Mitchell Stevens

Read this paper

Research and Practice in Assessment, 2014  

An Ethically Ambitious Higher Education Data Science

Mitchell L. Stevens

How do we pivot from rule compliance to ethical proaction?

Read this paper