Publications

change view

listexpanded

Science  


From pipelines to pathways in the study of academic progress

Rene F. Kizilcec, Rachel B. Baker, Elizabeth Bruch, Kalena E. Cortes, Laura T. Hamilton, Zachary A. Pardos, Marissa E. Thompson, and Mitchell L. Stevens

Enabling the study education and learning as sequential phenomena.

Read this paper

Contemporary Social Science, 2022  


Supporting interviews with technology: How software integration can benefit participants and interviewers

Monique H. Harrison and Philip A. Hernandez

How can interview researcher take advantage of web-based platforms?

Drawing on their own design for a longitudinal cohort study of undergraduate students, Harrison and Hernandez share techniques for smoothing research relationships and logistics through widely available digital media.

Read this paper

AERA Open, 2022  


Forecasting undergraduate majors: A natural language approach

David Lang, Alex Wang, Nathan Dalal, Andreas Paepcke and Mitchell L. Stevens

Predicting college pathways based on early coursework

Elective curriculums require undergraduates to choose from a large roster of courses for enrollment each term. It has proven difficult to characterize this fateful choice process because it remains largely unobserved. Using digital trace data to observe this process at scale at a private research university, together with qualitative student interviews, we provide a novel empirical study of course consideration as an important component of course selection. Clickstream logs from a course exploration platform used by most undergraduates at the case university reveal that students consider on average nine courses for enrollment for their first fall term (<2% of available courses) and these courses predict which academic major students declare two years later. Twenty-nine interviews confirm that students experience consideration as complex and reveal variation in consideration strategies that may influence how consideration unfolds. Consideration presents a promising site for intervention in problems of equity, career funneling, and college completion.

Read this paper

Socius, 2022  


Signaled or suppressed? How gender informs women’s undergraduate applications in biology and engineering

Sonia Giebel, AJ Alvero, Ben Gebre-Medhin and anthony lising antonio

How is gender reflected in college application materials?

Analyzing 60,000 undergraduate applications to the University of California, the authors find that extant gender segregation of academic disciplines also manifests in intended major choice. Gender and SAT Math scores together strongly predict intent to major in biology and engineering, the most popular and gender-segregated majors. Using natural language processing, the authors also find that author gender is more predictive of essay topics written by prospective engineers than prospective biologists. Women intending to major in engineering write about essay topics that signal their gender identity to a greater degree than women intending to major in biology, perhaps to mitigate gender-transgressive academic commitments.

Read this paper

Poetics, 2022  


Application essays and the ritual production of merit in US selective admissions

Ben Gebre-Medhin, Sonia Giebel, AJ Alvero, anthony lising antonio, Benjamin W. Domingue, and Mitchell L. Stevens

What are college application essays for?

US colleges and universities are defined by their exclusivity, and the most prestigious schools reject most of those who apply. Yet these same schools also widely advertise their inclusiveness, encouraging students from all backgrounds to submit applications and highlighting evaluation protocols that identify many characteristics worthy of consideration for admission. We surface this paradox and use it as motivation to theorize a little studied component of college applications: personal essays.

Read this paper

American Sociological Review, 2022  


From bat mitzvah to the bar: Religious habitus, self-concept, and women’s educational outcomes

Ilana M. Horwitz, Kaylee T. Matheny, Krystal Laryea, and Landon Schnabel

How religious commitment shapes educational progress across the early life course

This study considers the role of religious habitus and self-concept in educational stratification. The authors follow 3,238 adolescents for 13 years by linking the National Study of Youth and Religion to the National Student Clearinghouse. Survey data reveal that girls with a Jewish upbringing have two distinct postsecondary patterns compared to girls with a non-Jewish upbringing, even after controlling for social origins: (1) they are 23 percentage points more likely to graduate college, and (2) they graduate from much more selective colleges. They also analyze 107 interviews with 33 girls from comparable social origins interviewed repeatedly between adolescence and emerging adulthood to develop fuller portraits of how these patterns unfold.

Read this paper

Sociology of Education, 2022  


Should I start at MATH 101? Content repetition as an academic strategy in elective curriculums

Monique H. Harrison, A. Philip Hernandez, and Mitchell L. Stevens

How do undergraduates make their first course decisions, and are these decisions fateful?

Drawing on serial interviews (N = 200) of 53 students at an admissions-selective university, we show that incoming students with disparate precollege experiences differ in their orientations toward and strategies for considering first college math courses. Content repeaters opt for courses that repeat material covered in prior coursework, whereas novices opt for courses covering material new to them. Content repeaters receive high grades and report confidence in their math ability, whereas novices in the same classes receive lower grades and report invidious comparisons with classmates.

Read this paper

2022  


NSF report: An applied science to support working learners

Mitchell L. Stevens, Galeana Drew Alston, Marie Cini, Sean Gallagher, Ilana Horwitz, Cathrael Kazin, Pamela Clouser McCann, Zach Pardos, Elizabeth A. Roumell, Hadass Sheffer, Holly Zanville, Richard Settersten

Concise recommendations from a national peer review

Supported with funds from the National Science Foundation, Stanford University hosted a virtual convening in July 2021 to frame an applied science to support working learners. The goal of this science is to measurably improve educational opportunities and mechanisms of occupational mobility for adult Americans. We forward nine recommendations.

Read this paper

Stanford Center on Longevity, 2021  


Reimagining education for a new map of life

Ilana M. Horwitz & Mitchell L. Stevens

How should we change education to better serve longer lives?

Horwitz and Stevens synthesize several scholarly literatures to call for investments in early childhood education and flexible alternatives to four-year college degrees. Together these investments enable serial career transitions and meaningful lifelong learning.

Read this paper

Science Advances, 2021  


Essay content is strongly related to household Income and SAT Scores: Evidence from 60,000 undergraduate applications

AJ Alvero, Sonia Giebel, Ben Gebre-Medhin, anthony lising antonio, Mitchell L. Stevens and Benjamin W. Domingue

College application essays in an era of machine reading

We utilize a corpus of 240,000 admissions essays submitted by 60,000 applicants to the University of California in November 2016 to measure the relationship between the content of application essays, reported household income, and standardized test scores (SAT) at scale. We find that essays have a stronger correlation to reported household income than SAT scores.

Read this paper

Journal of Higher Education, 2021  


Ambiguous credentials: How learners use and make sense of massively open online courses

Krystal Laryea, Kathy Mirzaei, Andreas Paepcke & Mitchell Stevens

What good is a MOOC?

As low-status academic offerings purveyed by high-status institutions, massively open online courses (MOOCs) are ambiguous credentials. In interviews with 60 people who devoted substantial time to at least one MOOC between 2014-2017, we find that people use MOOCs to build skills for application at work and home, build relationships, navigate life transitions, and enhance formal presentations of self.

Read this paper

AERA Open, 2021  


Studying undergraduate course consideration at scale

Sorathan Chaturapruek, Tobias Dalberg, Rene F. Kizilcec, Marissa E. Thompson, Sonia Giebel, Monique Harrison, Ramesh Johari, and Mitchell L. Stevens

What role does course consideration play in college students' pathways?

Using digital trace data to observe this process at scale at a private research university, together with qualitative student interviews, we provide a novel empirical study of course consideration as part of the course selection process.

Read this paper

PNAS, 2020  


The confidence gap predicts the gender pay gap among STEM graduates

Adina D. Sterling, Marissa E. Thompson, Shiya Wang, Abisola Kusimo, Shannon Gilmartin, and Sheri Sheppard

How is self-confidence related to the gender wage gap in STEM?

Is there a gender pay gap among graduates in some science, technology, engineering, and math (STEM) fields? Women and men have near-identical human capital at college exit, but cultural beliefs about men as more fit for STEM professions than women may lead to self-beliefs that affect pay. We hypothesized that women and men would be paid differently upon college exit, and that gender gaps in self-beliefs about one’s abilities, or self-efficacy, would correspond to this difference. Using data from a three-wave longitudinal study of graduates of engineering programs from 2015–2017, we find a confidence gap that aligns with a gender pay gap. Overall, these findings point to the role that cultural beliefs play in creating gender disparities among STEM degree-holders.

Read this paper

Hamilton Project / Brookings, 2020  


Building tomorrow’s workforce today: Twin proposals for the future of learning, opportunity and work

Richard Arum & Mitchell L. Stevens

A federal policy proposal to assist adult learners in the wake of the pandemic

The US federal government has serially called upon colleges and universities to assist the nation in moments of national crisis. This brief outlines an ambitious plan to enlist the postsecondary sector in helping millions of Americans get back to work in the wake of the pandemic and equip them for ongoing prosperity in a highly dynamic economy.

Read this paper

Teaching online in 2020: Experiments, empathy, discovery

Maxwell Bigman and John C. Mitchell

What happened in college classrooms in the wake of COVID?

The authors attended discussions and interviewed instructors in Stanford’s Computer Science Department to identify successful approaches and problem areas in the rapid transition to online learning.

Read this paper

Course reviews reveal gender differences and other scientific insight about the students who submit them

David Lang, Youjie Chin, Andreas Paepcke, and Mitchell L. Stevens

What's in a course review?

College and university students submit millions of course reviews each year, yet these instruments are only rarely leveraged for scientific inquiry. This paper examines 11,255 reviews submitted to computer science courses to illustrate how such inquiry might proceed.

Read this paper

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020  


Reinforcement learning for the adaptive scheduling of educational activities

Jonathan Bassen, Bharathan Balaji, Michael Schaarschmidt, Candace Thille, Jay Painter, Dawn Zimmaro, Alex Games, Ethan Fast, and John C. Mitchell

Can reinforcement models improve learning?

We show that a Reinforcement Learning (RL) model produces better learning gains using fewer educational activities than a linear assignment condition, and produces similar learning gains to a self-directed condition using fewer educational activities and with lower dropout rates.

Read this paper

Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2020  


AI and holistic review: Informing human reading in college admissions

AJ Alvero, Noah Arthurs, anthony lising antonio, Benjamin W. Domingue, Ben Gebre-Medhin, Sonia Giebel, and Mitchell L. Stevens

Can AI improve holistic review?

We use a variety of text classification algorithms on a large corpus of college admissions essays to model the extent and ways that AI might inform human evaluation.

Read this paper

Learning Analytics and Knowledge, 2020  


Is faster better? A study of video playback speed

David Lang, Guanling Chen, Kathy Mirzaei, and Andreas Paepcke

Fast forward?

Intriguing relationships between video pacing, course persistence and academic achievement.

Read this paper

Sociology Compass, 2020  


Universities as peculiar organizations

Charlie Eaton and Mitchell L. Stevens

What is a university?

An appraisal of universities as distinctive institutions, on three dimensions.

Read this paper

The New York Times 2020  


What is a college education in the time of coronavirus?

an editorial by Richard Arum and Mitchell L. Stevens

Read this paper

Proceedings of The 13th International Conference on Educational Data Mining (EDM), 2020  


Whose truth is the “ground truth”? College admissions essays and bias in word vector evaluation methods

Noah Arthurs and AJ Alvero

Are word vector evaluation methods biased?

Widely used word vector evaluation methods are biased towards the language of high income students. This problematically suggests that some students’ language is closer to the “ground truth” than others.

Read this paper

Proceedings of the 12th Annual Conference of Educational Data Mining (EDM), 2019  


Using latent variable models to observe academic pathways

Nate Gruver, Ali Malik, Brahm Capoor, Chris Piech, Mitchell L. Stevens, and Andreas Paepcke

How are student enrollment decisions best modeled with large-scale datasets?

Using ten years of anonymized transcript data, the authors use a probabilistic modeling approach to model and predict student choices. This allows us to capture the complex relationships between courses, such as the tendency of some courses to serve as prerequisites for others without a formal requirement.

Read this paper

Proceedings of the Sixth ACM Conference on Learning at Scale (L@S), 2019  


Via: Illuminating undergraduate academic pathways at scale

Geoffrey Angus, Richard Diehl Martinez, Mitchell L. Stevens, and Andreas Paepcke

Which courses, and why?

We offer an analytic toolkit, called Via, which transforms commonly available enrollment data into formal graphs that are amenable to interactive visualizations and computational exploration. The tool is intended for a variety of stakeholders: college students, instructors, and administrators.

Read this paper

Annual Meeting of the American Sociological Association (ASA), 2018  


Choices, identities, paths: Understanding college students’ academic decisions

Mitchell L. Stevens, Monique H. Harrison, Marissa E. Thompson, Arik Lifschitz, and Sorathan Chaturapruek

How do students choose their courses?

Drawing on preliminary empirical research involving a web-based course exploration and planning tool in use at a private U.S. research university, we develop a conceptual framework for studying college students’ academic choices.

Read this paper

Proceedings of the Fifth ACM Conference on Learning at Scale (L@S), 2018  


How a data-driven course planning tool affects college students’ GPA: Evidence from two field experiments

Sorathan Chaturapruek, Thomas S. Dee, Ramesh Johari, René F. Kizilcec and Mitchell L. Stevens

How does a course planning tool affect college students’ grades?

We conducted a large-scale field experiment in which all undergraduates were randomly encouraged to use Carta, a web-based course planning tool. We found that use of the platform lowered students’ GPA by 0.28 standard deviations on average.

Read this paper

Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 2018  


OARS: Exploring instructor analytics for online learning

Jonathan Bassen, Iris Howley, Ethan Fast, John Mitchell, and Candace Thille

How can learning analytics systems improve online education?

Our study suggests new design goals for learning analytics systems, the importance of real-time analytics to many instructors, and the value of flexibility in data selection and aggregation for an instructor when working with an analytics system.

Read this paper

Issues in Science and Technology, 2018  


Research universities and the future of work

Mitchell L. Stevens

What do universities owe the future?

Read this paper

EDUCAUSE Review, 2018  


Setting the table: Responsible use of student data in higher education

Martin Kurzweil and Mitchell L. Stevens

What does it mean to use student data responsibly?

The higher education community must set the table and invite others to help us define ethical practice and responsible use of student data in the rapidly changing digital world of the academic enterprise.

Read this paper

Educational Data Mining, 2017  


Making the grade: How learner engagement changes after passing a course

Ben Domingue, Alex Kindel, and Andreas Paepcke

How does online learners' course engagement evolve?

Read this paper

InsideHigherEd, 2015  


What we’ve learned from MOOCs

editorial by Candace Thille, John Mitchell and Mitchell Stevens

Read this paper

Research and Practice in Assessment, 2014  


An ethically ambitious higher education data science

Mitchell L. Stevens

How do we pivot from rule compliance to ethical proaction?

Read this paper