Science
From pipelines to pathways in the study of academic progress
Rene F. Kizilcec, Rachel B. Baker, Elizabeth Bruch, Kalena E. Cortes, Laura T. Hamilton, Zachary A. Pardos, Marissa E. Thompson, and Mitchell L. Stevens
Enabling the study education and learning as sequential phenomena.
Contemporary Social Science, 2022
Supporting interviews with technology: How software integration can benefit participants and interviewers
Monique H. Harrison and Philip A. Hernandez
How can interview researcher take advantage of web-based platforms?
Drawing on their own design for a longitudinal cohort study of undergraduate students, Harrison and Hernandez share techniques for smoothing research relationships and logistics through widely available digital media.
AERA Open, 2022
Forecasting undergraduate majors: A natural language approach
David Lang, Alex Wang, Nathan Dalal, Andreas Paepcke and Mitchell L. Stevens
Predicting college pathways based on early coursework
Elective curriculums require undergraduates to choose from a large roster of courses for enrollment each term. It has proven difficult to characterize this fateful choice process because it remains largely unobserved. Using digital trace data to observe this process at scale at a private research university, together with qualitative student interviews, we provide a novel empirical study of course consideration as an important component of course selection. Clickstream logs from a course exploration platform used by most undergraduates at the case university reveal that students consider on average nine courses for enrollment for their first fall term (<2% of available courses) and these courses predict which academic major students declare two years later. Twenty-nine interviews confirm that students experience consideration as complex and reveal variation in consideration strategies that may influence how consideration unfolds. Consideration presents a promising site for intervention in problems of equity, career funneling, and college completion.
Socius, 2022
Signaled or suppressed? How gender informs women’s undergraduate applications in biology and engineering
Sonia Giebel, AJ Alvero, Ben Gebre-Medhin and anthony lising antonio
How is gender reflected in college application materials?
Analyzing 60,000 undergraduate applications to the University of California, the authors find that extant gender segregation of academic disciplines also manifests in intended major choice. Gender and SAT Math scores together strongly predict intent to major in biology and engineering, the most popular and gender-segregated majors. Using natural language processing, the authors also find that author gender is more predictive of essay topics written by prospective engineers than prospective biologists. Women intending to major in engineering write about essay topics that signal their gender identity to a greater degree than women intending to major in biology, perhaps to mitigate gender-transgressive academic commitments.
Poetics, 2022
Application essays and the ritual production of merit in US selective admissions
Ben Gebre-Medhin, Sonia Giebel, AJ Alvero, anthony lising antonio, Benjamin W. Domingue, and Mitchell L. Stevens
What are college application essays for?
US colleges and universities are defined by their exclusivity, and the most prestigious schools reject most of those who apply. Yet these same schools also widely advertise their inclusiveness, encouraging students from all backgrounds to submit applications and highlighting evaluation protocols that identify many characteristics worthy of consideration for admission. We surface this paradox and use it as motivation to theorize a little studied component of college applications: personal essays.
American Sociological Review, 2022
From bat mitzvah to the bar: Religious habitus, self-concept, and women’s educational outcomes
Ilana M. Horwitz, Kaylee T. Matheny, Krystal Laryea, and Landon Schnabel
How religious commitment shapes educational progress across the early life course
This study considers the role of religious habitus and self-concept in educational stratification. The authors follow 3,238 adolescents for 13 years by linking the National Study of Youth and Religion to the National Student Clearinghouse. Survey data reveal that girls with a Jewish upbringing have two distinct postsecondary patterns compared to girls with a non-Jewish upbringing, even after controlling for social origins: (1) they are 23 percentage points more likely to graduate college, and (2) they graduate from much more selective colleges. They also analyze 107 interviews with 33 girls from comparable social origins interviewed repeatedly between adolescence and emerging adulthood to develop fuller portraits of how these patterns unfold.
Sociology of Education, 2022
Should I start at MATH 101? Content repetition as an academic strategy in elective curriculums
Monique H. Harrison, A. Philip Hernandez, and Mitchell L. Stevens
How do undergraduates make their first course decisions, and are these decisions fateful?
Drawing on serial interviews (N = 200) of 53 students at an admissions-selective university, we show that incoming students with disparate precollege experiences differ in their orientations toward and strategies for considering first college math courses. Content repeaters opt for courses that repeat material covered in prior coursework, whereas novices opt for courses covering material new to them. Content repeaters receive high grades and report confidence in their math ability, whereas novices in the same classes receive lower grades and report invidious comparisons with classmates.
2022
NSF report: An applied science to support working learners
Mitchell L. Stevens, Galeana Drew Alston, Marie Cini, Sean Gallagher, Ilana Horwitz, Cathrael Kazin, Pamela Clouser McCann, Zach Pardos, Elizabeth A. Roumell, Hadass Sheffer, Holly Zanville, Richard Settersten
Concise recommendations from a national peer review
Supported with funds from the National Science Foundation, Stanford University hosted a virtual convening in July 2021 to frame an applied science to support working learners. The goal of this science is to measurably improve educational opportunities and mechanisms of occupational mobility for adult Americans. We forward nine recommendations.
Stanford Center on Longevity, 2021
Reimagining education for a new map of life
Ilana M. Horwitz & Mitchell L. Stevens
How should we change education to better serve longer lives?
Horwitz and Stevens synthesize several scholarly literatures to call for investments in early childhood education and flexible alternatives to four-year college degrees. Together these investments enable serial career transitions and meaningful lifelong learning.
Science Advances, 2021
Essay content is strongly related to household Income and SAT Scores: Evidence from 60,000 undergraduate applications
AJ Alvero, Sonia Giebel, Ben Gebre-Medhin, anthony lising antonio, Mitchell L. Stevens and Benjamin W. Domingue
College application essays in an era of machine reading
We utilize a corpus of 240,000 admissions essays submitted by 60,000 applicants to the University of California in November 2016 to measure the relationship between the content of application essays, reported household income, and standardized test scores (SAT) at scale. We find that essays have a stronger correlation to reported household income than SAT scores.
Journal of Higher Education, 2021
Ambiguous credentials: How learners use and make sense of massively open online courses
Krystal Laryea, Kathy Mirzaei, Andreas Paepcke & Mitchell Stevens
What good is a MOOC?
As low-status academic offerings purveyed by high-status institutions, massively open online courses (MOOCs) are ambiguous credentials. In interviews with 60 people who devoted substantial time to at least one MOOC between 2014-2017, we find that people use MOOCs to build skills for application at work and home, build relationships, navigate life transitions, and enhance formal presentations of self.
AERA Open, 2021
Studying undergraduate course consideration at scale
Sorathan Chaturapruek, Tobias Dalberg, Rene F. Kizilcec, Marissa E. Thompson, Sonia Giebel, Monique Harrison, Ramesh Johari, and Mitchell L. Stevens
What role does course consideration play in college students' pathways?
Using digital trace data to observe this process at scale at a private research university, together with qualitative student interviews, we provide a novel empirical study of course consideration as part of the course selection process.
PNAS, 2020
The confidence gap predicts the gender pay gap among STEM graduates
Adina D. Sterling, Marissa E. Thompson, Shiya Wang, Abisola Kusimo, Shannon Gilmartin, and Sheri Sheppard
How is self-confidence related to the gender wage gap in STEM?
Is there a gender pay gap among graduates in some science, technology, engineering, and math (STEM) fields? Women and men have near-identical human capital at college exit, but cultural beliefs about men as more fit for STEM professions than women may lead to self-beliefs that affect pay. We hypothesized that women and men would be paid differently upon college exit, and that gender gaps in self-beliefs about one’s abilities, or self-efficacy, would correspond to this difference. Using data from a three-wave longitudinal study of graduates of engineering programs from 2015–2017, we find a confidence gap that aligns with a gender pay gap. Overall, these findings point to the role that cultural beliefs play in creating gender disparities among STEM degree-holders.
Hamilton Project / Brookings, 2020
Building tomorrow’s workforce today: Twin proposals for the future of learning, opportunity and work
Richard Arum & Mitchell L. Stevens
A federal policy proposal to assist adult learners in the wake of the pandemic
The US federal government has serially called upon colleges and universities to assist the nation in moments of national crisis. This brief outlines an ambitious plan to enlist the postsecondary sector in helping millions of Americans get back to work in the wake of the pandemic and equip them for ongoing prosperity in a highly dynamic economy.
Teaching online in 2020: Experiments, empathy, discovery
Maxwell Bigman and John C. Mitchell
What happened in college classrooms in the wake of COVID?
The authors attended discussions and interviewed instructors in Stanford’s Computer Science Department to identify successful approaches and problem areas in the rapid transition to online learning.
Course reviews reveal gender differences and other scientific insight about the students who submit them
David Lang, Youjie Chin, Andreas Paepcke, and Mitchell L. Stevens
What's in a course review?
College and university students submit millions of course reviews each year, yet these instruments are only rarely leveraged for scientific inquiry. This paper examines 11,255 reviews submitted to computer science courses to illustrate how such inquiry might proceed.
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020
Reinforcement learning for the adaptive scheduling of educational activities
Jonathan Bassen, Bharathan Balaji, Michael Schaarschmidt, Candace Thille, Jay Painter, Dawn Zimmaro, Alex Games, Ethan Fast, and John C. Mitchell
Can reinforcement models improve learning?
We show that a Reinforcement Learning (RL) model produces better learning gains using fewer educational activities than a linear assignment condition, and produces similar learning gains to a self-directed condition using fewer educational activities and with lower dropout rates.
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2020
AI and holistic review: Informing human reading in college admissions
AJ Alvero, Noah Arthurs, anthony lising antonio, Benjamin W. Domingue, Ben Gebre-Medhin, Sonia Giebel, and Mitchell L. Stevens
Can AI improve holistic review?
We use a variety of text classification algorithms on a large corpus of college admissions essays to model the extent and ways that AI might inform human evaluation.
Learning Analytics and Knowledge, 2020
Is faster better? A study of video playback speed
David Lang, Guanling Chen, Kathy Mirzaei, and Andreas Paepcke
Fast forward?
Intriguing relationships between video pacing, course persistence and academic achievement.
Sociology Compass, 2020
Universities as peculiar organizations
Charlie Eaton and Mitchell L. Stevens
What is a university?
An appraisal of universities as distinctive institutions, on three dimensions.
The New York Times 2020
What is a college education in the time of coronavirus?
an editorial by Richard Arum and Mitchell L. Stevens
Proceedings of The 13th International Conference on Educational Data Mining (EDM), 2020
Whose truth is the “ground truth”? College admissions essays and bias in word vector evaluation methods
Noah Arthurs and AJ Alvero
Are word vector evaluation methods biased?
Widely used word vector evaluation methods are biased towards the language of high income students. This problematically suggests that some students’ language is closer to the “ground truth” than others.
Proceedings of the 12th Annual Conference of Educational Data Mining (EDM), 2019
Using latent variable models to observe academic pathways
Nate Gruver, Ali Malik, Brahm Capoor, Chris Piech, Mitchell L. Stevens, and Andreas Paepcke
How are student enrollment decisions best modeled with large-scale datasets?
Using ten years of anonymized transcript data, the authors use a probabilistic modeling approach to model and predict student choices. This allows us to capture the complex relationships between courses, such as the tendency of some courses to serve as prerequisites for others without a formal requirement.
Proceedings of the Sixth ACM Conference on Learning at Scale (L@S), 2019
Via: Illuminating undergraduate academic pathways at scale
Geoffrey Angus, Richard Diehl Martinez, Mitchell L. Stevens, and Andreas Paepcke
Which courses, and why?
We offer an analytic toolkit, called Via, which transforms commonly available enrollment data into formal graphs that are amenable to interactive visualizations and computational exploration. The tool is intended for a variety of stakeholders: college students, instructors, and administrators.
Annual Meeting of the American Sociological Association (ASA), 2018
Choices, identities, paths: Understanding college students’ academic decisions
Mitchell L. Stevens, Monique H. Harrison, Marissa E. Thompson, Arik Lifschitz, and Sorathan Chaturapruek
How do students choose their courses?
Drawing on preliminary empirical research involving a web-based course exploration and planning tool in use at a private U.S. research university, we develop a conceptual framework for studying college students’ academic choices.
Proceedings of the Fifth ACM Conference on Learning at Scale (L@S), 2018
How a data-driven course planning tool affects college students’ GPA: Evidence from two field experiments
Sorathan Chaturapruek, Thomas S. Dee, Ramesh Johari, René F. Kizilcec and Mitchell L. Stevens
How does a course planning tool affect college students’ grades?
We conducted a large-scale field experiment in which all undergraduates were randomly encouraged to use Carta, a web-based course planning tool. We found that use of the platform lowered students’ GPA by 0.28 standard deviations on average.
Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 2018
OARS: Exploring instructor analytics for online learning
Jonathan Bassen, Iris Howley, Ethan Fast, John Mitchell, and Candace Thille
How can learning analytics systems improve online education?
Our study suggests new design goals for learning analytics systems, the importance of real-time analytics to many instructors, and the value of flexibility in data selection and aggregation for an instructor when working with an analytics system.
Issues in Science and Technology, 2018
Research universities and the future of work
Mitchell L. Stevens
What do universities owe the future?
EDUCAUSE Review, 2018
Setting the table: Responsible use of student data in higher education
Martin Kurzweil and Mitchell L. Stevens
What does it mean to use student data responsibly?
The higher education community must set the table and invite others to help us define ethical practice and responsible use of student data in the rapidly changing digital world of the academic enterprise.
Educational Data Mining, 2017
Making the grade: How learner engagement changes after passing a course
Ben Domingue, Alex Kindel, and Andreas Paepcke
How does online learners' course engagement evolve?
InsideHigherEd, 2015
What we’ve learned from MOOCs
editorial by Candace Thille, John Mitchell and Mitchell Stevens
Research and Practice in Assessment, 2014
An ethically ambitious higher education data science
Mitchell L. Stevens
How do we pivot from rule compliance to ethical proaction?