Machine Learning Mixed Methods Text Analysis: An Illustration From Automated Scoring Models of Student Writing in Biology Education

Abstract

Assessing student knowledge based on their writing using traditional qualitative methods is time-consuming. To improve speed and consistency of text analysis, we present our mixed methods development of a machine learning predictive model to analyze student writing. Our approach involves two stages: first an exploratory sequential design, and second an iterative complex design. We first trained our predictive model using qualitative coding of categories (ideas) in student writing. We next revised our model based on feedback from instructor-users. The model itself highlighted categories in need of revision. The contribution to mixed methods research lies in our innovative use of the machine learning tool as a rapid, consistent additional coder, and a resource that can predict codes for new student writing.

Author

Kamali Sripathi, Rosa Moscarella, Matthew Steele, Rachel Yoho, Hyesun You, Luanna Prevost, Mark Urban-Lurain, John Merrill, Kevin Haudek

Year of Publication

2023

Journal

Journal of Mixed Methods Research

Number of Pages

15586898231153946

Date Published

02/2023

ISSN Number

1558-6898

URL

https://doi.org/10.1177/15586898231153946

DOI

10.1177/15586898231153946