Automated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses

TitleAutomated Text Analysis Facilitates Using Written Formative Assessments for Just-in-Time Teaching in Large Enrollment Courses
Publication TypeConference Proceedings
Year of Publication2013
AuthorsPrevost, LB, Haudek, KC, Norton-Henry, E, Berry, MC, Urban-Lurain, M
Conference NameASEE Annual Conference
Date Published06/2013
Conference LocationAtlanta, GA
KeywordsAACR, Biology education, JiTT, Lexical analysis
AbstractWritten formative assessments can provide instructors with rich insight into students’ thinking about scientific concepts. However, the time and effort involved in grading deter instructors from having students write in large courses. As large-enrollment introductory STEM courses become increasingly common, the need for innovations that facilitate the use of written assessments continues to grow. We piloted the use of automated text analysis to overcome these obstacles and facilitate the use of written formative assessment in a large-enrollment introductory biology course. Student responses to online homework on thermodynamics, metabolism, central dogma (genetics) and acid-base chemistry were collected in three 300+-person course sections. We used automated text analysis to extract and categorize concepts from student writing. Then, we used k-means cluster analysis to aggregate responses into distinct groups. From these analyses, we created feedback reports to provide instructors with an assessment of students’ responses before the next class period (less than one working day), so that instructors could use this feedback to inform their instruction. We present the results of this pilot study, including a description of the feedback reports and faculty instruction in response to feedback on student writing. We also describe lessons learned to improve the use of written assessments, automated analysis, rapid feedback reports and instruction in large enrollment courses. Finally, we suggest some future directions for research based on our analysis of student writing.
Refereed DesignationRefereed


thumbnail of small NSF logo in color without shading

This material is based upon work supported by the National Science Foundation (DUE grants: 1438739, 1323162, 1347740, 0736952 and 1022653). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.