Using computerized lexical analysis of student writing to support just-in-time teaching in large enrollment STEM courses

TitleUsing computerized lexical analysis of student writing to support just-in-time teaching in large enrollment STEM courses
Publication TypeConference Proceedings
Year of Publication2013
AuthorsUrban-Lurain, M, Prevost, LB, Haudek, KC, Norton-Henry, E, Berry, MC, Merrill, JE
Conference NameFrontiers in Education
Date Published10/2013
Conference LocationOklahoma City
Keywordsconstructed responses, JiTT, Just-in-Time Teaching, large-enrollment introductory courses, Lexical analysis
AbstractWe have been exploring a variety of computerized techniques for analyzing student writing in introductory biology. We achieve computer-to-expert inter-rater reliability (IRR) on par with expert-to-expert IRR (> .8). In Fall, 2012, we piloted the use of automated text analysis to facilitate the use of written formative assessment for Just-in-Time Teaching (JiTT) in a large-enrollment introductory biology course at a large public Midwestern university. A total of 12,677 student responses to 15 online homework questions were collected in three 300+ student course sections with four instructors. We used automated analysis to create feedback for instructors before the next class period (less than one working day), so that instructions could use this feedback to inform their instruction. Instructors used many of the questions pre- and post-instruction and the reports we provided to them allowed them to see how their students' answers changed as a result of their instruction. Focus groups with the instructors revealed that they already knew some of the topics that challenged students, as revealed in previous semesters with multiple-choice examinations. However, the instructors pointed out that the written assessments were particularly important for gaining insight as to why students have struggled continuously with these ideas.
Refereed DesignationRefereed


thumbnail of small NSF logo in color without shading

This material is based upon work supported by the National Science Foundation (DUE grants: 1438739, 1323162, 1347740, 0736952 and 1022653). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.