Automated Analysis of Written Assessments in STEM: Methodological Issues

TitleAutomated Analysis of Written Assessments in STEM: Methodological Issues
Publication TypeConference Paper
Year of Publication2016
AuthorsVarious-Authors, M
Conference NameNational Assocation for Research in Science Teaching Annual Conference
Date Published04/2016
Conference LocationBaltimore, MD
AbstractConstructed response (CR) assessments, in which respondents use their own language to demonstrate knowledge, are widely viewed as providing greater insight into student cognition than closed form (e.g., multiple-choice) assessments. In the past, financial and time constraints made CR assessments challenging to execute and evaluate for large numbers of responses. The papers in this set use computerized tools to identify concepts (normative and naive) in written responses, allow iterative development of emergent coding schemes and create statistical scoring models of text. The goal is to develop accurate and reliable scoring models that are able to score written responses at levels equal to human expert scorers. This paper set provides an overview of computerized text analysis and features four papers highlighting different applications, methodological challenges and approaches to using these techniques in STEM teaching and teacher professional development. Attendees will 1) gain an understanding of the techniques applied to automated analysis of written assessments; 2) see the methods applied in different domains of STEM education, including science teacher education; 3) learn about methodological constraints; and 4) learn about the current status of this large collaborative project and its future directions.


thumbnail of small NSF logo in color without shading

This material is based upon work supported by the National Science Foundation (DUE grants: 1438739, 1323162, 1347740, 0736952 and 1022653). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.