Construct validity of computer scored constructed response items in undergraduate introductory biology courses

Abstract

"Psychometrically sound" assessments possess the ability to make valid inferences of students conceptual understanding. However, there is a lack of validated assessments in college biology education. This study assesses the psychometric properties of the constructed response (CR) biology questions to establish construct validity and reliability. We used a polytomous Rasch Partial Credit Model to validate the eight CR items. Responses from 437 students were scored by automated scoring models that were trained on data scored by experts. All items reflected unidimensional construct of biology and local independency. The findings suggest that the instrument is a promising and valid tool to assess undergraduates biology concepts from introductory biology courses, but needs further revision to strengthen the psychometric properties, adding more items for low performing students and improving person reliability. The significance of this study lies in its potential to contribute to creating more valid and reliable CR assessments through Rasch calibration.

Author

Hye You, Kevin Haudek, John Merrill, Mark Urban-Lurain

Year of Publication

2020

Book Title

Rasch Measurement: Applications in Quantitative Educational Research

Edition

1

Chapter

12

Number of Pages

223-240

Publisher

Springer Singapore

City

Singapore

DOI

10.1007/978-981-15-1800-3_12