Designing and validating three-dimensional, learning-progression-aligned cognitive feedback for AI-generation
Abstract
To support learners as they develop knowledge-in-use abilities, it is critical that formative feedback be designed with intentionality and grounded in learning theory, such as learning progressions (LPs). In this paper, we share a process for generating cognitive feedback that aligns with a validated LP and a set of four general principles. These principles were previously established based on a literature review and interviews with teachers. The principles, along with the LP, guide the generation of the feedback statements (in this case, for a particular assessment item); however, analysis of the student response is equally important so as to make it personalized and meaningful. To validate the presence of the principles in our feedback statements, we conducted 13 think-aloud, semi-structured interviews with high school students. While the coding scheme represents a range of themes, this paper focuses on the broader theme of Evidence of feedback principles, sharing example excerpts as validity evidence. For example, several students found the feedback to be constructive and actionable, which they expressed by providing reasoning for changes from their pre- to post-feedback responses. The results here inform our ongoing efforts of training AI to generate feedback that aligns with these principles and the validated LP.