Exploring Data Augmentation and Resampling Strategies for Transformer-Based Models to Address Class Imbalance in AI Scoring of Scientific Explanations in NGSS Classroom
Abstract
Automated scoring of students’ scientific explanations offers the potential for immediate, accurate feedback, yet class imbalance in rubric categories—particularly those capturing advanced reasoning—remains a challenge. This study investigates augmentation strategies to improve transformer-based text classification of student responses to a physics assessment based on an NGSS-aligned learning progression. The dataset consists of 1,466 high school responses scored on 11 binary-coded analytic categories. We focus on Categories 5 and 6, which represent sophisticated energy and Coulombic reasoning but are highly imbalanced (3.9% and 1.2% positive cases, respectively). Using SciBERT as a baseline, we applied fine-tuning and test two augmentation strategies: (1) GPT-4–generated synthetic responses, and (2) EASE, a word-level extraction and filtering approach. While fine-tuning SciBERT improved recall over baseline, augmentation substantially enhanced performance, with GPT-4 data boosting both precision and recall, and EASE achieving perfect precision, recall, and F1 scores. We selected augmentation over traditional over/undersampling to avoid overfitting and retain novice-level data critical for learning progression alignment. Findings demonstrate that targeted augmentation can address severe imbalance while preserving conceptual coverage, offering a scalable solution for automated learning progression-aligned scoring in science education.