Better Model, Worse Predictions: The Dangers in Student Model Comparisons

Warning

This publication doesn't include Faculty of Arts. It includes Faculty of Informatics. Official publication website can be found on muni.cz.

Authors

ČECHÁK Jaroslav PELÁNEK Radek

Year of publication 2021
Type Article in Proceedings
Conference International Conference on Artificial Intelligence in Education
MU Faculty or unit

Faculty of Informatics

Citation
Doi http://dx.doi.org/10.1007/978-3-030-78292-4_40
Keywords Additive factor model; Student modeling; Simulation; Model comparison
Description The additive factor model is a widely used tool for analyzing educational data, yet it is often used as an off-the-shelf solution without considering implementation details. A common practice is to compare multiple additive factor models, choose the one with the best predictive accuracy, and interpret the parameters of the model as evidence of student learning. In this work, we use simulated data to show that in certain situations, this approach can lead to misleading results. Specifically, we show how student skill distribution affects estimates of other model parameters.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.