Benchmarking Seven Classifiers for Student Dropout Prediction on the UCI Higher Education Dataset

Abstract

Attrition among students remains an ongoing problem in universities, so classifiers whose results can be compared and reproduced are needed. In this study, seven classical classifiers are tested using the public UCI Predict Students' Dropout and Academic Success data set containing 4,424 undergraduates with 36 attributes. The data set is recoded into 1,421 dropouts and 3,003 graduated or enrolled students. Logistic regression, decision tree, random forest, support vector machine, naive Bayes, k-nearest neighbors, and XGBoost are compared with one stratified 5-fold cross-validation scheme. Class weights are applied to the five classifiers that support them while the decision threshold for each classifier is optimized for F1 score. XGBoost yields the highest results (accuracy 0.8804, ROC-AUC 0.9242, F1-score 0.8076 at a decision threshold of 0.53), followed closely by random forest and logistic regression. A repeat with fixed seeds and in-fold scaling gives accuracy, ROC-AUC, and F1-score close to those reported (within 0.0012), and the bootstrap 95% confidence intervals for the top three classifiers overlap, so these models are not reliably separated. Nested cross-validation caused a drop in the XGBoost F1-score of only 0.0002 on the primary split and of 0.0037 on average over six splits. Gain, permutation, and SHAP importance all show that approved second-semester units and tuition fee status play the most important role, but differ on the lower-ranked features. Without the second-semester features, the XGBoost ROC-AUC dropped from 0.9241 to 0.9037. These results refer to one particular dataset of one institution.