Impact of Regressand Stratification in Dataset Shift Caused by Cross-Validation
Metadata
Show full item recordEditorial
MDPI
Materia
Cross-validation Dataset shift Target shift Stratification Regression
Date
2022-07-21Referencia bibliográfica
Sáez, J.A.; Romero-Béjar, J.L. Impact of Regressand Stratification in Dataset Shift Caused by Cross-Validation. Mathematics 2022, 10, 2538. [https://doi.org/10.3390/math10142538]
Sponsorship
MCIU/AEI/ERDF, UE PGC2018098860-B-I00; ERDF Operational Programme 2014-2020; Economy and Knowledge Council of the Regional Government of Andalusia, Spain MCIN/AEI CEX2020-001105-M A-FQM-345-UGR18Abstract
Data that have not been modeled cannot be correctly predicted. Under this assumption, this
research studies how k-fold cross-validation can introduce dataset shift in regression problems. This
fact implies data distributions in the training and test sets to be different and, therefore, a deterioration
of the model performance estimation. Even though the stratification of the output variable is widely
used in the field of classification to reduce the impacts of dataset shift induced by cross-validation, its
use in regression is not widespread in the literature. This paper analyzes the consequences for dataset
shift of including different regressand stratification schemes in cross-validation with regression data.
The results obtained show that these allow for creating more similar training and test sets, reducing
the presence of dataset shift related to cross-validation. The bias and deviation of the performance
estimation results obtained by regression algorithms are improved using the highest amounts of
strata, as are the number of cross-validation repetitions necessary to obtain these better results.