Impact of Regressand Stratification in Dataset Shift Caused by Cross-Validation Sáez Muñoz, José Antonio Romero Béjar, José Luis Cross-validation Dataset shift Target shift Stratification Regression Data that have not been modeled cannot be correctly predicted. Under this assumption, this research studies how k-fold cross-validation can introduce dataset shift in regression problems. This fact implies data distributions in the training and test sets to be different and, therefore, a deterioration of the model performance estimation. Even though the stratification of the output variable is widely used in the field of classification to reduce the impacts of dataset shift induced by cross-validation, its use in regression is not widespread in the literature. This paper analyzes the consequences for dataset shift of including different regressand stratification schemes in cross-validation with regression data. The results obtained show that these allow for creating more similar training and test sets, reducing the presence of dataset shift related to cross-validation. The bias and deviation of the performance estimation results obtained by regression algorithms are improved using the highest amounts of strata, as are the number of cross-validation repetitions necessary to obtain these better results. 2022-09-12T10:40:45Z 2022-09-12T10:40:45Z 2022-07-21 info:eu-repo/semantics/article Sáez, J.A.; Romero-Béjar, J.L. Impact of Regressand Stratification in Dataset Shift Caused by Cross-Validation. Mathematics 2022, 10, 2538. [https://doi.org/10.3390/math10142538] http://hdl.handle.net/10481/76646 10.3390/math10142538 eng http://creativecommons.org/licenses/by/4.0/ info:eu-repo/semantics/openAccess Atribución 4.0 Internacional MDPI