Tackling the problem of noisy IoT sensor data in smart agriculture: regression noise filters for enhanced evapotranspiration prediction

Abstract

In smart agriculture, the accurate prediction of evapotranspiration plays a crucial role in optimizing water usage and maximizing crop yield. However, the increasing adoption of IoT sensor technologies has resulted in the accumulation of large amounts of data, which are frequently contaminated by noise and pose a significant challenge to extract reliable knowledge through data modeling. This research addresses the problem of noisy IoT sensor data and its impact on evapotranspiration prediction, an essential aspect of agricultural practices. The effect of noise on sensor variables and evapotranspiration is extensively analyzed by simulating different noise levels in evapotranspiration datasets collected from various agricultural areas in Spain, enabling a comprehensive evaluation of its impact on the performance of data science models. Despite the potential consequences of this type of errors, a noise preprocessing stage is often overlooked in existing literature in this field, which is necessary to improve data quality prior to modeling. In order to address this challenge, this paper proposes the usage of regression noise filters as approach to mitigate the detrimental effects of noisy IoT sensor data on evapotranspiration prediction. Additionally, we introduce the rgnoisefilt R package, which offers a practical and efficient implementation of noise filtering techniques for regression datasets, providing a valuable solution for handling noisy data in smart agriculture applications. The experimental results obtained emphasize the negative impacts of noise on evapotranspiration prediction performance and highlight the importance of an appropriate data treatment to mitigate system deterioration. Furthermore, the findings of this research emphasize the efficacy of the regression noise filters implemented in the rgnoisefilt software, enhancing the performance of the models built and providing a valuable tool for improving data quality in smart agriculture.