Modelos de Aprendizaje Profundo y Transformers para Series Temporales. Detección de anomalías

Abstract

Introducción al problema La detección de anomalías [NTND21] es una técnica utilizada para identificar patrones, comportamientos o datos que se desvían significativamente de lo esperado o considerado normal. Se trata, en la mayoría de los casos, de un problema no supervisado, ya que generalmente no se dispone de etiquetas que indiquen explícitamente qué instancias son anómalas y cuáles no, lo que exige que los modelos aprendan a distinguir lo inusual a partir de los datos disponibles. En aplicaciones prácticas, es fundamental que los algoritmos empleados no solo presenten un buen rendimiento en la detección de anomalías, sino que también sean lo suficientemente rápidos para operar en tiempo real o casi real, especialmente en entornos con grandes volúmenes de datos [SDSVC22]. Además, la interpretabilidad de los modelos es clave para garantizar que las decisiones tomadas por el sistema puedan ser comprendidas e interpretadas por expertos humanos, lo cual resulta especialmente importante en contextos críticos [LZVL23]. La detección de anomalías tiene aplicaciones muy amplias tanto en datos estáticos como en series temporales, permitiendo identificar comportamientos inusuales que pueden ser indicativos de errores, fraudes o situaciones críticas [CA22]. En el caso de datos estáticos, como registros de usuarios o características de productos, esta técnica permite detectar casos atípicos que pueden señalar fraudes, errores de entrada o incluso perfiles únicos que requieren atención especial. Por otro lado, cuando se aplica a series temporales, la detección de anomalías cobra especial relevancia en el mantenimiento predictivo, ya que permite anticipar fallos en maquinaria o sistemas antes de que ocurran, analizando desviaciones en variables monitorizadas a lo largo del tiempo. Además, este enfoque facilita el estudio de escenarios relevantes, como la identificación de patrones que preceden a eventos significativos (por ejemplo, caídas de rendimiento o picos de demanda), lo que ayuda a mejorar la toma de decisiones en sectores como la industria, la salud o las finanzas. En ambos casos, la detección temprana de anomalías contribuye a una gestión más eficiente de los recursos, a la reducción de costes y a una mayor capacidad de respuesta ante situaciones imprevistas. La predicción de series temporales tiene como objetivo estimar los valores futuros de una secuencia de datos basada en su comportamiento pasado [MMM23]. Esta tarea es especialmente relevante en contextos donde el tiempo juega un papel crucial, como en la monitorización de sensores, el análisis financiero o la previsión de demanda. La predicción de series temporales y la detección de anomalías está estrechamente ligada, ya que al comparar el valor predicho con el valor observado, podemos identificar desviaciones significativas que pueden indicar un comportamiento anómalo [CYPY21]. En este ámbito, las redes neuronales profundas han demostrado un rendimiento notable. Arquitecturas como las redes convolucionales, las redes recurrentes y los modelos basados en transformers han sido ampliamente utilizadas para capturar tanto patrones locales como dependencias a largo plazo en los datos temporales. Las convoluciones son eficaces para detectar patrones repetitivos o locales, las recurrentes destacan en el manejo de dependencias temporales y los transformers ofrecen ventajas en la paralelización y captación de relaciones globales dentro de la serie, lo que los convierte en herramientas muy potentes para modelar secuencias y detectar anomalías de forma precisa. El eje central de esta tesis es el desarrollo y aplicación de técnicas de detección de anomalías tanto para datos estáticos como para series temporales, teniendo en cuenta que la predicción de series temporales resulta un problema equivalente a la detección de anomalías. En particular, se abordan los siguientes tres objetivos: 1. El primero implica el desarrollo de algoritmos propios de detección de anomalías en entornos de grandes volúmenes de datos y flujos de datos. 2. El segundo objetivo plantea el estudio de arquitecturas de aprendizaje profundo para el modelado de series temporales, elaborando una librería como resultado tangible del objetivo. 3. El tercer objetivo se relaciona con el desarrollo de modelos propios basados en la arquitectura Transformer para predicción de series temporales.

Anomaly detection is a technique used to identify patterns or behaviors that deviate significantly from expected norms. It is commonly approached as an unsupervised problem, given the typical lack of labeled data indicating which instances are anomalous. Effective anomaly detection models must not only demonstrate high performance but also operate efficiently in real-time environments with large data volumes. Moreover, model interpretability is essential, particularly in critical domains where human experts must understand and validate the system’s decisions.

Time series forecasting involves estimating future values of a sequence based on historical observations. It plays a key role in domains such as sensor monitoring, financial analysis, and demand prediction. Anomaly detection and time series forecasting are closely related, as discrepancies between predicted and observed values may reveal anomalous behavior. In this context, deep neural networks have shown remarkable performance. Convolutional networks are adept at identifying local patterns, recurrent networks excel in handling temporal dependencies, and transformer-based models provide advantages in parallelization and capturing global relationships, making them powerful tools for modeling sequences and detecting anomalies.

Anomaly detection has wide-ranging applications in both static data and time series. In static contexts, such as user records or product features, it can uncover outliers indicative of fraud, data entry errors, or unique cases requiring special attention. In time series, it is especially relevant for predictive maintenance, enabling early fault detection through the analysis of deviations in monitored variables. This capability also supports the identification of patterns preceding significant events, thereby enhancing decision-making across sectors such as industry, healthcare, and finance. Ultimately, early anomaly detection leads to more efficient resource management, cost reduction, and improved responsiveness to unforeseen situations.

This thesis is structured into three main parts, each aligned with specific research objectives. The first part focuses on the development of interpretable models for anomaly detection in large-scale and streaming data environments. It introduces an extension of the Histogram-Based Outlier Scores algorithm, which enables batch updates of histograms. This approach allows for unsupervised anomaly detection using an interpretable model capable of providing valuable insights into its decision-making process. Its update capability makes it suitable for real-world and industrial applications, where data is both voluminous and incrementally served over time.

The second part explores the adaptation of convolutional and recurrent neural networks, commonly used in computer vision and natural language processing, to the domain of time series analysis. A wide range of such architectures is compiled and modularized, facilitating feature extraction and enabling their customization for tasks such as classification, time series forecasting, and anomaly detection. The outcome is a versatile library that supports multiple applications within the time series domain.

The final part presents a novel attention-based transformer model specifically designed for time series modeling. While transformer architectures have achieved significant success in language and vision tasks, their application to time series often overlooks the importance of local information. This proposal addresses that limitation, and the results obtained demonstrate high performance, positioning the model among the best-known transformer-based approaches for time series data.

This doctoral thesis has successfully fulfilled its initial objectives through three main contributions. First, it introduced original algorithms for anomaly detection in both static and temporal data environments. Notably, the Multi-step Histogram-Based Outlier Scores (MHBOS) algorithm was developed and validated using synthetic datasets and a real-world industrial predictive maintenance case. MHBOS demonstrated superior efficiency and performance compared to its predecessor (HBOS) and other state-of-the-art methods, achieving lower execution times and improved anomaly detection accuracy in time series, thus meeting the first objective.

Second, the thesis led to the development of a Python library named TSFEDL, which integrates twenty deep learning methods combining convolutional and recurrent layers for time series modeling. Built on Keras and PyTorch Lightning, the library allows flexible and efficient model creation and customization, making it suitable for integration into machine learning pipelines. It provides high-performance neural networks applicable to various tasks, thereby addressing the second objective by offering a robust tool for anomaly detection via distance metrics between predictions and actual values.

Third, a novel attention mechanism for transformers was proposed, specifically tailored for time series forecasting. Featuring logarithmic theoretical efficiency, this model outperformed advanced architectures such as Informer and Reformer. The work also emphasized the necessity of suitable datasets for long-term forecasting evaluation and introduced more precise benchmark datasets. The proposed model showed consistent improvements, particularly when employing deeper attention layers in complex scenarios, fulfilling the third objective and contributing significantly to the use of transformers in time series analysis.

These contributions not only advance the current state of the art but also open several avenues for future research. For the first contribution, further development could focus on enhancing the algorithm to combine multiple features and increase its explainability—especially relevant in unsupervised anomaly detection. For the second, the inclusion of transformer models into the TSFEDL library is proposed, along with improvements in its software architecture to enhance compatibility and usability. Finally, the third contribution suggests the potential for an integrated architecture based on data locality to complement the proposed Local Attention mechanism, potentially further improving current results.