Multivariate Statistical Network Monitoring for Network Security based on Principal Component Analysis

Abstract

Currently we live in hyper-connected world, which is one of the main causes for the fast propagation of Information Technology (IT) Security attacks. An IT Security incident can impact both in the economy and the reputation of the organization that suffers it. Thus, IT Security is a prior concern for any organization. Another important issue related to IT Security threats is that the time required for compromising a network is, on average, in the order of minutes, while the security team may need months to detect an incident after it takes place. This makes it necessary to enhance the mechanisms of intrusion detection to improve the capability of prioritization and classification of IT security alarms. With the appropriate tools, the security team can detect the incidents timely without being overwhelmed by an excessive number of alarms. Network security is of utmost importance within IT Security, and it aims to make the communications infrastructure secure from the point of view of the IT. In general, there are three approaches for network security: prevention, detection and response. These approaches can be combined to achieve a comprehensive security system. A practical combination of the detection and response dimensions is the so-called Network Security Monitoring (NSM), which is an approach that aims to detect the incidents in a network by monitoring the network traffic. NSM is carried out by collecting, combining and analyzing different sources of information, in order to detect and notify intrusions. There are two main techniques for incident detection: Signature based, which allows to detect attacks from previously defined patterns; and Anomaly-based, which allows to detect deviations from the normal behavior in a network, captured in a previously trained model. Multivariate Statistical Network Monitoring (MSNM) is an NSM methodology that follows an anomaly-based detection scheme that extends the Multivariate Statistical Process Control (MSPC) theory, developed in the area of industrial process research. MSPC consists in two phases: phase I, detection of assignable causes of variation in the calibration data that are corrected and eliminated until the process is under Normal Operation Condition (NOC); and phase II, monitoring of new data to detect (and diagnose) anomalies. MSNM applies this philosophy to traffic network data, adding two prior steps: parsing and fusion, which are needed to combine information from different data sources in NSM. MSNM is useful to prioritize and diagnose anomalies, which is congruent with the security team’s workflow. In this PhD, we start from the MSNM methodology and introduce a number of enhancements: i) a pre-processing method to consider the cyclostationarity of the data (e.g. the cycles existing during day and night or weeks and weekends), ii) a methodology for the comparison of diagnosis methods, and iii) a univariate method for diagnosis. Furthermore, the pre-processing and diagnosis methods, as well as some of other existing extensions for MSNM are evaluated and compared with other reference methods using a real network data set for the first time. The application on real network data allows to assess the MSNM extensions under realistic conditions, yielding a more accurate perspective of their performance. This research work shows the existing symbiosis between industrial processes and network security, introducing enhancements that are of interest for both topics and that open new lines of research exploring the synergy between MSPC and MSNM.