A Supervised Rare Anomaly Detection Technique via Cooperative Co-evolution-Based Feature Selection Using Benchmark UNSW_NB15 Dataset

Anomaly detection is important in many domains, including cybersecurity. There are a number of rare anomalies in cybersecurity datasets, and detection of these rare anomalies is computationally expensive. Cybersecurity datasets consist of many features, mostly irrelevant, resulting in lower classification performance of many machine learning algorithms. Therefore, a feature selection approach to select only the relevant features from a dataset is an important preprocessing step in anomaly detection. Many feature selection approaches are available in the literature. However, to deal with Big Data, cooperative co-evolution, a meta-heuristic algorithm-based feature selection approach is more suitable for cybersecurity datasets for its preprocessing step. This paper has applied our previously proposed cooperative co-evolution-based feature selection with random grouping (CCFSRFG) approach to the UNSW_NB15 cybersecurity dataset as the preprocessing step. Then, the original dataset and the dataset with a reduced number of features are used to detect the rare anomalies. The experimental analysis was performed and evaluated using five widely used supervised classifiers. Hence, the proposed anomaly detection approach is called Supervised Rare Anomaly Detection (SRAD). The experimental results were compared with and without feature selection in terms of true positive rate (TPR). The experimental analysis indicates that the naïve Bayes classifier increased the TPR by 25.55% for all rare anomaly detection. Furthermore, the k-NN classifier increased the TPR of Exploits anomaly detection by 58.91%.