The proposed method (aka DNA of HTTP) can identify the HTTP service packet newly captured by the Glastopf honeypot and distinguish whether it is a new abnormal type. The main technical steps of DNA of HTTP are as follows:
1.Signature-based and approximate signature-based methods are first applied in the process to take advantages of their low false positive rate. After that, a new anomaly-based detection method is used to detect new anomalies that might be missed by signature-based methods.
2.To avoid pairwise comparisons among all historic packets, Minibatch K-Means++ clustering algorithm is used.
3.A new anomaly index is computed to indicate the novelty of the packet.
4.Finally the detection results are presented on the Kibana dashboard for easy inspection by analysts.
5.The experimental results show that the proposed DNA of HTTP can reduce the number of packets inspected by analysts by a factor of 10 to 100, and the F1 score is about 99% on detection new anomalies.

The proposed DNA of HTTP system combines the advantages of three types of detection methods including signature-based, approximate signature-based, and new anomaly-based methods. It can learn abnormal traffic captured by honeypots with unlabeled, complex patterns, and even every imbalanced data. It can identify new anomalies accurately and reduce the time required for security personnels to inspect the log.

This system aims at establishing a semi-supervised new anomaly detection system for the honeypot deployed by information security research institutions. The proposed methods can automatically and quickly identify whether a packet captured by the honeypot is a new anomaly and compute a new anomaly index to significantly reduce the time required for analysts to inspect the log.