Anteater: Advanced Persistent Threat Detection With Program Network Traffic Behavior

Yangzong Zhang, Wenjian Liu, Kaiian Kuok, Ngai Cheong

Research output: Contribution to journalArticlepeer-review


Recent stealth attacks cleverly disguise malicious activities, masquerading as ordinary connections to popular online services through seemingly innocuous applications. These methods often evade detection by traditional network monitoring or signature-based techniques, as attackers frequently hide Command and Control (C&C) servers within well-known cloud service providers, making the traffic anomalies appear normal. In this paper, we introduce an application-level monitoring system, Anteater. Anteater constructs a detailed profile for each legitimate software's network traffic behavior, outlining the expected traffic patterns. By scrutinizing a program's network traffic configuration, Anteater efficiently pinpoints and intercepts the IP addresses associated with abnormal program access. Implemented in a real-world enterprise environment, Anteater was tested on a dataset containing over 400 million real-world network traffic sessions. The evaluation results demonstrate that Anteater achieves a high detection rate for malware injections, boasting a true positive rate of 94.5% and a false positive rate of less than 0.1%.

Original languageEnglish
Pages (from-to)8536-8551
Number of pages16
JournalIEEE Access
Publication statusPublished - 2024


  • Anteater
  • Malware injection detection
  • advanced persistent threat
  • network security
  • program traffic behavior


Dive into the research topics of 'Anteater: Advanced Persistent Threat Detection With Program Network Traffic Behavior'. Together they form a unique fingerprint.

Cite this