Towards understanding malware behaviour by the extraction of API calls

Mamoun Alazab*, Sitalakshmi Venkataraman, Paul Watters

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

113 Citations (Scopus)


One of the recent trends adopted by malware authors is to use packers or software tools that instigate code obfuscation in order to evade detection by antivirus scanners. With evasion techniques such as polymorphism and metamorphism malware is able to fool current detection techniques. Thus, security researchers and the anti-virus industry are facing a herculean task in extracting payloads hidden within packed executables. It is a common practice to use manual unpacking or static unpacking using some software tools and analyse the application programming interface (API) calls for malware detection. However, extracting these features from the unpacked executables for reverse obfuscation is labour intensive and requires deep knowledge of low-level programming that includes kernel and assembly language. This paper presents an automated method of extracting API call features and analysing them in order to understand their use for malicious purpose. While some research has been conducted in arriving at file birthmarks using API call features and the like, there is a scarcity of work that relates to features in malcodes. To address this gap, we attempt to automatically analyse and classify the behavior of API function calls based on the malicious intent hidden within any packed program. This paper uses four-step methodology for developing a fully automated system to arrive at six main categories of suspicious behavior of API call features.

Original languageEnglish
Title of host publication2010 Second Cybercrime and Trustworthy Computing Workshop (CTC 2010)
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages8
ISBN (Electronic)9780769541860
ISBN (Print)9781424480548
Publication statusPublished - Jul 2010
Externally publishedYes
Event2nd Cybercrime and Trustworthy Computing Workshop, CTC - 2010 - Ballarat, Australia
Duration: 19 Jul 201020 Jul 2010


Other2nd Cybercrime and Trustworthy Computing Workshop, CTC - 2010


Dive into the research topics of 'Towards understanding malware behaviour by the extraction of API calls'. Together they form a unique fingerprint.

Cite this