Mutual information and feature importance gradient boosting: automatic byte n-gram feature reranking for Android malware detection

Mahmood Yousefi-Azar*, Vijay Varadharajan, Len Hamey, Shiping Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)

Abstract

The fast pace evolving of Android malware demands for highly efficient strategy. That is, for a range of malware types, a malware detection scheme needs to be resilient and with minimum computation performs efficient and precise. In this paper, we propose Mutual Information and Feature Importance Gradient Boosting (MIFIBoost) tool that uses byte n-gram frequency. MIFIBoost consists of four steps in the model construction phase and two steps in the prediction phase. For training, first, n-grams 2 ≤ n ≤ 4 of both the classes.dex and AndroidManifest.xml binary files are obtained. Then, MIFIBoost uses Mutual Information (MI) to determine the top most informative items from the entire n-gram vocabulary. In the third phase, MIFIBoost utilizes the Gradient Boosting algorithm to re-rank these top n-grams. For testing, MIFIBoost uses the learned vocabulary of byte n-grams term-frequency (tf) to feed into the classifier for prediction. Thus, MIFIBoost does not require reverse engineering. A key insight from this work is that filtering using XGBoost helps us to address the hard problem of detecting obfuscated malware better while having a negligible impact on nonobfuscated malware. We have conducted a wide range of experiments on four different datasets one of which is obfuscated, and MIFIBoost outperforms state-of-the-art tools. MIFIBoost's f1-score for Drebin, DexShare, and AMD datasets is 99.1%, 98.87%, and 99.62%, respectively, a False Positive Rate of 0.41% using AMD dataset. On average, the False Negative Rate of MIFIBoost is 2.1% for the PRAGuard dataset in which seven different obfuscation techniques are implemented. In addition to fast run-time performance and resiliency against obfuscated malware, the experiments show that MIFIBoost performs quite efficiently for five zero-day families with 99.78% AUC.

Original languageEnglish
Pages (from-to)1518-1539
Number of pages22
JournalSoftware: Practice and Experience
Volume51
Issue number7
Early online date5 Apr 2021
DOIs
Publication statusPublished - Jul 2021

Keywords

  • automatic feature ranking
  • byte-level n-gram
  • gradient boosting
  • malware detection
  • static tool

Fingerprint

Dive into the research topics of 'Mutual information and feature importance gradient boosting: automatic byte n-gram feature reranking for Android malware detection'. Together they form a unique fingerprint.

Cite this