A Late Multi-Modal Fusion Model for Detecting Hybrid Spam E-mail. (arXiv:2210.14616v3 [cs.CR] UPDATED)

In recent years, spammers are now trying to obfuscate their intents by
introducing hybrid spam e-mail combining both image and text parts, which is
more challenging to detect in comparison to e-mails containing text or image
only. The motivation behind this research is to design an effective approach
filtering out hybrid spam e-mails to avoid situations where traditional
text-based or image-baesd only filters fail to detect hybrid spam e-mails. To
the best of our knowledge, a few studies have been conducted with the goal of
detecting hybrid spam e-mails. Ordinarily, Optical Character Recognition (OCR)
technology is used to eliminate the image parts of spam by transforming images
into text. However, the research questions are that although OCR scanning is a
very successful technique in processing text-and-image hybrid spam, it is not
an effective solution for dealing with huge quantities due to the CPU power
required and the execution time it takes to scan e-mail files. And the OCR
techniques are not always reliable in the transformation processes. To address
such problems, we propose new late multi-modal fusion training frameworks for a
text-and-image hybrid spam e-mail filtering system compared to the classical
early fusion detection frameworks based on the OCR method. Convolutional Neural
Network (CNN) and Continuous Bag of Words were implemented to extract features
from image and text parts of hybrid spam respectively, whereas generated
features were fed to sigmoid layer and Machine Learning based classifiers
including Random Forest (RF), Decision Tree (DT), Naive Bayes (NB) and Support
Vector Machine (SVM) to determine the e-mail ham or spam.