AI Powered Threat Detection Framework for Security Enhancement of MQTT based IoT Networks
DOI:
https://doi.org/10.62760/iteecs.5.1.2026.171Keywords:
MQTT, IoT Security, Machine Learning, Deep Learning, Intrusion DetectionAbstract
The Message Queuing Telemetry Transport (MQTT) protocol is widely adopted in IoT networks due to its lightweight and scalable design, making it ideal for resource-constrained devices. However, its minimal architecture lacks built-in encryption and robust identity authentication, leaving it vulnerable to a range of security threats. To address these vulnerabilities, this work proposes a hybrid framework that integrates machine learning (ML) and deep learning (DL) models to enhance intrusion and anomaly detection in MQTT based IoT networks. The framework leverages real world IoT traffic data from the MQTTEEB-D dataset with diverse preprocessing techniques to train the model. The framework is implemented and evaluated through a multi-metric approach, where each metric assesses a distinct aspect of the framework to identify lightweight processing models for resource-constrained environments. Results consistently showed that ML models outperformed their DL counterparts in terms of detection reliability, classification balance, and error minimization. While DL models demonstrated moderate effectiveness in capturing temporal patterns, they exhibited higher misclassification rates and reduced calibration. The findings underscore the effectiveness of lightweight ML models for scalable and dependable intrusion detection in MQTT based IoT networks.
References
R. Chataut, A. Phoummalayvane, R. Akl “Unleashing the power of IoT: A comprehensive review of IoT applications and future prospects in healthcare, agriculture, smart homes, smart cities, and Industry 4.0”, Sensors, Vol. 23, No. 16, art. no. 7194, 2023. https://doi.org/10.3390/s23167194
T. Xinyu, K. Pekka, H. Ismo “A design and modeling approach for resource constrained internet of things devices”, Design Automation for Embedded Systems, Vol. 29, No. 4, 2025. https://doi.org/10.1007/s10617-025-09294-6
A. J. Hintaw, S. Manickam, S. Karuppayah, M. F. Aboalmaaly “A brief review on MQTT’s security issues within the Internet of Things (IoT)”, Journal of Communications, Vol. 14, No. 6, pp. 463–469, 2019. https://doi.org/10.12720/jcm.14.6.463-469
T. Mazhar, D. B. Talpur, T. Al Shloul, Y. Y. Ghadi, I. Haq, I. Ullah, K. Ouahada, H. Hamam “Analysis of IoT security challenges and its solutions using artificial intelligence”, Brain Sciences, Vol. 13, No. 4, art. no. 683, 2023. https://doi.org/10.3390/brainsci13040683
A. Kumar, J. A. Gutierrez “Impact of machine learning on intrusion detection systems for the protection of critical infrastructure”, Information, Vol. 16, No. 7, art. no. 515, 2025. https://doi.org/10.3390/info16070515
F. Alwahedi, A. Aldhaheri, M. A. Ferrag, A. Battah, N. Tihanyi “Machine learning techniques for IoT security: Current research and future vision with generative AI and large language models”, Internet of Things and Cyber-Physical Systems, Vol. 4, pp. 167–185, 2024. https://doi.org/10.1016/j.iotcps.2023.12.003
M. S. Ahsan, A. S. K. Pathan “A comprehensive survey on the requirements, applications, and future challenges for access control models in IoT: The state of the art”, IoT, Vol. 6, No. 1, art. no. 9, 2025. https://doi.org/10.3390/iot6010009
Ch. Amarendra, D. Balakotaiah, P. V. Rajulu, A. P. Sridhar, T. K. Mohana “Decoding the Internet of Things: A comprehensive survey paper”, International Transactions on Electrical Engineering and Computer Science, Vol. 4, No. 2, pp. 103–118, 2025. https://doi.org/10.62760/iteecs.4.2.2025.136
J. D. Gadze, A. A. Bamfo-Asante, J. O. Agyemang, H. Nunoo-Mensah, K. A. B. Opare “An investigation into the application of deep learning in the detection and mitigation of DDoS attack on SDN controllers”, Technologies, Vol. 9, No. 1, art. no. 14, 2021. https://doi.org/10.3390/technologies9010014
N. Ahuja, G. Singal, D. Mukhopadhyay, N. Kumar “Automated DDoS attack detection in software defined networking”, Journal of Network and Computer Applications, Vol. 187, art. no. 103108, 2021. https://doi.org/10.1016/j.jnca.2021.103108
Z. Wang, Y. Zeng, Y. Liu, D. Li “Deep belief network integrating improved kernel-based extreme learning machine for network intrusion detection”, IEEE Access, Vol. 9, pp. 16062–16091, 2021. https://doi.org/10.1109/ACCESS.2021.3051074
S. Dotcenko, A. Vladyko, I. Letenko “A fuzzy logic based information security management for software defined networks”, 16th International Conference on Advanced Communication Technology (ICACT), pp. 167–171, 2014. https://doi.org/10.1109/ICACT.2014.6778942
T. Abhiroop, S. Babu, B. S. Manoj “A machine learning approach for detecting DoS attacks in SDN switches”, 2018 National Conference on Communications (NCC), pp. 1-6, 2018. https://doi.org/10.1109/NCC.2018.8600196
K. S. Sahoo, A. Iqbal, P. Maiti, B. Sahoo “A machine learning approach for predicting DDoS traffic in software defined networks”, 2018 International Conference on Information Technology (ICIT), pp. 199–203, 2018. https://doi.org/10.1109/ICIT.2018.00049
M. Carlier, K. Steenhaut, A. Braeken “Symmetric key based security for multicast communication in wireless sensor networks”, Computers, Vol. 8, No. 1, art. no. 27, 2019. https://doi.org/10.3390/computers8010027
P. Shabisha, A. Braeken, P. Kumar, K. Steenhaut “Fog-orchestrated and server-controlled anonymous group authentication and key agreement”, IEEE Access, Vol. 7, pp. 150247–150261, 2019. https://doi.org/10.1109/ACCESS.2019.2946713
A. Chandramouli, A. Choudhury, A. Patra “A survey on perfectly secure verifiable secret-sharing”, ACM Computing Surveys, Vol. 54, No. 11s, pp. 1-36, 2022. https://doi.org/10.1145/3512344
M. Tiloca, K. Nikitin, S. Raza “Axiom: DTLS-based secure IoT group communication”, ACM Transactions on Embedded Computing Systems, Vol. 16, No. 3, pp. 1-29, 2017. https://doi.org/10.1145/3047413
H. Zeghida, M. Boulaiche, R. Chikh, A. M. Bamhdi, A. L. B. Barros, D. Zeghida, A. Patel “Enhancing IoT cyber attacks intrusion detection through GAN-based data augmentation and hybrid deep learning models for MQTT network protocol cyber attacks”, Cluster Computing, Vol. 28, art. no. 58, 2025. https://doi.org/10.1007/s10586-024-04752-5
I. H. Putro, T. Ahmad, R. M. Ijtihadie “Enhancing MQTT intrusion detection in IoT using machine learning and feature engineering”, IEEE Open Journal of the Communications Society, Vol. 6, pp. 7855–7884, 2025. https://doi.org/10.1109/OJCOMS.2025.3610132
M. Solanki, S. Gupta “A novel intrusion detection framework using ensemble learning in MQTT IoT applications”, Annals of Mathematics and Artificial Intelligence, 2025. https://doi.org/10.1007/s10472-025-09993-7
M. Swain, N. Tripathi, K. Sethi “Identifying communication sequence anomalies to detect DoS attacks against MQTT”, Computers & Security, Vol. 157, art. no. 104526, 2025. https://doi.org/10.1016/j.cose.2025.104526
H. Run-Ze, S. Jun-Jian, Q. Su-Juan, J. Zheng-Ping, and F. Gao “QGAN-based data augmentation for hybrid quantum–classical neural networks”, Chinese Journal of Physics, Vol. 97, pp. 1453-1463, 2025. https://doi.org/10.1016/j.cjph.2025.07.017
A. Aqachtoul, K. Karam, A. Elamrani, M. Najib, N. Rafalia, M. Bakhouya “MQTTEEB-D: A real-world IoT cybersecurity dataset for AI-powered threat detection in MQTT networks”, Data in Brief, Vol. 62, art. no. 111897, 2025. https://doi.org/10.1016/j.dib.2025.111897
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Ali Kareem Alhossainy, Mina Malekzadeh
This work is licensed under a Creative Commons Attribution 4.0 International License.
This Journal and its metadata are licenced under a 