Evaluation of MQTT Traffic Security with and without Encryption in a Simulated 5G-IoT Environment

Abstract

This article analyzes vulnerabilities in a 5G simulation environment to assess the security of IoT device communication through the MQTT (Message Queuing Telemetry Transport) protocol. A laboratory scenario was designed with legitimate IoT messages and simulated attacks, using both unencrypted traffic and traffic protected with AES-CFB (Advanced Encryption Standard in Cipher Feedback mode). The analyzer classified messages, verified broker responses, and evaluated security measures. Results show that encryption partially protects but does not prevent attacks without strong authentication, highlighting the need for additional mechanisms such as mutual authentication and digital signatures to ensure comprehensive communication security. It was also found that although encryption provides confidentiality, it does not prevent attacks in the absence of robust authentication. In the encrypted scenario, 66.20% of the traffic corresponded to attacks, compared to 48.80% in the unencrypted scenario, indicating that encryption does not reduce the proportion of malicious traffic and, in fact, increased the acceptance of attack messages by 17.4% compared to the non-encrypted case. Additionally, the sending rate decreased from 19.02 msg/s without encryption to 13.98 msg/s with encryption, highlighting the impact of the cryptographic process on performance. In conclusion, effective IoT protection in 5G environments requires combining encryption with mutual authentication, digital signatures, and advanced integrity mechanisms. The results provide a replicable foundation for educational and training environments focused on IoT cybersecurity.