An Intelligent MPTCP Scheduling for Secure IoT Communications in 5G-6G Heterogeneous Networks: A Reinforcement Learning Approach

Abstract

The mass deployment of Internet of Things (IoT) devices in 5G and future 6G Heterogeneous Networks (HetNets) leads to substantial challenges on reliable data transmission, security, and quality of service (QoS). Multipath TCP (MPTCP) offers promising solutions by enabling simultaneous utilization of multiple network paths; however, conventional scheduling algorithms fail to address the dynamic, heterogeneous, and security sensitive nature of modern IoT environments. This paper proposes an Intelligent MPTCP Scheduling framework, termed I-MPTCP-Sec, that integrates a Deep Reinforcement Learning (DRL) based packet scheduler with a lightweight IoT security layer for 5G/6G networks. The proposed system dynamically selects optimal subflow paths while simultaneously detecting and mitigating security threats including DDoS attacks, eavesdropping, and man in the middle (MitM) attacks. A novel composite reward function incorporating throughput, latency, path reliability, energy efficiency, and security score is designed for the DRL agent, and its boundedness is formally proved. Extensive Python based discrete event simulation experiments using a custom engine with TensorFlow 2.12 for DRL/FIDS and 3GPP TR 38.901 stochastic channel models demonstrate that I-MPTCP-Sec achieves up to 33% improvement in throughput, 24.4% reduction in end-to-end latency, and 96.4% attack detection accuracy compared to LowRTT, Round Robin, and BLEST baselines. Energy consumption is reduced by 29.8% relative to Round Robin, and per decision processing overhead remains below 8 ms at 500 devices, satisfying 5G URLLC real time requirements. An ablation study confirms the individual contribution of each framework module. The framework is validated across smart healthcare, smart city, and industrial IoT (IIoT) scenarios.