Conditional Diffusion Model as High-Dimensional Offline Resource Allocation Planner in Clu...

Due to network delays and scalability limitations, clustered ad hoc networks widely adopt Reinforcement Learning (RL) for on-demand resource allocation. Albeit its demonstrated agility, traditional Model-Free RL (MFRL) solutions struggle to tackle the huge action space, which generally explodes exponentially along with the number of resource allocation units, enduring low sampling efficiency and high computational complexity. To mitigate these limitations, Model-Based RL (MBRL) offers a solution by generating simulated samples through an environment model, which boosts sample efficiency and stabilizes the training by avoiding extensive real-world interactions. However, establishing an accurate dynamic model for complex and noisy environments necessitates a careful balance between model accuracy and computational complexity & stability. To address these issues, we propose a conditional Diffusion Model (DM) as high-dimensional offline resource allocation planner in multi-frequency time division multiple access (MF-TDMA) wireless ad hoc networks. By leveraging the astonishing generative capability of DMs, our approach takes advantage of generated high-quality samples to guide exploration and learn optimal policy. Extensive experiments show that our model outperforms MFRL in average reward and Quality of Service (QoS) while demonstrating comparable performance to other MBRL algorithms.

Conditional Diffusion Model as High-Dimensional Offline Resource Allocation Planner in Clustered MF-TDMA Ad Hoc Networks

Kechen Meng, Sinuo Zhang, Rongpeng Li, Chan Wang, Ming Lei, Minjian Zhao, Zhejiang University; Zhifeng Zhao, Zhejiang Lab