Low-Rate Semantic Communication with Codebook-based Conditional Generative Models

Generative semantic communication models are reshaping semantic communication frameworks by moving beyond pixel-wise optimization to align with human perception. However, many existing approaches prioritize image-level perceptual quality, often neglecting alignment with downstream tasks, which can lead to suboptimal semantic representation. This paper introduces an Ultra-Low Bitrate Semantic Communication (ULBSC) system that employs a conditional generative model and a learnable condition codebook. By integrating saliency conditions and image-level semantic information, the proposed method enables high-perceptual-quality and controllable task-oriented image transmission. Recognizing shared patterns among objects, we propose a codebook-assisted condition transmission method, integrated with joint source-channel coding (JSCC)-based text transmission to establish ULBSC. The codebook serves as a knowledge base, reducing communication costs to achieve ultra-low bitrate while enhancing robustness against noise and inaccuracies in saliency detection. Simulation results indicate that, under ultra-low bitrate conditions with an average compression ratio of 0.57‰, the proposed system delivers superior visual quality compared to traditional JSCC techniques and achieves higher saliency similarity between the generated and source images compared to state-of-the-art generative semantic communication methods.

Low-Rate Semantic Communication with Codebook-based Conditional Generative Models

Kailang Ye, Mingze Gong, Shuoyao Wang, Daquan Feng, Shenzhen University