ePrints

MMCANet: A Multi-Modal and Cross-Attention Network for Cloud Removal and Exploration of Progressive Remote Sensing Images Restoration Algorithm

Lookup NU author(s): Professor Raj Ranjan

Downloads

Full text for this publication is not currently held within this repository. Alternative links are provided below where available.

Abstract

In Earth observation, cloud severely affects the interpretation of optical satellites generated high-resolution images. Cloud-free optical images are vital for downstream tasks such as semantic segmentation, object detection, etc. Thus, the elimination of clouds from optical imagery has emerged as a significant topic in remote sensing. Currently, most existing methods are proposed to leverage the texture information from auxiliary Synthetic Aperture Radar (SAR) images to restore cloud-free images via direct channel merging. However, such a unified feature extraction approach often neglects the inherent distribution disparity between SAR and optical images - the result of differing imaging principles-potentially leading to significant feature loss. To this end, we introduce a network by jointing SAR and optical images (MMCANet) to effectively extract multi-scale contextual features from SAR imagery and integrate them with optical features. Specifically, instead of simple concatenation of the channels of SAR and optical images, we obtain high-dimensional features from them through independent feature extractors. The integration of these features is facilitated by a cross-attention mechanism that provides a more fine-grained amalgamation of information. Meanwhile, an Atrous spatial pyramid pooling module is introduced into the integration of high-level features, which captures multi-scale contextual information around clouded areas. In addition, we propose four advanced remote sensing image restoration algorithms that approach image restoration as a series of subtasks, gradually eliminating clouds to enhance performance. Comprehensive assessments show that MMCANet performs well on the SEN 12 MS-CR dataset with PSNR of 39.8871, SSIM of 0.9672, MAE of 0.0081, and SAM of 2.9884.

Publication metadata

Author(s): Zhou Y, Suo J, Wang Y, Su J, Xiao W, Hong Z, Ranjan R, Wang L, Wen Z

Publication type: Article

Publication status: Published

Journal: IEEE Transactions on Geoscience and Remote Sensing

Year: 2025

Volume: 63

Online publication date: 01/04/2025

Acceptance date: 26/03/2025

ISSN (print): 0196-2892

ISSN (electronic): 1558-0644

Publisher: IEEE

URL: https://doi.org/10.1109/TGRS.2025.3556560

DOI: 10.1109/TGRS.2025.3556560

Altmetrics

Share