Skip to yearly menu bar Skip to main content


Poster

MFTN: A Multi-scale Feature Transfer Network Based on IMatchFormer for Hyperspectral Image Super-Resolution

Shuying Huang · Mingyang Ren · Yong Yang · Xiaozheng Wang · Yingzhi Wei


Abstract:

Hyperspectral image super-resolution (HISR) aims to fuse a low-spatial-resolution hyperspectral image (LR-HSI) with a high-spatial-resolution multispectral image (HR-MSI) to obtain a hyperspectral image with high spectral and spatial resolution (HR-HSI). Due to some existing HISR methods ignoring the significant feature difference between LR-HSI and HR-MSI, the reconstructed HR-HSI typically exhibits spectral distortion and blurring of spatial texture. To solve this issue, we propose a multi-scale feature transfer network (MFTN) based on the improved feature matching Transformer (IMatchFormer) for HISR. Firstly, three multi-scale feature extractors with the same structure are constructed to extract features of different scales from the three input images, namely LR-HSI, HR-MSI, and degraded HR-MSI. Then, a multi-scale feature transfer module (MFTM) consisting of three IMatchFormers are designed to learn the detail features of different scales from HR-MSI by establishing the cross-model feature correlation between the LR-HSI and degraded HR-MSI. Finally, a multi-scale dynamic aggregation module (MDAM) containing three spectral aware aggregation modules (SAAMs) is constructed to reconstruct the final HR-HSI by gradually aggregating features of different scales. In SAAM, a spectral aware module (SAM) is designed to correct spectral features using shallow features of LR-HSI, thereby suppressing spectral information distortion. Extensive experimental results on three commonly used datasets demonstrate that the proposed model achieves better performance compared to state-of-the-art (SOTA) methods.

Live content is unavailable. Log in and register to view live content