Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution

Jing Yao, Danfeng Hong, Jocelyn Chanussot, Deyu Meng, Xiaoxiang Zhu, and Zongben Xu

---

Code for the paper: Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution.

Fig.1. An illustration of the proposed unsupervised hyperspectral super-resolution networks, called Coupled Unmixing Nets with Cross-Attention (CUCaNet), inspired by spectral unmixing techniques, which mainly consists of two important modules: cross-attention and spatial-spectral consistency.

Training

Please simply run ./Main_CAVE.py demo to reproduce our HSISR results on two HSIs (fake and real food and chart and staffed toy) of the CAVE dataset (Using PyTorch with Python 3.7 implemented on Windows OS).

• Before: Please click drive.google or pan.baidu (PIN:8zgj) to manully download the two demo HSIs (.mat files) and relocate them to ./CAVE/. For the required packages, please refer to detailed .py files.
• Parameters: The trade-off parameters as train_opt.lambda_* could be better tuned and the network hyperparameters are flexible.
• Results: Please see the five evaluation metrics (PSNR, SAM, ERGAS, SSIM, and UIQI) logged in ./checkpoints/CAVE_*name*/precision.txt and the output .mat files saved in ./Results/CAVE/.
• Runtime: ca. 1 hour per HSI using a single GTX2080.

Testing

#TODO

References

If you find this code helpful, please kindly cite:

[1] Yao, Jing, et al. "Cross-attention in coupled unmixing nets for unsupervised hyperspectral super-resolution." In Proceedings of the European Conference on Computer Vision (ECCV) (2020).

[2] Zheng, Ke, et al. "Coupled convolutional neural network with adaptive response function learning for unsupervised hyperspectral super-resolution." IEEE Transactions on Geoscience and Remote Sensing (2020), DOI: 10.1109/TGRS.2020.3006534.

BibTex

coming soon

Licensing

Copyright (C) 2020 Jing Yao and Danfeng Hong

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3 of the License.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program.

Contact

If you encounter any bugs while using this code, please do not hesitate to contact us.

Jing Yao (:incoming_envelope: jasonyao92@gmail.com) is with Xi'an Jiaotong University, China;

Danfeng Hong (:incoming_envelope: hongdanfeng1989@gmail.com) is with the Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Germany, and also with the Singnal Processing in Earth Oberservation (SiPEO), Technical University of Munich (TUM), Germany.