Jiaxin Li, Ke Zheng, Jing Yao, Lianru Gao, and Danfeng Hong
Our paper is accpeted by IEEE Geoscience and Remote Sensing Letters (GRSL).
The final version can be downloaded in my researchgate or 🖼️PDF
More information can be found in my Google Scholar Citations.
Fig.1. Architecture of the proposed unsupervised degradations adaptive learning network, abbreviated as UDALN, for the task of HSI-MSI fusion.
Fig.2. Directory structure. There are three folders and six .py files in UDALN_GRSL-master.
This folder is used to store the training results and a folder named houston18_5_S1=0.001_20000_10000_S2=0.001_30000_20000_S3=6e-05_15000_5000 is given as a example.
-
convolution_hr2msi.pthis the trained result of SpeDnet,PSF.pthis the trained result of SpaDnet, andspectral_upsample.pthis the trained result of SpeUnet. -
opt.txtis used to store the training configuration. -
precision.txtis used to store the training precision. -
My_Out.matis the final reconstructed HHSI.
This folder is used to store the ground true HHSI and corresponding spectral response of multispectral imager. The HSI data used in 2018 IEEE GRSS Data Fusion Contest and spectral response of WorldView 2 multispectral imager are given as a example here.
This folder consists four .py files, including spatial_downsample.py(SpaDnet), spectral_downsample.py(SpeDnet), spectral_upsample.py(SpeUnet), and __init__.py.
-
config.py: all the parameters in our methed. -
Data_loader.py: generate the simulated low HSI and high MSI. -
evaluation.py: compute five metrics, which will be stored inprecision.txt. -
func.py: some functions used intrain_all_special.py. -
train_all_special.py: main.py
-
Requirements: codes of networks were tested using PyTorch 1.9.0 version (CUDA 11.4) in Python 3.8.10 on Windows system. For the required packages, please refer to detailed .py files.
-
Parameters: all the parameters need fine-tunning can be found in
config.py, including the learning rate decay strategy of three training stages. -
Data: put your HSI data and MSI spectral reponse in
./data/data_nameand./data/spectral_response, respectively.The HSI data used in 2018 IEEE GRSS Data Fusion Contest and spectral response of WorldView 2 multispectral imager are given as a example here. -
Run: just simply run
train_all_special.pyafter adjust the parameters inconfig.py. -
Results: one folder named
dataname_SF_S1=x1_y1_z1_S2=x2_y2_z2_S3=x3_y3_z3will be generated oncetrain_all_special.pyis run and all the results will be stored in the new folder. A folder namedhouston18_5_S1=0.001_20000_10000_S2=0.001_30000_20000_S3=6e-05_15000_5000is given as a example here.
Our work is inspired by the following paper
[1] Zheng, Ke, et al. "Coupled convolutional neural network with adaptive response function learning for unsupervised hyperspectral super-resolution." IEEE Transactions on Geoscience and Remote Sensing (2020), DOI: 10.1109/TGRS.2020.3006534.
[2] Yao, Jing, et al. "Cross-attention in coupled unmixing nets for unsupervised hyperspectral super-resolution." In Proceedings of the European Conference on Computer Vision (ECCV) (2020), pp. 208-224.
[3] Han, Xiaolin, et al. "Hyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks" Remote Sensing (2019), DOI: 10.3390/rs11101173.
If you encounter any bugs while using this code, please do not hesitate to contact us.
Jiaxin Li (:incoming_envelope: [email protected]) is currently pursuing the Ph.D. degree in cartography and geographic information system with the Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China.