高光譜影像(Hyperspectral Image)以及多光譜影像(Multispectral Image)融合常被用來解決高光譜影像問題，旨在融合低解析度高光譜影像(LRHSI)以及高解析度多光譜影像(HRMSI)，是目前最常見的方法之一，通常高光譜影像的空間解析度較低，且直接取得高解析度之高光譜影像具有高昂的成本，而透過融合獲取高解析度高光譜影像是一種經濟實惠的方法。在影像處理領域融合方法是一種關鍵技術，因為高解析高光譜影像很好的促進了遠程材料辨識及分類任務，從而在衛星遙感領域獲得很大的關注。在衛星遙感領域很少有人嘗試使用Transformer，而Transformer在很多高級視覺任務中表現出驚人的成果，在本文中，我們提出了處理HSI/MSI融合任務的網路模型，基於SwinTansformer以及深度卷積網路(DCNN)之融合網路，稱為SwinDFN，SwinDFN由兩個部分組成：1)傳統卷積神經網路對HSI以及MSI影像初步融合，其中引入了Depthwise卷積技術來更有效地結合 HSI 和 MSI 之間的光譜響應函數以及對網路參數量做壓縮，2)具有殘差結構的SwinTansformer特徵提取模塊，來對影像特徵做增強，所提出之方法實現了基於規模較小的網路達到較好的HSI/MSI融合性能。

Hyperspectral image and multispectral image fusion are often used to solve hyperspectral image problems, aiming at fusing low-resolution hyperspectral image (LRHSI) and high-resolution multispectral image (HRMSI), which is currently the most common one of the methods. Generally, the spatial resolution of hyperspectral images is low, and obtaining high-resolution hyperspectral images directly is costly, while obtaining high-resolution hyperspectral images through fusion is an economical and affordable method. The fusion method is a key technology in the field of image processing, because high-resolution hyperspectral images can well promote remote material identification and classification tasks, thus gaining great attention in the field of satellite remote sensing. Few people have tried to use Transformer in the field of satellite remote sensing, and Transformer has shown amazing results in many advanced vision tasks. In this paper, we propose a network model for processing HSI/MSI fusion tasks, based on the fusion network of SwinTanformer and deep convolutional network, called SwinDFN, and SwinDFN consists of two parts: 1) The traditional convolutional neural network initially fuses HSI and MSI images, which introduces Depthwise convolution technology to more effectively combine the spectral response function between HSI and MSI and compress the network parameters. 2) SwinTansformer feature extraction module with residual structure to enhance image.

Transon, Julie, et al. “Survey of Hyperspectral Earth Observation Applications from Space in the Sentinel-2 Context.” Remote Sensing, vol. 10, no. 3, Jan. 2018, p. 157.
Chaudhuri, Subhasis, and Ketan Kotwal. Hyperspectral image fusion. New York: Springer, 2013.
Zhang, Yifan, and Mingyi He. "Multi-spectral and hyperspectral image fusion using 3-D wavelet transform." Journal of electronics (China) 24 (2007): 218-224.
Gomez, Richard B., Amin Jazaeri, and Menas Kafatos. "Wavelet-based hyperspectral and multispectral image fusion." Geo-Spatial Image and Data Exploitation II. Vol. 4383. SPIE, 2001.
Tucker, Ledyard R. "Some mathematical notes on three-mode factor analysis." Psychometrika 31.3 (1966): 279-311.
Masi, Giuseppe, et al. "Pansharpening by convolutional neural networks." Remote Sensing 8.7 (2016): 594.
Zhang, Xueting, et al. "SSR-NET: Spatial–spectral reconstruction network for hyperspectral and multispectral image fusion." IEEE Transactions on Geoscience and Remote Sensing 59.7 (2020): 5953-5965.
Shao, Zhenfeng, and Jiajun Cai. "Remote sensing image fusion with deep convolutional neural network." IEEE journal of selected topics in applied earth observations and remote sensing 11.5 (2018): 1656-1669.
Yang, Jingxiang, Yong-Qiang Zhao, and Jonathan Cheung-Wai Chan. "Hyperspectral and multispectral image fusion via deep two-branches convolutional neural network." Remote Sensing 10.5 (2018): 800.
Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).
Karras, Tero, Samuli Laine, and Timo Aila. "A style-based generator architecture for generative adversarial networks." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
Isola, Phillip, et al. "Image-to-image translation with conditional adversarial networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Zhu, Jun-Yan, et al. "Unpaired image-to-image translation using cycle-consistent adversarial networks." Proceedings of the IEEE international conference on computer vision. 2017.
Ledig, Christian, et al. "Photo-realistic single image super-resolution using a generative adversarial network." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Ma, Jiayi, et al. "Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion." Information Fusion 62 (2020): 110-120.
Xiao, Jiajun, et al. "Physics-based GAN with iterative refinement unit for hyperspectral and multispectral image fusion." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14 (2021): 6827-6841.
Azarang, Arian, Hafez E. Manoochehri, and Nasser Kehtarnavaz. "Convolutional autoencoder-based multispectral image fusion." IEEE access 7 (2019): 35673-35683.
Dosovitskiy, Alexey, et al. "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020).
Carion, Nicolas, et al. "End-to-end object detection with transformers." European conference on computer vision. Cham: Springer International Publishing, 2020.
Liu, Ze, et al. "Swin transformer: Hierarchical vision transformer using shifted windows." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
Hu, Jin-Fan, et al. "Fusformer: A transformer-based fusion network for hyperspectral image super-resolution." IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.
Lin, Chia-Hsiang, et al. "A convex optimization-based coupled nonnegative matrix factorization algorithm for hyperspectral and multispectral data fusion." IEEE Transactions on Geoscience and Remote Sensing 56.3 (2017): 1652-1667.
Liang, Jingyun, et al. "Swinir: Image restoration using swin transformer." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
AVIRIS Free Standard Data Products. [Online]. Available: http: //aviris.jpl.nasa.gov/html/aviris.freedata.html.
Hsu, Chih-Chung, et al. "DCSN: Deep compressed sensing network for efficient hyperspectral data transmission of miniaturized satellite." IEEE Transactions on Geoscience and Remote Sensing 59.9 (2020): 7773-7789.
Wald, Lucien, Thierry Ranchin, and Marc Mangolini. "Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images." Photogrammetric engineering and remote sensing 63.6 (1997): 691-699.
Zhu, Zhiyu, et al. "Hyperspectral image super-resolution via deep progressive zero-centric residual learning." IEEE Transactions on Image Processing 30 (2020): 1423-1438.
Xiao, Jiajun, et al. "A dual-UNet with multistage details injection for hyperspectral image fusion." IEEE Transactions on Geoscience and Remote Sensing 60 (2021): 1-13.
Min, Zhichao, Yifan Wang, and Sen Jia. "Multiscale spatial-spectral joint feature learning for multispectral and hyperspectral image fusion." 2021 IEEE 23rd Int Conf on High Performance Computing & Communications; 7th Int Conf on Data Science & Systems; 19th Int Conf on Smart City; 7th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). IEEE, 2021.
Huang, Tao, et al. "Deep hyperspectral image fusion network with iterative spatio-spectral regularization." IEEE Transactions on Computational Imaging 8 (2022): 201-214.