Xiaobo Qu^a, Jingwen Yan^b, Hongzhi Xiao^b, Ziqian Zhu^c

a Department of Electronic Science, Xiamen University, Xiamen 361005, P.R.China
b Key Laboratory of Digital Signal and Image Processing of Guangdong Province, Shantou University, Shantou 515063, P. R. China
c Research Institute of Chinese Radar Electronic Equipment, Wuxi 214063, P.R.China
Xiaobo Qu's Email: quxiaobo <at> xmu.edu.cn or quxiaobo2009 <at> gmail.com.

Project-Image Fusion:

To download the sourceimages, please click on the name of images or hyperlink of website. Just try it...
Multifocus images: clockA.png and clockB.png ;disk1.png and disk2.png;mulfocus1.png and mulfocus2.png
Visible and infrared images: forestA.jpg and forestB.jpg
To download the matlab code for objective evaluation criteria of image fusion, please click on the name of criteria. Just try it...
Mutual information & Qab/f

Abstract:

Nonsubsampled contourlet transform (NSCT) provides flexible multiresolution, anisotropy and directional expansion for images. Compared with the original contourlet transform, it is shift-invariant and can overcome the pseudo-Gibbs phenomena around singularities. Pulse Coupled Neural Networks (PCNN) is a visual cortex-inspired neural network and characterized by the global coupling and pulse synchronization of neurons. It has been proven suitable for image processing and successfully employed in image fusion. In this paper, NSCT is associated with PCNN and employed in image fusion to make full use of the characteristics of them. Spatial frequency in NSCT domain is input to motivate PCNN and coefficients in NSCT domain with large firing times are selected as coefficients of the fused image. Experimental results demonstrate that the proposed algorithm outperforms typical wavelet-based, contourlet-based, PCNN-based and contourlet-PCNN-based fusion algorithms in term of objective criteria and visual appearance.(download this paper )
Key words: Contourlet, pulse coupled neural networks (PCNN), wavelet, image fusion, multiscale transform

Fig.1 Schematic diagram of NSCT-SF-PCNN fusion algorithm

In this paper, a spatial frequency motivated PCNN in NSCT domain, NSCT-SF-PCNN, is proposed. The flexible multiresolution, anisotropy, and directional expansion for images of NSCT are associated with global coupling and pulse synchronization characteristic of PCNN. Furthermore, a spatial frequency motivated PCNN, rather than pure use of coefficients value in traditional PCNN in image processing, is presented. Experiments on MSD methods, activity-level measurements, and typical PCNN-based algorithms demonstrate that the proposed NSCT-SF-PCNN is successful in multifocus image fusion and visible and infrared image fusion.

Acknowledgements:

The authors would like to thank Mr. David Dwyer of Octec Ltd, Dr. Lex Toet of TNO Human Factors and Dr. Oliver Rockinger for providing the images used in this work and also thank Hu Chang-Wei for his help in the preparation of the manuscript. You can contact qxb xmu@yahoo.com.cn for the images.

References:

1. Hall D L, Llinas J. An introduction to multisensor data fusion. Proceedings of the IEEE, 1997, 85(1): 6−23
2. Zhang Z, Blum R S. A categorization of multiscaledecomposition-based image fusion schemes with a performance study for a digital camera application. Proceedings of the IEEE, 1999, 87(8): 1315−1326
3. Le P E, Mallat S. Sparse geometric image representation with bandelets. IEEE Transactions on Image Processing, 2005, 14(4): 423−438
4. Do M N, Vetterli M. The contourlet transform: an effi- cient directional multiresolution image representation. IEEE Transactions on Image Processing, 2005, 14(12): 2091−2106
5. Qu X B, Yan J W, Xie G F, Zhu Z Q, Chen B G. A novel image fusion algorithm based on bandelet transform. Chinese Optics Letters, 2007, 5(10): 569−572
6. Choi M, Kim R Y, Nam M R, Kim H O. Fusion of multispectral and panchromatic satellite images using the curvelet 1514 ACTA AUTOMATICA SINICA Vol. 34 transform. IEEE Geoscience and Remote Sensing Letters, 2005, 2(2): 136−140
7. Qu X B, Xie G F, Yan J W, Zhu Z Q, Chen B G. Image fusion algorithm based on neighbors and cousins information in nonsubsampled contourlet transform domain. In: Proceedings of International Conference on Wavelet Analysis and Pattern Recognition. Beijing, China: IEEE, 2007. 1797−1802
8. Zheng Yong-An, Song Jian-She, Zhou Wen-Ming, Wang RuiHua. False color fusion for multi-band SAR images based on contourlet transform. Acta Automatica Sinica, 2007, 33(4): 337−341
9. Fang Yong, Liu Sheng-Peng. Infared Image Fusion Algorithm Based on Contourlet Transform and Improved Pulse Coupled Neural Networks, China Patent 1873693A, December 2006 (in Chinese)
10. Da Cunha A L, Zhou J P, Do M N. The nonsubsampled contourlet transform: theory, design, and applications. IEEE Transactions on Image Processing, 2006, 15(10): 3089−3101
11. Eckhorn R, Reitboeck H J, Arndt M, Dicke P. Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex. Neural Computation, 1990, 2(3): 293−307
12. Johnson J L, Padgett M L. PCNN models and applications. IEEE Transactions on Neural Networks, 1999, 10(3): 480−498
13. Broussard R P, Rogers S K, Oxley M E, Tarr G L. Physiologically motivated image fusion for object detection using a pulse coupled neural network. IEEE Transactions on Neural Networks, 1999, 10(3): 554−563
14. Li M, Cai W, Tan Z. Pulse coupled neural network based image fusion. In: Proceedings of the 2nd International Symposium on Neural Networks. Chongqing, China: Springer, 2005. 741−746
15. Li W, Zhu X F. A new algorithm of multi-modality medical image fusion based on pulse-coupled neural networks. In: Proceedings of International Conference on Advances in Natural Computation. Changsha, China: Springer, 2005. 995−1001
16. Xu B C, Chen Z. A multisensor image fusion algorithm based on PCNN. In: Proceeding of the 5th World Congress on Intelligent Control and Automation. Hangzhou, China: IEEE, 2004. 3679−3682
17. Qu Xiao-Bo, Yan Jing-Wen, Zhu Zi-Qian, Chen Ben-Gang. Multi-focus image fusion algorithm based on regional firing characteristic of pulse coupled neural networks. In: Proceedings of International Conference on Bio-Inspired Computing: Theories and Applications. Zhengzhou, China: Publishing House of Electronics Industry, 2007. 563−565
18. Eskicioglu A M, Fisher P S. Image quality measures and their performance. IEEE Transactions on Communications, 1995, 43(12): 2959−2965
19. Qu G H, Zhang D L, Yan P F. Information measure for performance of image fusion. Electronics Letters, 2002, 38(7): 313−315
20. Petrovic V, Xydeas C. On the effects of sensor noise in pixellevel image fusion performance. In: Proceedings of the 3rd International Conference on Image Fusion. Paris, France: IEEE, 2000. 14−19