VDFEFuse: A novel fusion approach to infrared and visible images

•A novel fusion algorithm based on visual differentiation feature extraction is proposed.•Visual Saliency Map - Weighted Local Energy is designed to merge the base layers.•A new hierarchical structure decision model is formulated to optimize the fusion result.•Our algorithm can effectively restrain energy loss and preserve texture features in the fusion process.