Fusion of deep and local gradient-based features for multimodal finger knuckle print identification

This paper proposes a novel efficient descriptor for finger knuckle print (FKP) identification. Our proposed descriptor is called MDL (Magnitude, Direction, and Local patterns), as it describes the rich structure of FKP images by jointly encoding three types of discriminative information, namely gradient magnitude, direction, and local patterns. This joint representation allows obtaining reliable features as gradient values that are merged with local features could offer abundant information on the FKP structure and the edges contained therein. Note that MDL is strengthened by extracting the features in a block-wise manner to include the spatial relationship information. Furthermore, we consider a multimodal learning scheme in which our MDL and deep features are extracted from major and minor FKP modalities. For the sake of efficiency, we opt for the VGG-F deep network, which has a small depth, to extract the deep features. The VGG-F is well-suited to the nature of FKP images (i.e., lines and edges without complicated backgrounds), where networks with a few layers/parameters can efficiently learn the intrinsic characteristics of the FKP without suffering from the overfitting. The discriminant correlation analysis (DCA) algorithm is used to perform feature fusion. DCA has the advantage of efficiently and jointly performing features fusion and dimensionality reduction. We conduct comprehensive experiments on a public finger knuckle dataset. Experimental results demonstrate the effectiveness of the proposed method, which has outperformed several related methods. The obtained results confirm the complementarity of our MDL and deep features. Moreover, the proposed method has shown a high robustness to occlusion scenario, where some parts of the FKP image are missed.