Deep Multi-Person Kinship Matching and Recognition for Family Photos

•First, we design a deep kinship matching and recognition (DKMR) framework for understanding kinship in a nuclear family automatically. It makes the first attempt to generate a nuclear family tree end-to-end, to our best knowledge.•Second, compared with previous kinship understanding methods focusing on pairwise face images, 70 the input of our DKMR framework extends to one nuclear family photo. The experiments on Group-Face, TSkinFace and FIW datasets demonstrate its effectiveness.•Third, our proposed reasoning conditional random field (R-CRF) algorithm fully exploits the common kinship rules to well boost matching and recognition accuracy and ensure the output family tree is optimal.