{Home} {Publications and Codes} {Research} {Projects} {Professional Activities} {People}

Machine Vision, Image Recognition and Machine Learning

[AdvNet for Deep Kinship Recognition]

Paper:

Qingyan Duan, Lei Zhang*, and Wangmeng Zuo, AdvNet: Adversarial Contrastive Residual Net for 1 Million Kinship Recognition, ACM Int'Conf. Multimedia (ACM MM), in RFIW Workshop Challenge, Ranking Top 3rd, 2017. [paper], [Github source code]

Abstract:

Kinship verification in the wild is an interesting and challenging problem. The goal of kinship verification is to determine whether a pair of faces are blood relatives or not. Most previous methods for kinship verification can be divided as hand-crafted features based shallow learning methods and convolutional neural network (CNN) based deep learning methods. Nevertheless, these methods are still posed with the challenging task of recognizing kinship cues from facial images. Part of the reason for this may be that, the family information and the distribution difference of pairwise kin-face data based kinship verification issue are rarely considered. Inspired by maximum mean discrepancy (MMD) and generative adversarial net (GAN), family ID based Adversarial contrastive residual Network (AdvNet) is proposed for largescale (1 Million) kinship recognition in this paper. The MMD based adversarial loss (AL), pairwise contrastive loss (CL) and family ID based softmax loss (SL) are jointly formulated in the proposed AdvNet for kin-relation enhancement and discovery. Further, the deep nets ensemble is used for deep kin-feature augmentation. Finally, Euclidean distance metric is used for kinship recognition. Extensive experiments on the 1st Large-Scale Kinship Recognition Data Challenge (Families in the wild) show the effectiveness of our proposed AdvNet and ensemble based feature augmentation.

Approach:

Overview of the proposed multiple deep nets based ensemble method for kinship verification. Three AdvNets with different loss and architecture, and one VGG-Face model are fused for feature augmentation. Euclidean distance is used for face verification.

The Family ID based AdvNet with multiple losses residual architecture

Experiments:

Recognition accuracy of different model, loss and feature augmentation, and the ROC curves of different models on 7 types of kin-relation.

L2-distances of kinship pairs on 7 types of kin-relation. The points in red and blue denote the kinship pairs and non-kinship pairs, respectively. The black line denote the searched threshold for verification.

References:

[1] J. Robinson, M. Shao, Y. Wu, and Y. Fu. 2016. Families in the Wild (FIW): Large-Scale Kinship Image Database and Benchmarks. In Proceedings of the 2016 ACM on Multimedia Conference. 242–246.

[2] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672–2680.

[3] K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.

[4] M. Long, Y. Cao, J. Wang, and M. Jordan. 2015. Learning transferable features with deep adaptation networks. In International Conference on Machine Learning. 97–105.