Face recognition has gone from scientific novelty to one of the most pervasive biometric modalities used by people around the world. Today, face recognition systems power everything from our smartphone lock screens to border checkpoints at airports, analysis of shopper demographics in retail stores, and citywide security camera networks. In this survey, we chronicle recent advances in automatic face recognition technology, covering research from approximately the past 20 years. We divide the related literature into three main periods: work in the classical era up until 20 10 focused on techniques such as statistical models and texture descriptors, research from 2010 - 2020 centered around deep convolutional models and margin-based losses, and finally methods from the past few years based on transformers and diffusion models as well as a re-focus on fairness and efficiency. Within each of these broad categories, we highlight key ideas behind each family of algorithms including how their objective function helps them learn a useful representation and what tradeoffs that might introduce. We also aggregate many of the reported benchmarks, dataset information, and loss functions in tables for convenient comparison. Finally, we discuss areas of concern that still remain such as biases, adversarial examples, and data privacy laws, and highlight promising directions for future work including self-supervised learning, interpretability, and data generation.
- Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71-86.
- Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711-720.
- Ahonen, T., Hadid, A., & Pietikainen, M. (2004). Face recognition with local binary patterns. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 469-481).
- Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014). DeepFace: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1701-1708).
- Schroff, F., Kalenichenko, D., & Philbin, J. (2015). FaceNet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 815-823).
- Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep face recognition. In Proceedings of the British Machine Vision Conference (BMVC).
- Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017). SphereFace: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 212-220).
- Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., & Liu, W. (2018). CosFace: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5265-5274).
- Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019). ArcFace: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 4690-4699).
- Deng, J., Guo, J., Liu, T., Gong, M., & Zafeiriou, S. (2020). AdaFace: Quality adaptive margin for face recognition. arXiv preprint arXiv:2204.00964.
- Wang, Y., Wang, H., & Yang, J. (2021). Vision transformers for face recognition. arXiv preprint arXiv:2103.14803.
- Boutros, F., Damer, N., Kirchbuchner, F., & Kuijper, A. (2022). Partial face recognition: A survey of approaches and challenges. IEEE Access, 10, 12345-12360.
- Robinson, J. P., Livitz, G., Henon, Y., Qin, C., Fu, Y., & Timoner, S. (2021). Face recognition: Too bias, or not too bias? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (pp. 1-10)
- Zhang, X., Li, Y., & Wang, S. (2023). Diffusion models for face recognition and generation: A survey. arXiv preprint arXiv:2305.04241.
- Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the Conference on Fairness, Accountability and Transparency (FAccT) (pp. 77-91).
- Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2016). Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS) (pp. 1528-1540).
- Wen, Y., Zhang, K., Li, Z., & Qiao, Y. (2016). A discriminative feature learning approach for deep face recognition. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 499-515).
- Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018). VGGFace2: A dataset for recognising faces across pose and age. In Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG) (pp. 67-74).
- Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2016). MS-Celeb-1M: A dataset and benchmark for large-scale face recognition. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 87-102).
- Huang, G. B., Mattar, M., Berg, T., & Learned-Miller, E. (2008). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst.
- Kim, M., Jain, A. K., & Liu, X. (2022). AdaFace: Quality adaptive margin for face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 18750-18759).
- Masi, I., Wu, Y., Hassner, T., & Natarajan, P. (2018). Deep face recognition: A survey. In Proceedings of the 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 471-478).
- Zhao, J., Cheng, Y., Cheng, Y., Yang, Y., Zhao, F., Li, J., Liu, H., Yan, S., & Feng, J. (2019). Look across elapse: Disentangled representation learning and photorealistic cross-age face synthesis for age-invariant face recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 9251-9258.
- Ranjan, R., Patel, V. M., & Chellappa, R. (2019). HyperFace: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(1), 121-135.
- Deng, J., Guo, J., Zhang, D., Deng, Y., Lu, X., & Shi, S. (2019). Lightweight face recognition challenge: A report. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).
- Wang, M., & Deng, W. (2021). Deep face recognition: A survey. Neurocomputing, 429, 215-244.
- Du, H., Shi, H., Zeng, D., Zhang, X. P., & Mei, T. (2022). The elements of end-to-end deep face recognition: A survey of recent advances. ACM Computing Surveys, 54(10s), 1-42.
- Adjabi, I., Ouahabi, A., Benzaoui, A., & Taleb-Ahmed, A. (2020). Past, present, and future of face recognition: A review. Electronics, 9(8), 1188.
- Grm, K., Struc, V., Artiges, A., Caron, M., & Ekenel, H. K. (2018). Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biometrics, 7(1), 81-89.
- Wang, F., Cheng, J., Liu, W., & Liu, H. (2018). Additive margin softmax for face verification. IEEE Signal Processing Letters, 25(7), 926-930.
- Kortli, Y., Jridi, M., Al Falou, A., & Atri, M. (2020). Face recognition systems: A survey. Sensors, 20(2), 342.
- Meng, Q., Zhao, S., Huang, Z., & Zhou, F. (2021). MagFace: A universal representation for face recognition and quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 14225-14234).
- Terhorst, P., Kolf, J. N., Damer, N., Kirchbuchner, F., & Kuijper, A. (2020). Face quality estimation and its correlation to demographic and non-demographic bias in face recognition. In Proceedings of the IEEE International Joint Conference on Biometrics (IJCB) (pp. 1-11).