Facial identity recognition using StyleGAN3 inversion and improved tiny YOLOv7 model.

Akhil Kumar, Swarnava Bhattacharjee, Ambrish Kumar, Dushantha Nalin K Jayakody
Author Information
  1. Akhil Kumar: School of Computer Science Engineering and Technology, Bennett University, Greater Noida, India.
  2. Swarnava Bhattacharjee: Liverpool John Moores University, Liverpool, England.
  3. Ambrish Kumar: School of Computer Science Engineering and Technology, Bennett University, Greater Noida, India.
  4. Dushantha Nalin K Jayakody: COPELABS, Lusófona University, Lisboa, Portugal. dushantha.jayakody@ulusofona.pt.

Abstract

Facial identity recognition is one of the challenging problems in the domain of computer vision. Facial identity comprises the facial attributes of a person's face ranging from age progression, gender, hairstyle, etc. Manipulating facial attributes such as changing the gender, hairstyle, expressions, and makeup changes the entire facial identity of a person which is often used by law offenders to commit crimes. Leveraging the deep learning-based approaches, this work proposes a one-step solution for facial attribute manipulation and detection leading to facial identity recognition in few-shot and traditional scenarios. As a first step towards performing facial identity recognition, we created the Facial Attribute Manipulation Detection (FAM) Dataset which consists of twenty unique identities with thirty-eight facial attributes generated by the StyleGAN3 inversion. The Facial Attribute Detection (FAM) Dataset has 11,560 images richly annotated in YOLO format. To perform facial attribute and identity detection, we developed the Spatial Transformer Block (STB) and Squeeze-Excite Spatial Pyramid Pooling (SE-SPP)-based Tiny YOLOv7 model and proposed as FIR-Tiny YOLOv7 (Facial Identity Recognition-Tiny YOLOv7) model. The proposed model is an improvised variant of the Tiny YOLOv7 model. For facial identity recognition, the proposed model achieved 10.0% higher mAP in the one-shot scenario, 30.4% higher mAP in the three-shot scenario, 15.3% higher mAP in the five-shot scenario, and 0.1% higher mAP in the traditional 70% - 30% split scenario as compared to the Tiny YOLOv7 model. The results obtained with the proposed model are promising for general facial identity recognition under varying facial attribute manipulation.

Keywords

References

  1. Multimed Tools Appl. 2023;82(6):9243-9275 [PMID: 35968414]
  2. IEEE Trans Image Process. 2019 Nov;28(11):5464-5478 [PMID: 31107649]
  3. IEEE Trans Pattern Anal Mach Intell. 2021 Dec;43(12):4217-4228 [PMID: 32012000]
  4. Cognition. 2024 Feb;243:105668 [PMID: 38043180]
  5. IEEE Trans Pattern Anal Mach Intell. 2023 Dec;45(12):14777-14788 [PMID: 37616132]
  6. IEEE Trans Image Process. 2023;32:5893-5908 [PMID: 37889810]
  7. Arch Comput Methods Eng. 2021;28(7):4503-4521 [PMID: 33824572]
  8. IEEE Trans Pattern Anal Mach Intell. 2015 Sep;37(9):1904-16 [PMID: 26353135]
  9. IEEE Trans Image Process. 2004 Apr;13(4):600-12 [PMID: 15376593]

Grants

  1. SPARC/2024-2025/NXTG/P3524/Scheme for Promotion of Academic and Research Collaboration

MeSH Term

Humans
Male
Female
Facial Recognition
Deep Learning
Automated Facial Recognition
Face
Adult
Young Adult
Algorithms
Image Processing, Computer-Assisted

Word Cloud

Created with Highcharts 10.0.0facialidentityFacialmodelrecognitionYOLOv7attributeTinyproposedhighermAPscenarioattributesmanipulationdetectionStyleGAN3genderhairstyletraditionalAttributeDetectionFAMDatasetinversionSpatialonechallengingproblemsdomaincomputervisioncomprisesperson'sfacerangingageprogressionetcManipulatingchangingexpressionsmakeupchangesentirepersonoftenusedlawoffenderscommitcrimesLeveragingdeeplearning-basedapproachesworkproposesone-stepsolutionleadingfew-shotscenariosfirststeptowardsperformingcreatedManipulationconsiststwentyuniqueidentitiesthirty-eightgenerated11560imagesrichlyannotatedYOLOformatperformdevelopedTransformerBlockSTBSqueeze-ExcitePyramidPoolingSE-SPP-basedFIR-TinyIdentityRecognition-Tinyimprovisedvariantachieved100%one-shot304%three-shot153%five-shot01%70%- 30%splitcomparedresultsobtainedpromisinggeneralvaryingusingimprovedtinyFace

Similar Articles

Cited By

No available data.