3-Dimensional facial expression recognition in human using multi-points warping.

Olalekan Agbolade, Azree Nazri, Razali Yaakob, Abdul Azim Ghani, Yoke Kqueen Cheah
Author Information
  1. Olalekan Agbolade: Department of Computer Science, Faculty of Computer Science & IT, Universiti Putra Malaysia, Serdang, Selangor, Malaysia. lokoprof@yahoo.com. ORCID
  2. Azree Nazri: Department of Computer Science, Faculty of Computer Science & IT, Universiti Putra Malaysia, Serdang, Selangor, Malaysia. azree@upm.edu.my.
  3. Razali Yaakob: Department of Computer Science, Faculty of Computer Science & IT, Universiti Putra Malaysia, Serdang, Selangor, Malaysia.
  4. Abdul Azim Ghani: Department of Software Engineering, Faculty of Computer Science & IT, Universiti Putra Malaysia, Serdang, Selangor, Malaysia.
  5. Yoke Kqueen Cheah: Department of Biomedical Science, Faculty of Medicine and Health Sciences, Universiti Putra Malaysia, Serdang, Selangor, Malaysia.

Abstract

BACKGROUND: Expression in H-sapiens plays a remarkable role when it comes to social communication. The identification of this expression by human beings is relatively easy and accurate. However, achieving the same result in 3D by machine remains a challenge in computer vision. This is due to the current challenges facing facial data acquisition in 3D; such as lack of homology and complex mathematical analysis for facial point digitization. This study proposes facial expression recognition in human with the application of Multi-points Warping for 3D facial landmark by building a template mesh as a reference object. This template mesh is thereby applied to each of the target mesh on Stirling/ESRC and Bosphorus datasets. The semi-landmarks are allowed to slide along tangents to the curves and surfaces until the bending energy between a template and a target form is minimal and localization error is assessed using Procrustes ANOVA. By using Principal Component Analysis (PCA) for feature selection, classification is done using Linear Discriminant Analysis (LDA).
RESULT: The localization error is validated on the two datasets with superior performance over the state-of-the-art methods and variation in the expression is visualized using Principal Components (PCs). The deformations show various expression regions in the faces. The results indicate that Sad expression has the lowest recognition accuracy on both datasets. The classifier achieved a recognition accuracy of 99.58 and 99.32% on Stirling/ESRC and Bosphorus, respectively.
CONCLUSION: The results demonstrate that the method is robust and in agreement with the state-of-the-art results.

Keywords

References

  1. Alcohol. 2010 Nov-Dec;44(7-8):649-57 [PMID: 20060678]
  2. Evolution. 2000 Aug;54(4):1273-85 [PMID: 11005294]
  3. J Anat. 2013 Oct;223(4):329-36 [PMID: 23964811]
  4. J Anat. 2006 Jun;208(6):769-84 [PMID: 16761977]
  5. Evolution. 1998 Oct;52(5):1363-1375 [PMID: 28565401]
  6. IEEE Trans Pattern Anal Mach Intell. 2011 Apr;33(4):852-8 [PMID: 21079272]
  7. Prog Orthod. 2017 Dec 1;18(1):38 [PMID: 29192356]
  8. IEEE Trans Pattern Anal Mach Intell. 2011 Apr;33(4):858-64 [PMID: 21135443]
  9. PeerJ. 2015 Nov 19;3:e1417 [PMID: 26618086]
  10. Mol Ecol Resour. 2011 Mar;11(2):353-7 [PMID: 21429143]
  11. PLoS Comput Biol. 2013;9(12):e1003375 [PMID: 24339768]
  12. J Anat. 2014 Jul;225(1):19-30 [PMID: 24836555]
  13. Med Image Anal. 1997 Apr;1(3):225-43 [PMID: 9873908]
  14. J Hum Evol. 2012 Jan;62(1):155-64 [PMID: 22178399]
  15. PLoS One. 2014 Jun 12;9(6):e99483 [PMID: 24923319]
  16. Biostatistics. 2007 Jan;8(1):86-100 [PMID: 16603682]
  17. Int J Legal Med. 2016 May;130(3):863-79 [PMID: 26662189]
  18. Dev Genes Evol. 2016 Jun;226(3):139-58 [PMID: 27038025]
  19. IEEE Trans Pattern Anal Mach Intell. 2007 Apr;29(4):640-9 [PMID: 17299221]
  20. IEEE Trans Pattern Anal Mach Intell. 2013 Sep;35(9):2270-83 [PMID: 23868784]
  21. Int J Approx Reason. 2008 Jan;47(1):17-36 [PMID: 19079753]
  22. J Pers Soc Psychol. 1971 Feb;17(2):124-9 [PMID: 5542557]
  23. Am J Phys Anthropol. 2018 Jun;166(2):373-385 [PMID: 29446438]
  24. Eur J Orthod. 2019 May 24;41(3):264-272 [PMID: 30212892]
  25. J Theor Biol. 2012 May 21;301:1-14 [PMID: 22342680]
  26. Evolution. 2002 Oct;56(10):1909-20 [PMID: 12449478]

Grants

  1. 9538100/Putra Grant Scheme, Malaysia (MY)

MeSH Term

Algorithms
Analysis of Variance
Databases as Topic
Discriminant Analysis
Facial Expression
Humans
Imaging, Three-Dimensional
Pattern Recognition, Automated
Principal Component Analysis

Word Cloud

Created with Highcharts 10.0.0expressionfacialrecognitionusing3DhumantemplatemeshdatasetsresultslandmarktargetStirling/ESRCBosphoruslocalizationerrorPrincipalAnalysisPCALDAstate-of-the-artfacesaccuracy99warpingBACKGROUND:ExpressionH-sapiensplaysremarkablerolecomessocialcommunicationidentificationbeingsrelativelyeasyaccurateHoweverachievingresultmachineremainschallengecomputervisionduecurrentchallengesfacingdataacquisitionlackhomologycomplexmathematicalanalysispointdigitizationstudyproposesapplicationMulti-pointsWarpingbuildingreferenceobjecttherebyappliedsemi-landmarksallowedslidealongtangentscurvessurfacesbendingenergyformminimalassessedProcrustesANOVAComponentfeatureselectionclassificationdoneLinearDiscriminantRESULT:validatedtwosuperiorperformancemethodsvariationvisualizedComponentsPCsdeformationsshowvariousregionsindicateSadlowestclassifierachieved5832%respectivelyCONCLUSION:demonstratemethodrobustagreement3-Dimensionalmulti-pointsAutomaticFacialMulti-point

Similar Articles

Cited By