CSI-Former: Pay More Attention to Pose Estimation with WiFi.

Yue Zhou, Caojie Xu, Lu Zhao, Aichun Zhu, Fangqiang Hu, Yifeng Li
Author Information
  1. Yue Zhou: School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China.
  2. Caojie Xu: School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China.
  3. Lu Zhao: School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China.
  4. Aichun Zhu: School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China.
  5. Fangqiang Hu: School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China.
  6. Yifeng Li: School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China. ORCID

Abstract

Cross-modal human pose estimation has a wide range of applications. Traditional image-based pose estimation will not work well in poor light or darkness. Therefore, some sensors such as LiDAR or Radio Frequency (RF) signals are now using to estimate human pose. However, it limits the application that these methods require much high-priced professional equipment. To address these challenges, we propose a new WiFi-based pose estimation method. Based on the Channel State Information (CSI) of WiFi, a novel architecture CSI-former is proposed to innovatively realize the integration of the multi-head attention in the WiFi-based pose estimation network. To evaluate the performance of CSI-former, we establish a span-new dataset Wi-Pose. This dataset consists of 5 GHz WiFi CSI, the corresponding images, and skeleton point annotations. The experimental results on Wi-Pose demonstrate that CSI-former can significantly improve the performance in wireless pose estimation and achieve more remarkable performance over traditional image-based pose estimation. To better benefit future research on the WiFi-based pose estimation, Wi-Pose has been made publicly available.

Keywords

References

  1. IEEE Trans Inf Technol Biomed. 2010 Sep;14(5):1166-72 [PMID: 20529753]
  2. Entropy (Basel). 2022 Jul 14;24(7): [PMID: 35885197]
  3. Entropy (Basel). 2022 Mar 23;24(4): [PMID: 35455104]
  4. Entropy (Basel). 2022 Apr 25;24(5): [PMID: 35626484]
  5. Sensors (Basel). 2019 May 21;19(10): [PMID: 31117175]
  6. Entropy (Basel). 2021 May 10;23(5): [PMID: 34068635]
  7. Sensors (Basel). 2020 Jul 20;20(14): [PMID: 32698482]
  8. Phys Med Biol. 2006 Oct 7;51(19):4951-65 [PMID: 16985280]
  9. Entropy (Basel). 2021 Sep 04;23(9): [PMID: 34573789]
  10. IEEE Trans Pattern Anal Mach Intell. 2013 Jan;35(1):221-31 [PMID: 22392705]

Grants

  1. 62101245/National Natural Science Foundation of China
  2. 2019M661999/China Postdoctoral Science Foundation
  3. 19KJB520009/Natural Science Research of Jiangsu Higher Education Institutions of China
  4. FNSRFP-2021-YB-21/Future Network Scientific Research Fund Project

Word Cloud

Created with Highcharts 10.0.0poseestimationWiFiWiFi-basedCSICSI-formerperformanceWi-Posehumanimage-basedmulti-headattentiondatasetCross-modalwiderangeapplicationsTraditionalwillworkwellpoorlightdarknessThereforesensorsLiDARRadioFrequencyRFsignalsnowusingestimateHoweverlimitsapplicationmethodsrequiremuchhigh-pricedprofessionalequipmentaddresschallengesproposenewmethodBasedChannelStateInformationnovelarchitectureproposedinnovativelyrealizeintegrationnetworkevaluateestablishspan-newconsists5GHzcorrespondingimagesskeletonpointannotationsexperimentalresultsdemonstratecansignificantlyimprovewirelessachieveremarkabletraditionalbetterbenefitfutureresearchmadepubliclyavailableCSI-Former:PayAttentionPoseEstimation

Similar Articles

Cited By (3)