Human Activity Recognition

This project proposes an SDP3 framework for the study of sensing-communication tradeoff with the particular application of human motion recognition. Specifically：
1. The SDP3 data simulator with a data-driven hybrid channel model is proposed to generate the received sensing signals in a virtual environment.
2. The SDP3 performance predictor is then introduced to approximate the motion recognition accuracy via analytical expression with the simulated dataset of sensing signals.
3. The recognition accuracy and communication throughput tradeoff is characterized by the SDP3 performance optimizer.

Fig.1: The framework of the proposed DAHC model

It is demonstrated that the dataset generated by the SDP3 data simulator matches the experiment dataset in KL divergence, grayscale PMF and motion recognition accuracy. Hence, the sensing-communication tradeoff can be investigated without extensive experiments. It is also shown that the sensing and communication performance is balanced in the sensingcommunication adversarial zone of the A-T region, where both performance varies sensitively with respect to each other.

Fig.2: Generated human motion datasets

Fig.3: The calibration results of DAHC model for walking in scenario 1 and 2.

Fig.4: Comparison of recognition accuracy among the datasets generated by real experiments

This project proposes a computer-vision-assisted simulation method to address the issue of training dataset acquisition for wireless hand gesture recognition. Specifically:

1. This project addresses the critical issue of massive real-world data requirements for data-driven activity recognition. By leveraging a high-fidelity wireless channel simulator, it enables cost-effective generation of diverse human activity data for training.
2. An unsupervised sim-to-real transfer learning approach bridges the domain gap, enabling models trained on synthesized channel data to achieve significantly improved recognition accuracy when applied to actual measured data.

Fig.1: An Overview of the Simulation-Based Human Activity Recognition Methodology

Fig.2: wireless channel simulator

Experimental results indicate that by first pre-training the model extensively on labeled synthetic data and then fine-tuning it with limited unlabeled real measurements, the simulation-to-real inference accuracy for human activity and gesture recognition can be boosted from 73% and 83% to 93.75% and 96%, respectively.

Fig.3: Illustration of the simulated and experimental gesture dataset

Fig.4: Illustration of the simulated and experimental activity dataset

Fig.5: Gesture Recognition Results: Before Fine-Tuning (Left) vs. After (Right) Fine-Tuning

Fig.6: Activity Recognition Results: Before Fine-Tuning (Left) vs. After (Right) Fine-Tuning

This project proposes an integrated passive sensing and communication system working in 60 GHz band, and the sensing performance is investigated in an application of hand gesture recognition. Specifically:
1. A hardware setup featuring a single transmitter and a receiver with two RF chains enables analog beamforming. The transmitter emits a communication stream via dual beams: one for the main link and another directed at a gesture target. The receiver captures both signals separately for subsequent joint processing.
2. Through cross-ambiguity coherent processing of the two received signals, time-Doppler spectrograms of hand gestures are generated. A dataset containing three gesture types is built using both LoS and NLoS reference channels. This dataset is used to train a neural network model for motion detection and classification.

Fig.1: Block diagram of system implementation

Fig.2: Experiment Layout.

It is shown by experiments that passive sensing in 60 GHz has a good resolution on the micro-Doppler effect of hand gestures as the classification accuracy is greater than 90%. It is also robust to link blockage as good classification accuracy can be achieved even the NLoS path is used as the reference channel.

Demo