HaGRID Dynamic Gestures

HaGRID main repo

Overview

This repository contains using HaGRID dataset for dynamic gesture recognition. The dataset is available here.

Project

├── ocsort/ # source code for Observation-Centric Sorting │ ├── kalmanfilter.py # Kalman filter │ ├── kalmanboxtracker.py # Kalman box tracker │ ├── association.py # Association of boxes with trackers ├── utils/ # useful utils │ ├── action_controller.py # Action controller for dynamic gestures │ ├── box_utils_numpy.py # Box utils for numpy │ ├── enums.py # Enums for dynamic gestures and actions │ ├── hand.py # Hand class for dynamic gestures recognition │ ├── drawer.py # Debug drawer ├── onnx_models.py # ONNX models for gesture recognition ├── main_controller.py # Main controller for dynamic gestures recognition, uses ONNX models, ocsort and utils ├── run_demo.py # Demo script for dynamic gestures recognition

Installation

Clone and install required python packages: ```bash git clone https://github.com/ai-forever/dynamic_gestures.git

or mirror link:

cd dynamic_gestures

Create virtual env by conda or venv

conda create -n dynamicgestures python=3.9 -y conda activate dynamicgestures

Install requirements

pip install -r requirements.txt ```

Demo

To run demo, you just need to run run_demo.py script.

bash python run_demo.py --detector <path_to_detector> --classifier <path_to_classifier> --debug --detector (optional) Path to the hand detector model. Default: models/hand_detector.onnx

--classifier (optional) Path to the crops classifier model. Default: models/crops_classifier.onnx

--debug (optional) Enables debug mode to see bounding boxes and class labels.

Dynamic gestures

Next, we will show dynamic gestures in user mode and debug mode. In user mode, we show only the final result of dynamic gesture recognition. In debug mode, we show the result of each step of dynamic gesture recognition: 1. hand detection 2. hand tracking 3. gesture recognition 4. action recognition

At the moment the code supports 6 groups of dynamic gestures:

ZOOM DRAG AND DROP

FAST SWIPE UP / DOWN CLICK SWIPES LEFT / RIGHT

SWIPES UP / DOWN

License

This work is licensed under a variant of Apache License, Version 2.0.

Please see the specific license.

Citation

You can cite the paper using the following BibTeX entry:

@misc{nuzhdin2024hagridv21mimagesstatic,
      title={HaGRIDv2: 1M Images for Static and Dynamic Hand Gesture Recognition},
      author={Anton Nuzhdin and Alexander Nagaev and Alexander Sautin and Alexander Kapitanov and Karina Kvanchiani},
      year={2024},
      eprint={2412.01508},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2412.01508},
}