versatile-xr-user-identification
Implementation of "Versatile User Identification in Extended Reality Using Pretrained Similarity-Learning". Deep metric learning approach for identifying VR users from their movements.
Science Score: 67.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 1 DOI reference(s) in README -
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.4%) to scientific vocabulary
Repository
Implementation of "Versatile User Identification in Extended Reality Using Pretrained Similarity-Learning". Deep metric learning approach for identifying VR users from their movements.
Basic Info
- Host: GitHub
- Owner: cschell
- License: other
- Language: Python
- Default Branch: main
- Size: 149 KB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
Readme.md
Versatile User Identification in Extended Reality Using Pretrained Similarity-Learning
This repository contains the code for our paper "Versatile User Identification in Extended Reality Using Pretrained Similarity-Learning".
About
This work combines distance-based and classification-based approaches to identify VR users from their movements using deep metric learning. The models are trained on data from players of "Half-Life: Alyx" and demonstrate:
- Ability to identify new users from non-specific movements with minimal enrollment data
- Fast new user enrollment (seconds vs days for retraining traditional classifiers)
- More reliable performance with limited enrollment data
- Cross-dataset generalization to different VR devices
Repository Structure
The codebase is organized into data_preparation and machine_learning. You find in each folder the corresponding Readmes.
Citation
If you use this code in your research, please cite:
bibtex
@online{RackVersatileUserIdentification2023,
title = {Versatile {{User Identification}} in {{Extended Reality}} Using {{Pretrained Similarity-Learning}}},
author = {Rack, Christian and Kobs, Konstantin and Fernando, Tamara and Hotho, Andreas and Latoschik, Marc Erich},
date = {2023-07-03},
eprint = {2302.07517},
eprinttype = {arXiv},
doi = {10.48550/arXiv.2302.07517}
}
License
This work by Christian Rack, Konstantin Kob, Tamara Fernando, Andreas Hotho and Marc E. Latoschik is licensed under CC BY-NC-SA 4.0.
Owner
- Name: Christian Schell
- Login: cschell
- Kind: user
- Location: Würzburg, Bavaria
- Company: Chair for Human Computer Interaction, University of Würzburg, Bavaria
- Repositories: 23
- Profile: https://github.com/cschell
I'm a Phd student from Würzburg, Bavaria, focussing on applying deep learning techniques on biometric data.
Citation (Citation.cff)
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Rack"
given-names: "Christian"
- family-names: "Kobs"
given-names: "Konstantin"
- family-names: "Fernando"
given-names: "Tamara"
- family-names: "Hotho"
given-names: "Andreas"
- family-names: "Latoschik"
given-names: "Marc Erich"
title: "VR User Identification via Pretrained Similarity Learning"
version: 1.0.0
doi: 10.48550/arXiv.2302.07517
date-released: 2023-07-03
url: "https://github.com/cschell/Versatile-XR-User-Identification"
preferred-citation:
type: article
authors:
- family-names: "Rack"
given-names: "Christian"
- family-names: "Kobs"
given-names: "Konstantin"
- family-names: "Fernando"
given-names: "Tamara"
- family-names: "Hotho"
given-names: "Andreas"
- family-names: "Latoschik"
given-names: "Marc Erich"
doi: "10.48550/arXiv.2302.07517"
journal: "arXiv"
title: "Versatile User Identification in Extended Reality Using Pretrained Similarity-Learning"
year: 2023
month: 7
eprint: "2302.07517"
eprinttype: "arXiv"
url: "http://arxiv.org/abs/2302.07517"
abstract: "In this paper, we combine the strengths of distance-based and classification-based approaches for the task of identifying extended reality users by their movements. For this we explore an embedding-based model that leverages deep metric learning. We train the model on a dataset of users playing the VR game \"Half-Life: Alyx\" and conduct multiple experiments and analyses using a state of the art classification-based model as baseline. The results show that the embedding-based method 1) is able to identify new users from non-specific movements using only a few minutes of enrollment data, 2) can enroll new users within seconds, while retraining the baseline approach takes almost a day, 3) is more reliable than the baseline approach when only little enrollment data is available, 4) can be used to identify new users from another dataset recorded with different VR devices. Altogether, our solution is a foundation for easily extensible XR user identification systems, applicable to a wide range of user motions. It also paves the way for production-ready models that could be used by XR practitioners without the requirements of expertise, hardware, or data for training deep learning models."
GitHub Events
Total
- Push event: 1
- Create event: 2
Last Year
- Push event: 1
- Create event: 2
Dependencies
- nvidia/cuda 11.4.1-cudnn8-runtime-ubuntu20.04 build
- faiss-gpu *
- hydra-colorlog *
- hydra-core >=1.2.0
- matplotlib *
- numpy *
- pandas *
- pudb *
- pyarrow *
- python-dotenv *
- pytorch-lightning *
- pytorch-metric-learning *
- rich *
- scikit-learn *
- scipy *
- seaborn *
- sh *
- tables *
- torch *
- torchmetrics *
- torchvision *
- tqdm *
- wandb *