Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/ami-iit/paper_cardenas_2024_ral_xbg

https://github.com/ami-iit/paper_cardenas_2024_ral_xbg

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.0%) to scientific vocabulary
Last synced: 11 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: ami-iit
  • License: bsd-3-clause
  • Language: Python
  • Default Branch: main
  • Size: 68.4 KB
Statistics
  • Stars: 5
  • Watchers: 3
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created about 2 years ago · Last pushed about 2 years ago
Metadata Files
Readme License

README.md

XBG: End-to-end Imitation Learning for Autonomous Behaviour in Human-Robot Interaction and Collaboration

C. Cardenas-Perez, G. Romualdi, M. Elobaid, S. Dafarra, G. L'Erario, S. Traversaro, P. Morerio, A. Del Bue and D. Pucci.

IEEE Robotics and Automation Letters
Installation | Paper | Experiments Video | Website

Abstract

This paper presents XBG (eXteroceptive Behaviour Generation), a multimodal end-to-end Imitation Learning (IL) system for a whole-body autonomous humanoid robot used in real-world Human-Robot Interaction (HRI) scenarios. The main contribution of this paper is an architecture for learning HRI behaviors using a data-driven approach. Through teleoperation, a diverse dataset is collected, comprising demonstrations across multiple HRI scenarios, including handshaking, handwaving, payload reception, walking, and walking with a payload. After synchronizing, filtering, and transforming the data, different Deep Neural Networks (DNN) models are trained. The final system integrates different modalities comprising exteroceptive and proprioceptive sources of information to provide the robot with an understanding of its environment and its own actions. The robot takes sequence of images (RGB and depth) and joints state information during the interactions and then reacts accordingly, demonstrating learned behaviors. By fusing multimodal signals in time, we encode new autonomous capabilities into the robotic platform, allowing the understanding of context changes over time. The models are deployed on ergoCub, a real-world humanoid robot, and their performance is measured by calculating the success rate of the robot's behavior under the mentioned scenarios.

Installation

In order to reproduce the results related to this work, please configure your environment using either the requirements.txt file or the Pipfiles provided.

Citing this work

If you find the work useful, please consider citing:

bibtex @ARTICLE{, author={Cardenas-Perez, Carlos and Romualdi, Giulio and Elobaid, Mohammed and Dafarra, Stefano and L'erario, Giuseppe and Traversaro, Silvio and Morerio, Pietro and Del Bue, Alessio and Pucci, Daniele}, journal={Arxiv}, title={}, year={2024}, volume={}, number={}, pages={}, doi={}}

Maintainer

This repository is maintained by:

| | | |:---:|:------------------------------------------------------:| | | @carloscp3009 |

Owner

  • Name: Artificial and Mechanical Intelligence
  • Login: ami-iit
  • Kind: organization
  • Location: Italy

GitHub Events

Total
  • Watch event: 2
Last Year
  • Watch event: 2

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 5
  • Total Committers: 2
  • Avg Commits per committer: 2.5
  • Development Distribution Score (DDS): 0.4
Past Year
  • Commits: 2
  • Committers: 1
  • Avg Commits per committer: 2.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Carlos c****s@i****t 3
Daniele Pucci d****5@g****m 2
Committer Domains (Top 20 + Academic)
iit.it: 1

Issues and Pull Requests

Last synced: about 1 year ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

Dockerfiles/Dockerfile docker
  • pytorch/pytorch latest build
Pipfile pypi
  • ipykernel * develop
  • graphviz *
  • h5py *
  • matplotlib *
  • opencv-python *
  • pandas *
  • pyyaml *
  • torchview *
  • ultralytics *
  • wandb *
Pipfile.lock pypi
  • asttokens ==2.4.1 develop
  • comm ==0.2.0 develop
  • debugpy ==1.8.0 develop
  • decorator ==5.1.1 develop
  • exceptiongroup ==1.2.0 develop
  • executing ==2.0.1 develop
  • ipykernel ==6.27.0 develop
  • ipython ==8.17.2 develop
  • jedi ==0.19.1 develop
  • jupyter-client ==8.6.0 develop
  • jupyter-core ==5.5.0 develop
  • matplotlib-inline ==0.1.6 develop
  • nest-asyncio ==1.5.8 develop
  • packaging ==23.2 develop
  • parso ==0.8.3 develop
  • pexpect ==4.8.0 develop
  • platformdirs ==4.0.0 develop
  • prompt-toolkit ==3.0.41 develop
  • psutil ==5.9.6 develop
  • ptyprocess ==0.7.0 develop
  • pure-eval ==0.2.2 develop
  • pygments ==2.17.2 develop
  • python-dateutil ==2.8.2 develop
  • pyzmq ==25.1.1 develop
  • six ==1.16.0 develop
  • stack-data ==0.6.3 develop
  • tornado ==6.3.3 develop
  • traitlets ==5.13.0 develop
  • wcwidth ==0.2.12 develop
  • appdirs ==1.4.4
  • certifi ==2023.11.17
  • charset-normalizer ==3.3.2
  • click ==8.1.7
  • contourpy ==1.2.0
  • cycler ==0.12.1
  • docker-pycreds ==0.4.0
  • filelock ==3.13.1
  • fonttools ==4.45.0
  • fsspec ==2023.10.0
  • gitdb ==4.0.11
  • gitpython ==3.1.40
  • graphviz ==0.20.1
  • h5py ==3.10.0
  • idna ==3.4
  • jinja2 ==3.1.2
  • kiwisolver ==1.4.5
  • markupsafe ==2.1.3
  • matplotlib ==3.8.2
  • mpmath ==1.3.0
  • networkx ==3.2.1
  • numpy ==1.26.2
  • nvidia-cublas-cu12 ==12.1.3.1
  • nvidia-cuda-cupti-cu12 ==12.1.105
  • nvidia-cuda-nvrtc-cu12 ==12.1.105
  • nvidia-cuda-runtime-cu12 ==12.1.105
  • nvidia-cudnn-cu12 ==8.9.2.26
  • nvidia-cufft-cu12 ==11.0.2.54
  • nvidia-curand-cu12 ==10.3.2.106
  • nvidia-cusolver-cu12 ==11.4.5.107
  • nvidia-cusparse-cu12 ==12.1.0.106
  • nvidia-nccl-cu12 ==2.18.1
  • nvidia-nvjitlink-cu12 ==12.3.101
  • nvidia-nvtx-cu12 ==12.1.105
  • opencv-python ==4.8.1.78
  • packaging ==23.2
  • pandas ==2.1.3
  • pillow ==10.1.0
  • protobuf ==4.25.1
  • psutil ==5.9.6
  • py-cpuinfo ==9.0.0
  • pyparsing ==3.1.1
  • python-dateutil ==2.8.2
  • pytz ==2023.3.post1
  • pyyaml ==6.0.1
  • requests ==2.31.0
  • scipy ==1.11.4
  • seaborn ==0.13.0
  • sentry-sdk ==1.36.0
  • setproctitle ==1.3.3
  • setuptools ==69.0.2
  • six ==1.16.0
  • smmap ==5.0.1
  • sympy ==1.12
  • thop ==0.1.1.post2209072238
  • torch ==2.1.1
  • torchview ==0.2.6
  • torchvision ==0.16.1
  • tqdm ==4.66.1
  • triton ==2.1.0
  • typing-extensions ==4.8.0
  • tzdata ==2023.3
  • ultralytics ==8.0.215
  • urllib3 ==2.1.0
  • wandb ==0.16.0
requirements.txt pypi
  • channels *
  • dependencies *
  • name *