Data

Tools

Indexes

Applications

Experiments

Open Source Science

dex-retargeting

Various retargeting optimizers to translate human hand motion to robot hand motion.

https://github.com/dexsuite/dex-retargeting

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.6%) to scientific vocabulary

Keywords

dexterous-manipulation dexterous-robotic-hand motion-retargeting
Last synced: 6 months ago · JSON representation ·

Repository

Various retargeting optimizers to translate human hand motion to robot hand motion.

Basic Info
Statistics
  • Stars: 535
  • Watchers: 6
  • Forks: 56
  • Open Issues: 13
  • Releases: 1
Topics
dexterous-manipulation dexterous-robotic-hand motion-retargeting
Created over 2 years ago · Last pushed 11 months ago
Metadata Files
Readme License Citation

README.md

Dex Retargeting

Various retargeting optimizers to translate human hand motion to robot hand motion.

Test Status Issues Closed Issues Releases pypi License

This repo originates from AnyTeleop Project

Retargeting with different hands.

Installation

shell pip install dex_retargeting

To run the example, you may need additional dependencies for rendering and hand pose detection.

shell git clone https://github.com/dexsuite/dex-retargeting cd dex-retargeting pip install -e ".[example]"

Changelog

v0.5.0

  • Numpy Support Update: Starting from this version, dex-retargeting supports numpy >= 2.0.0. If you need to use numpy < 2.0.0, you can install an earlier version of dex-retargeting using: bash pip install "dex-retargeting<0.5.0"

  • Mediapipe Compatibility: Although mediapipe lists numpy 1.x as a dependency, it is compatible with numpy >= 2.0.0. You can safely ignore any warnings related to this and continue using numpy 2.0.0 or higher.

  • Dependency Cleanup: Removed trimesh as a dependency to simplify installation and reduce potential conflicts. The core functionality of dex-retargeting no longer requires mesh processing capabilities.

Examples

Retargeting from human hand video

This type of retargeting can be used for applications like teleoperation, e.g. AnyTeleop.

Tutorial on retargeting from human hand video

Retarget from hand object pose dataset

teaser

This type of retargeting can be used post-process human data for robot imitation, e.g. DexMV.

Tutorial on retargeting from hand-object pose dataset

FAQ and Troubleshooting

Joint Orders for Retargeting

URDF parsers, such as ROS, physical simulators, real robot driver, and this repository, may parse URDF files with different joint orders. To use dex-retargeting results with other libraries, handle joint ordering explicitly using joint names, which are unique within a URDF file.

Example: Using dex-retargeting with the SAPIEN simulator

```python from dexretargeting.seqretarget import SeqRetargeting

retargeting: SeqRetargeting sapienjointnames = [joint.getname() for joint in robot.getactivejoints()] retargetingjointnames = retargeting.jointnames retargetingtosapien = np.array([retargetingjointnames.index(name) for name in sapienjointnames]).astype(int)

Use the index map to handle joint order differences

sapienrobot.setqpos(retargetqpos[retargetingto_sapien]) ```

This example retrieves joint names from the SAPIEN robot and SeqRetargeting object, creates a mapping array (retargeting_to_sapien) to map joint indices, and sets the SAPIEN robot's joint positions using the retargeted joint positions.

Citation

This repository is derived from the AnyTeleop Project and is subject to ongoing enhancements. If you utilize this work, please cite it as follows:

shell @inproceedings{qin2023anyteleop, title = {AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System}, author = {Qin, Yuzhe and Yang, Wei and Huang, Binghao and Van Wyk, Karl and Su, Hao and Wang, Xiaolong and Chao, Yu-Wei and Fox, Dieter}, booktitle = {Robotics: Science and Systems}, year = {2023} }

Acknowledgments

The robot hand models in this repository are sourced directly from dex-urdf. The robot kinematics in this repo are based on pinocchio. Examples use SAPIEN for rendering and visualization.

The PositionOptimizer leverages methodologies from our earlier project, From One Hand to Multiple Hands. Additionally, the DexPilotOptimizeris crafted using insights from DexPilot.

Owner

  • Name: dexsuite
  • Login: dexsuite
  • Kind: organization

Citation (CITATION.cff) 

cff-version: 1.2.0
authors:
  - family-names: "Qin"
    given-names: "Yuzhe"
  - family-names: "Yang"
    given-names: "Wei"
  - family-names: "Huang"
    given-names: "Binghao"
  - family-names: "Van Wyk"
    given-names: "Karl"
  - family-names: "Su"
    given-names: "Hao"
  - family-names: "Wang"
    given-names: "Xiaolong"
  - family-names: "Chao"
    given-names: "Yu-Wei"
  - family-names: "Fox"
    given-names: "Dieter"
date-released: 2023-10-25
title: "AnyTeleop"
message: "Thanks for using dex-retargeting. If you use this software, please cite it as below."
url: "https://github.com/dexsuite/dex-retargeting"

GitHub Events

Total
  • Create event: 6
  • Issues event: 48
  • Release event: 8
  • Watch event: 276
  • Delete event: 3
  • Issue comment event: 51
  • Push event: 8
  • Pull request event: 4
  • Fork event: 25
Last Year
  • Create event: 6
  • Issues event: 48
  • Release event: 8
  • Watch event: 276
  • Delete event: 3
  • Issue comment event: 51
  • Push event: 8
  • Pull request event: 4
  • Fork event: 25

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 22
  • Total pull requests: 1
  • Average time to close issues: 14 days
  • Average time to close pull requests: 6 minutes
  • Total issue authors: 17
  • Total pull request authors: 1
  • Average comments per issue: 0.27
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 22
  • Pull requests: 1
  • Average time to close issues: 14 days
  • Average time to close pull requests: 6 minutes
  • Issue authors: 17
  • Pull request authors: 1
  • Average comments per issue: 0.27
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • anonymouseea (3)
  • bb0928 (3)
  • xbkaishui (3)
  • Lingling006688 (2)
  • levi3001 (2)
  • jkim447 (2)
  • Simease (2)
  • Smilels (1)
  • On-The-Ways (1)
  • dbdxnuliba (1)
  • CodingCatMountain (1)
  • boreep (1)
  • haoyangzhanglab (1)
  • AII6 (1)
  • MARMOTatZJU (1)
Pull Request Authors
  • yzqin (4)
  • Dingry (2)
  • xbkaishui (2)
  • tylerlum (1)
  • ixygo (1)
  • git-xuefu (1)
  • giuliano-97 (1)
  • AII6 (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 18,221 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 17
  • Total maintainers: 1
pypi.org: dex-retargeting

Hand pose retargeting for dexterous robot hand.

  • Versions: 17
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 18,221 Last month
Rankings
Dependent packages count: 9.8%
Average: 37.4%
Dependent repos count: 65.0%
Maintainers (1)
yuzheqin
Last synced: 7 months ago