Overview

This repository is a port of pbrshumanoid from the Biomimetic Robotics Lab which itself is a port of legged_gym from the RSL research group

Installation

This code was tested on Ubuntu 20.04 1. Create a new python virtual env with python 3.6, 3.7 or 3.8 (3.8 recommended) 2. Install pytorch 1.10 with cuda-11.3: - pip install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html 3. Install Isaac Gym - Download and install Isaac Gym Preview 4 from https://developer.nvidia.com/isaac-gym (extract the zip package, copy the isaacgym folder within the package whereever you want it to live - I prefer in the directory with my virtual enviornment) - cd isaacgym/python && pip install -e . to install the requirements - Try running an example cd examples && python 1080_balls_of_solitude.py (you need to execute the examples from the examples directory) - For troubleshooting check docs isaacgym/docs/index.html) 4. Install gpurl (PPO implementation) - `cd gpurl && pip install -e . 5. Install gpuGym -cd .. && pip install -e . 6. Install other dependencies -pip install statsmodels && pip install seaborn && pip install numpy==1.20.3`

CODE STRUCTURE

1. Each environment is defined by an env file (legged_robot.py) and a config file (legged_robot_config.py). The config file contains two classes: one containing all the environment parameters (LeggedRobotCfg) and one for the training parameters (LeggedRobotCfgPPo).
   
2. Both env and config classes use inheritance.
   
3. Each non-zero reward scale specified in cfg will add a function with a corresponding name to the list of elements which will be summed to get the total reward.
   
4. Tasks must be registered using task_registry.register(name, EnvClass, EnvConfig, TrainConfig). This is done in envs/__init__.py, but can also be done from outside of this repository.
   

Use

1. Train:
   python gpugym/scripts/train.py --task=pbrs:humanoid
   • To run on CPU add following arguments: --sim_device=cpu, --rl_device=cpu (sim on CPU and rl on GPU is possible).
   • To run headless (no rendering) add --headless. (Recommended)
   • To plot the rewards at the end of training add && python plotgraph.py.
   • Important: To improve performance, once the training starts press v to stop the rendering. You can then enable it later to check the progress.
   • The trained policy is saved in gpugym/logs/<experiment_name>/<date_time>_<run_name>/model_<iteration>.pt. Where <experiment_name> and <run_name> are defined in the train config.
   • The following command line arguments override the values set in the config files:
     • --task TASK: Task name.
     • --resume: Resume training from a checkpoint
     • --experimentname EXPERIMENTNAME: Name of the experiment to run or load.
     • --runname RUNNAME: Name of the run.
     • --loadrun LOADRUN: Name of the run to load when resume=True. If -1: will load the last run.
     • --checkpoint CHECKPOINT: Saved model checkpoint number. If -1: will load the last checkpoint.
     • --numenvs NUMENVS: Number of environments to create.
     • --seed SEED: Random seed.
     • --maxiterations MAXITERATIONS: Maximum number of training iterations.
2. Play a trained policy:
   python gpugym/scripts/play.py --task=pbrs:humanoid
   • By default the loaded policy is the last model of the last run of the experiment folder.
   • Other runs/model iteration can be selected by setting load_run and checkpoint in the train config.

Troubleshooting

1. If you get the following error: ImportError: libpython3.8m.so.1.0: cannot open shared object file: No such file or directory, do: sudo apt install libpython3.8
2. If you get the following error: RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error?, try restarting your computer.

Known Issues

1. The contact forces reported by net_contact_force_tensor are unreliable when simulating on GPU with a triangle mesh terrain. A workaround is to use force sensors, but the force are propagated through the sensors of consecutive bodies resulting in an undesireable behaviour. However, for a legged robot it is possible to add sensors to the feet/end effector only and get the expected results. When using the force sensors make sure to exclude gravity from trhe reported forces with sensor_options.enable_forward_dynamics_forces. Example: ``` sensorpose = gymapi.Transform() for name in feetnames: sensoroptions = gymapi.ForceSensorProperties() sensoroptions.enableforwarddynamicsforces = False # for example gravity sensoroptions.enableconstraintsolverforces = True # for example contacts sensoroptions.useworldframe = True # report forces in world frame (easier to get vertical components) index = self.gym.findassetrigidbodyindex(robotasset, name) self.gym.createassetforcesensor(robotasset, index, sensorpose, sensor_options) (...)

   sensortensor = self.gym.acquireforcesensortensor(self.sim) self.gym.refreshforcesensortensor(self.sim) forcesensorreadings = gymtorch.wraptensor(sensortensor) self.sensorforces = forcesensorreadings.view(self.num_envs, 4, 6)..., :3

   self.gym.refreshforcesensortensor(self.sim) contact = self.sensorforces[:, :, 2] > 1. ```