Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/cheind/pytorch-blender-dr

https://github.com/cheind/pytorch-blender-dr

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (16.7%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: cheind
  • License: mit
  • Language: Jupyter Notebook
  • Default Branch: master
  • Size: 89.2 MB
Statistics
  • Stars: 5
  • Watchers: 4
  • Forks: 0
  • Open Issues: 1
  • Releases: 0
Created almost 6 years ago · Last pushed over 5 years ago
Metadata Files
Readme License

README.md

pytorch-blender-dr

Provides code to recreate our results on 2D object detection und keypoint detection using adaptive domain randomization. In particular, we show that training on massively randomized low-fidelity simulation generalizes to real world data.

Two industrial use cases

TLess - TextureLess Objects

Links - TLess details - TLess format - Other challenges

Kitchen cabinets

TODO

Generating data

Requires blendtorch + Blender 2.9 installed. For occluder objects we use this supershape library, that needs to be accessible from within Blender.

To generate data, run python record.py <scene> where <scene> is less or kitchen. This will generate offline data (.btr files) in tmp directory. Example below.

Invoke python record.py --help for more options.

Training and Evaluation

Create a new Conda environment conda create -n icpr python==3.7 pip conda activate icpr pip install -r requirements.txt

Setup BlendTorch: ``` Detailed instructions can be found at: https://github.com/cheind/pytorch-blender under 'Installation' in the readme.

git clone https://github.com/cheind/pytorch-blender.git pip install -e /pkg_pytorch ```

How to run scripts: ``` Run the main script for training with:

python -m pytorch.main --config exampletrain.txt Run the main script for mean average precision calculation with: python -m pytorch.main --config exampletest.txt NOTE: Change trainpath and inferencepath in the exampletrain.txt and exampletest.txt s.t. it matches your data location!

The example configuration above will replay from existing BlendTorch replay file(s) with a '.btr' extension. Thus, run the record.py script beforehand with:

python record.py tless as stated above.

We train and choose the best performing model only by training on BlendTorch generated data (~50k images) and evaluate the performance (mAP metric) with real world data taken by a primesense camera (~1000 images).

During training models are stored in the 'models' folder as modellast.pth or modelbest.pth!

Train and evaluate multiple times with the 'ntimes.sh' script. It will use the settings from exampletrain.txt and exampletest.txt for training and evaluation respectively.

Get detailed mAP summary report over single or multiple trainings runs (ntimes.sh) by running the 'coco_report.py' file:

python -m pytorch.cocoreport './evaluation' where './evaluation' is the path to gt.json and pred.json (or multiple pred*.json) file. Track/Visualize training progress with tensorboard: tensorboard --logdir runs --bindall ```

Augmented BlendTorch image with ground truth and prediction annotations:

Precision/Recall curves over single classes:

CenterNet's dense prediction output heatmaps (prediction, ground truth):

Owner

  • Name: Christoph Heindl
  • Login: cheind
  • Kind: user
  • Location: Austrian area

I am a computer scientist working at the interface of perception, robotics and deep learning.

GitHub Events

Total
Last Year

Issues and Pull Requests

Last synced: about 1 year ago

All Time
  • Total issues: 1
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • ggaziv (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements.txt pypi
  • albumentations >=0.4
  • pycocotools >=2.0
  • tensorboard >=2.3
  • torch >=1.6.0
  • torchvision >=0.7.0
  • tqdm >=4.40