https://github.com/google-research/kubric
A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.
Science Score: 36.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
3 of 32 committers (9.4%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (15.5%) to scientific vocabulary
Keywords from Contributors
Repository
A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.
Basic Info
Statistics
- Stars: 2,551
- Watchers: 39
- Forks: 251
- Open Issues: 73
- Releases: 2
Metadata Files
README.md
Kubric
A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.
Motivation and design
We need better data for training and evaluating machine learning systems, especially in the context of unsupervised multi-object video understanding. Current systems succeed on toy datasets, but fail on real-world data. Progress could be greatly accelerated if we had the ability to create suitable datasets of varying complexity on demand. Kubric is mainly built on-top of pybullet (for physics simulation) and Blender (for rendering); however, the code is kept modular to potentially support different rendering backends.
Getting started
For instructions, please refer to https://kubric.readthedocs.io
Assuming you have docker installed, to generate the data above simply execute:
git clone https://github.com/google-research/kubric.git
cd kubric
docker pull kubricdockerhub/kubruntu
docker run --rm --interactive \
--user $(id -u):$(id -g) \
--volume "$(pwd):/kubric" \
kubricdockerhub/kubruntu \
/usr/bin/python3 examples/helloworld.py
ls output
Kubric employs Blender 2.93 (see here), so if you want to inspect the generated *.blend scene file for interactive inspection (i.e. without needing to render the scene), please make sure you have installed the correct Blender version.
Requirements
- A pipeline for conveniently generating video data.
- Physics simulation for automatically generating physical interactions between multiple objects.
- Good control over the complexity of the generated data, so that we can evaluate individual aspects such as variability of objects and textures.
- Realism: Ideally, the ability to span the entire complexity range from CLEVR all the way to real-world video such as YouTube8. This is clearly not feasible, but we would like to get as close as possible.
- Access to rich ground truth information about the objects in a scene for the purpose of evaluation (eg. object segmentations and properties)
- Control the train/test split to evaluate compositionality and systematic generalization (for example on held-out combinations of features or objects)
Challenges and datasets
Generally, we store datasets for the challenges in this Google Cloud Bucket. More specifically, these challenges are dataset contributions of the Kubric CVPR'22 paper: * MOVi: Multi-Object Video * Texture-Structure in NeRF * Optical Flow * Pre-training Visual Representations * Robust NeRF * Multi-View Object Matting * Complex BRDFs * Single View Reconstruction * Video Based Reconstruction * Point Tracking
Pointers to additional datasets/workers: * ToyBox (from Neural Semantic Fields) * MultiShapeNet (from Scene Representation Transformer) * SyntheticTrio(from Controllable Neural Radiance Fields)
Bibtex
@article{greff2021kubric,
title = {Kubric: a scalable dataset generator},
author = {Klaus Greff and Francois Belletti and Lucas Beyer and Carl Doersch and
Yilun Du and Daniel Duckworth and David J Fleet and Dan Gnanapragasam and
Florian Golemo and Charles Herrmann and Thomas Kipf and Abhijit Kundu and
Dmitry Lagun and Issam Laradji and Hsueh-Ti (Derek) Liu and Henning Meyer and
Yishu Miao and Derek Nowrouzezahrai and Cengiz Oztireli and Etienne Pot and
Noha Radwan and Daniel Rebain and Sara Sabour and Mehdi S. M. Sajjadi and Matan Sela and
Vincent Sitzmann and Austin Stone and Deqing Sun and Suhani Vora and Ziyu Wang and
Tianhao Wu and Kwang Moo Yi and Fangcheng Zhong and Andrea Tagliasacchi},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2022},
}
Disclaimer
This is not an official Google Product
Owner
- Name: Google Research
- Login: google-research
- Kind: organization
- Location: Earth
- Website: https://research.google
- Repositories: 226
- Profile: https://github.com/google-research
GitHub Events
Total
- Issues event: 5
- Watch event: 234
- Issue comment event: 11
- Push event: 10
- Pull request review event: 5
- Pull request review comment event: 4
- Pull request event: 12
- Fork event: 41
- Create event: 3
Last Year
- Issues event: 5
- Watch event: 234
- Issue comment event: 11
- Push event: 10
- Pull request review event: 5
- Pull request review comment event: 4
- Pull request event: 12
- Fork event: 41
- Create event: 3
Committers
Last synced: 11 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Klaus Greff | k****g@g****m | 248 |
| Andrea Tagliasacchi | a****i@g****m | 176 |
| cdoersch | d****h@g****m | 10 |
| Issam H. Laradji | I****i | 9 |
| Walter Wu | w****n@g****m | 8 |
| Matthew Prasad Burruss | b****w@g****m | 7 |
| Mehdi S. M. Sajjadi | 7****g | 3 |
| Philipp Henzler | p****4@g****m | 3 |
| Yilun Du | y****u@f****m | 3 |
| Conchylicultor | e****t@g****m | 3 |
| Andrea Tagliasacchi | a****b@g****m | 2 |
| Ricardo Martin Brualla | r****a@g****m | 2 |
| Dilara Gokay | d****y@g****m | 2 |
| Daniel Rebain | d****n@c****a | 2 |
| Yi Yang | y****a@g****m | 2 |
| Francois Belletti | b****f@g****m | 1 |
| Frano Rajič | f****h@g****m | 1 |
| Henning Meyer | t****n@g****m | 1 |
| Leonard Bruns | r****9@g****m | 1 |
| Michael Bannert | m****t@t****e | 1 |
| MrXandbadas | k****a@g****m | 1 |
| Pavel Sountsov | 4****x | 1 |
| Rayne | R****e | 1 |
| Sarasra | s****r@g****m | 1 |
| Sosuke Kobayashi | s****k | 1 |
| Yilun Du | y****u@m****u | 1 |
| Austin Stone | a****e@g****m | 1 |
| Florian Golemo | f****o@g****m | 1 |
| ziyuw | z****g@g****m | 1 |
| wuchangchang | c****u@g****m | 1 |
| and 2 more... | ||
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 8 months ago
All Time
- Total issues: 91
- Total pull requests: 65
- Average time to close issues: 3 months
- Average time to close pull requests: about 1 month
- Total issue authors: 61
- Total pull request authors: 32
- Average comments per issue: 2.11
- Average comments per pull request: 0.85
- Merged pull requests: 38
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 7
- Pull requests: 14
- Average time to close issues: N/A
- Average time to close pull requests: about 4 hours
- Issue authors: 7
- Pull request authors: 8
- Average comments per issue: 0.29
- Average comments per pull request: 0.57
- Merged pull requests: 11
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- taiya (10)
- Qwlouse (4)
- wangyuyy (3)
- DelinQu (3)
- MrXandbadas (3)
- HAWLYQ (2)
- wenhsuanchu (2)
- heurainbow (2)
- Feeeeeel (2)
- chaitanya100100 (2)
- malfonsoarquimea (2)
- Wuziyi616 (2)
- fire (2)
- gabriel-v (2)
- SiegeLordEx (2)
Pull Request Authors
- cdoersch (15)
- Qwlouse (7)
- NU-iz-da-fam (5)
- taiya (5)
- henzler (5)
- burrussmp (5)
- rmbrualla (4)
- dilaragokay (4)
- yangyi02 (4)
- wuchangchang (2)
- jlidard (2)
- MrXandbadas (2)
- m43 (2)
- Xiaoming-Zhao (2)
- jdrinane (2)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 3
-
Total downloads:
- pypi 1,371 last-month
-
Total dependent packages: 0
(may contain duplicates) -
Total dependent repositories: 2
(may contain duplicates) - Total versions: 800
- Total maintainers: 1
proxy.golang.org: github.com/google-research/kubric
- Documentation: https://pkg.go.dev/github.com/google-research/kubric#section-documentation
- License: apache-2.0
-
Latest release: v0.1.1
published over 4 years ago
Rankings
pypi.org: kubric-nightly
A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation, depth maps, and optical flow.
- Homepage: https://github.com/google-research/kubric
- Documentation: https://kubric-nightly.readthedocs.io/
- License: Apache 2.0
-
Latest release: 2023.12.27
published over 2 years ago
Rankings
Maintainers (1)
pypi.org: kubric
A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation, depth maps, and optical flow.
- Homepage: https://github.com/google-research/kubric
- Documentation: https://kubric.readthedocs.io/
- License: Apache 2.0
-
Latest release: 0.1.1
published over 4 years ago
Rankings
Maintainers (1)
Dependencies
- sphinx ==3.3.1
- sphinx-rtd-theme ==0.5.1
- apache-beam *
- bidict *
- cloudml-hypertune *
- dataclasses *
- etils *
- google-cloud-storage *
- imageio *
- munch *
- numpy >=1.17
- pandas *
- pypng *
- pyquaternion *
- python-Levenshtein *
- scikit-learn *
- singledispatchmethod *
- tensorflow *
- tensorflow-datasets >=4.1.0
- traitlets *
- trimesh *
- flake8 * development
- hypothesis * development
- pylint * development
- pytest * development
- pytest-cov * development
- OpenEXR *
- matplotlib *
- mediapy *
- pybullet *
- tensorflow-graphics *
- actions/cache v2 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- docker/build-push-action v2 composite
- docker/login-action v1 composite
- docker/setup-buildx-action v1 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- actions/checkout v2 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- actions/checkout v2 composite
- codecov/codecov-action v1 composite
- actions/cache v2 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- docker/build-push-action v2 composite
- docker/login-action v1 composite
- docker/setup-buildx-action v1 composite
- actions/cache v2 composite
- actions/checkout v2 composite
- actions/setup-python v2 composite
- docker/build-push-action v2 composite
- docker/login-action v1 composite
- docker/setup-buildx-action v1 composite
- kubricdockerhub/kubruntu latest build