Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/cvat-ai/datumaro

Dataset Management Framework, a Python library and a CLI tool to build, analyze and manage Computer Vision datasets.

https://github.com/cvat-ai/datumaro

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.5%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Dataset Management Framework, a Python library and a CLI tool to build, analyze and manage Computer Vision datasets.

Basic Info
  • Host: GitHub
  • Owner: cvat-ai
  • License: mit
  • Language: Python
  • Default Branch: develop
  • Homepage:
  • Size: 14.1 MB
Statistics
  • Stars: 42
  • Watchers: 2
  • Forks: 17
  • Open Issues: 14
  • Releases: 0
Fork of open-edge-platform/datumaro
Created almost 4 years ago · Last pushed about 1 year ago
Metadata Files
Readme Changelog License

README.md

Dataset Management Framework (Datumaro)

Build status Codacy Badge Codacy Badge

A framework and CLI tool to build, transform, and analyze datasets.

VOC dataset ---> Annotation tool + / COCO dataset -----> Datumaro ---> dataset ------> Model training + \ CVAT annotations ---> Publication, statistics etc. <!--lint enable fenced-code-flag-->

Features

(Back to top)

Other formats and documentation for them can be found here. - Dataset building - Merging multiple datasets into one - Dataset filtering by a custom criteria: - remove polygons of a certain class - remove images without annotations of a specific class - remove occluded annotations from images - keep only vertically-oriented images - remove small area bounding boxes from annotations - Annotation conversions, for instance: - polygons to instance masks and vice-versa - apply a custom colormap for mask annotations - rename or remove dataset labels - Splitting a dataset into multiple subsets like train, val, and test: - random split - task-specific splits based on annotations, which keep initial label and attribute distributions - for classification task, based on labels - for detection task, based on bboxes - for re-identification task, based on labels, avoiding having same IDs in training and test splits - Sampling a dataset - analyzes inference result from the given dataset and selects the ‘best’ and the ‘least amount of’ samples for annotation. - Select the sample that best suits model training. - sampling with Entropy based algorithm - Dataset quality checking - Simple checking for errors - Comparison with model inference - Merging and comparison of multiple datasets - Annotation validation based on the task type(classification, etc) - Dataset comparison - Dataset statistics (image mean and std, annotation statistics) - Model integration - Inference (OpenVINO, Caffe, PyTorch, TensorFlow, MxNet, etc.) - Explainable AI (RISE algorithm) - RISE for classification - RISE for object detection

Check the design document for a full list of features. Check the user manual for usage instructions.

Contributing

(Back to top)

Feel free to open an Issue, if you think something needs to be changed. You are welcome to participate in development, instructions are available in our contribution guide.

Telemetry data collection note

The OpenVINO™ telemetry library is used to collect basic information about Datumaro usage.

To enable/disable telemetry data collection please see the guide.

Owner

  • Name: cvat.ai
  • Login: cvat-ai
  • Kind: organization
  • Email: contact+github@cvat.ai

CVAT.ai Corporation

GitHub Events

Total
  • Issues event: 11
  • Watch event: 12
  • Delete event: 47
  • Issue comment event: 266
  • Push event: 258
  • Pull request review event: 327
  • Pull request review comment event: 334
  • Pull request event: 89
  • Fork event: 6
  • Create event: 45
Last Year
  • Issues event: 11
  • Watch event: 12
  • Delete event: 47
  • Issue comment event: 266
  • Push event: 258
  • Pull request review event: 327
  • Pull request review comment event: 334
  • Pull request event: 89
  • Fork event: 6
  • Create event: 45

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 6
  • Total pull requests: 34
  • Average time to close issues: about 1 month
  • Average time to close pull requests: 5 days
  • Total issue authors: 3
  • Total pull request authors: 2
  • Average comments per issue: 0.0
  • Average comments per pull request: 1.35
  • Merged pull requests: 28
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 6
  • Pull requests: 34
  • Average time to close issues: about 1 month
  • Average time to close pull requests: 5 days
  • Issue authors: 3
  • Pull request authors: 2
  • Average comments per issue: 0.0
  • Average comments per pull request: 1.35
  • Merged pull requests: 28
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • zhiltsov-max (6)
  • archibald1418 (2)
  • dzcmingdi (2)
  • Eldies (1)
  • teja1412-hub (1)
  • husia777 (1)
Pull Request Authors
  • Eldies (51)
  • zhiltsov-max (13)
  • Bobronium (2)
  • ChanBong (1)
  • klakhov (1)
Top Labels
Issue Labels
bug (2) enhancement (1) documentation (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads: unknown
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 15
proxy.golang.org: github.com/cvat-ai/datumaro
  • Versions: 15
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 6.5%
Average: 6.7%
Dependent repos count: 6.9%
Last synced: 10 months ago

Dependencies

package-lock.json npm
  • 272 dependencies
package.json npm
  • remark-cli ^9.0.0 development
  • remark-frontmatter ^3.0.0 development
  • remark-gfm ^1.0.0 development
  • remark-lint-emphasis-marker ^2.0.0 development
  • remark-lint-list-item-spacing ^2.0.0 development
  • remark-lint-maximum-heading-length ^2.0.0 development
  • remark-lint-maximum-line-length ^2.0.0 development
  • remark-lint-no-dead-urls ^1.0.2 development
  • remark-lint-no-file-name-irregular-characters ^1.0.3 development
  • remark-lint-ordered-list-marker-style ^2.0.0 development
  • remark-lint-strong-marker ^2.0.0 development
  • remark-lint-unordered-list-marker-style ^2.0.0 development
  • remark-preset-lint-consistent ^4.0.0 development
  • remark-preset-lint-markdown-style-guide ^4.0.0 development
  • remark-preset-lint-recommended ^5.0.0 development
  • source-map-support ^0.5.19 development
  • vfile-reporter-json ^2.0.2 development
site/package-lock.json npm
  • 115 dependencies
site/package.json npm
  • autoprefixer ^9.8.6 development
  • postcss-cli ^7.1.2 development
requirements-core.txt pypi
  • Pillow >=6.1.0
  • PyYAML >=5.3.1
  • attrs >=21.3.0
  • defusedxml >=0.7.0
  • h5py >=2.10.0
  • lxml >=4.4.1
  • matplotlib >=3.3.1
  • networkx >=2.5
  • nibabel >=3.2.1
  • numpy >=1.17.3
  • orjson >=3.6.6
  • pandas >=1.1.5
  • pycocotools >=2.0.4
  • pycocotools-windows *
  • requests *
  • ruamel.yaml >=0.17.0
  • scipy *
  • tensorboardX >=1.8,
  • typing_extensions >=3.7.4.3
requirements-default.txt pypi
  • dvc >=2.7.0
requirements.txt pypi
  • bandit >=1.7.0
  • black >=22.1.0
  • coverage *
  • isort *
  • opencv-python-headless >=4.1.0.25
  • openvino-telemetry >=2022.1.0
  • pylint >=2.7.0
  • pytest >=5.3.5