Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
✓DOI references
Found 2 DOI reference(s) in README -
○Academic publication links
-
✓Committers with academic emails
2 of 5 committers (40.0%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (14.1%) to scientific vocabulary
Last synced: 11 months ago
·
JSON representation
Repository
Manage scientific data
Basic Info
- Host: GitHub
- Owner: jic-dtool
- License: mit
- Language: Python
- Default Branch: master
- Size: 428 KB
Statistics
- Stars: 16
- Watchers: 2
- Forks: 0
- Open Issues: 4
- Releases: 0
Created almost 9 years ago
· Last pushed about 2 years ago
Metadata Files
Readme
Changelog
License
README.rst
dtool: Manage Scientific Data
=============================
.. |dtool| image:: https://github.com/jic-dtool/dtool/blob/master/icons/22x22/dtool_logo.png?raw=True
:height: 20px
:target: https://github.com/jic-dtool/dtool
.. |pypi| image:: https://badge.fury.io/py/dtool.svg
:target: http://badge.fury.io/py/dtool
:alt: PyPi package
.. |test| image:: https://img.shields.io/github/actions/workflow/status/jic-dtool/dtool/test.yml?branch=master&label=tests
:target: https://github.com/jic-dtool/dtool/actions/workflows/test.yml
.. |docs| image:: https://readthedocs.org/projects/dtool/badge/?version=latest
:target: https://readthedocs.org/projects/dtool?badge=latest
:alt: Documentation Status
|dtool| |pypi| |test| |docs|
*Make your data more resilient, portable and easy to work with by packaging
files & metadata into self contained datasets.*
- Documentation: http://dtool.readthedocs.io
- Paper: https://doi.org/10.7717/peerj.6562
- Free software: MIT License
Overview
--------
dtool is a suite of software for managing scientific data and making it
accessible programmatically. It consists of a command line interface ``dtool``
and a Python API: `dtoolcore `_.
The ``dtool`` command line interface allows one to organise files into datasets
and to move datasets between different storage solutions, for example from
local disk to remote object storage. Importantly it also provides methods to
verify that the transfer has been successful.
The Python API gives complete access to the data and metadata in a dataset. It
makes it easy to create scripts for processing the items, or a subset of items,
in a dataset. The Python API also allows datasets to be constructed
programmatically.
dtool is extensible, meaning that it is possible to create plugins both for
adding functionality to the command line interface and for creating interfaces
to custom storage backends.
The ``dtool`` Python package is a meta package that installs the packages:
- `dtoolcore `_ - core API
- `dtool-cli `_ - CLI plugin scaffold
- `dtool-annotation `_ - CLI commands for working with dataset annotations
- `dtool-config `_ - CLI commands for configuring dtool
- `dtool-create `_ - CLI commands for creating datasets
- `dtool-info `_ - CLI commands for getting information about datasets
- `dtool-overlay `_ - CLI commands for working with per item metadata stored as overlays
- `dtool-symlink `_ - storage broker interface allowing symlinking to data
- `dtool-http `_ - storage broker interface allowing read only access to datasets over HTTP
Installation::
$ pip install dtool
There are support packages for several object storage solutions:
- `dtool-s3 `_ - storage broker interface to S3 object storage
- `dtool-smb `_ - storage broker interface to smb network share
- `dtool-azure `_ - storage broker interface to Azure Storage
- `dtool-ecs `_ - storage broker interface to ECS S3 object storage
- `dtool-irods `_ - storage broker interface to iRODS
If you have access to Amazon S3, Microsoft Azure, ECS S3 or iRODS storage you may also want to install support for these::
$ pip install dtool-s3 dtool-azure dtool-ecs dtool-irods
Usage::
$ dtool create my-awesome-dataset
Created proto dataset file:///Users/olssont/my-awesome-dataset
Next steps:
1. Add raw data, eg:
dtool add item my_file.txt file:///Users/olssont/my-awesome-dataset
Or use your system commands, e.g:
mv my_data_directory /Users/olssont/my-awesome-dataset/data/
2. Add descriptive metadata, e.g:
dtool readme interactive file:///Users/olssont/my-awesome-dataset
3. Convert the proto dataset into a dataset:
dtool freeze file:///Users/olssont/my-awesome-dataset
Owner
- Name: dtool
- Login: jic-dtool
- Kind: organization
- Repositories: 34
- Profile: https://github.com/jic-dtool
GitHub Events
Total
Last Year
Committers
Last synced: over 2 years ago
Top Committers
| Name | Commits | |
|---|---|---|
| Tjelvar Olsson | t****n@j****k | 177 |
| Tjelvar Olsson | t****n@g****m | 10 |
| Tjelvar Olsson | o****t@n****k | 9 |
| Jan Janssen | j****n@u****m | 1 |
| Matthew Hartley | m****y@c****t | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 12 months ago
All Time
- Total issues: 3
- Total pull requests: 4
- Average time to close issues: 17 days
- Average time to close pull requests: about 16 hours
- Total issue authors: 2
- Total pull request authors: 3
- Average comments per issue: 2.33
- Average comments per pull request: 0.25
- Merged pull requests: 2
- Bot issues: 0
- Bot pull requests: 2
Past Year
- Issues: 0
- Pull requests: 3
- Average time to close issues: N/A
- Average time to close pull requests: 24 minutes
- Issue authors: 0
- Pull request authors: 2
- Average comments per issue: 0
- Average comments per pull request: 0.0
- Merged pull requests: 1
- Bot issues: 0
- Bot pull requests: 2
Top Authors
Issue Authors
- tjelvar-olsson (2)
- robbytobby (1)
Pull Request Authors
- dependabot[bot] (3)
- jotelha (2)
- jan-janssen (1)
Top Labels
Issue Labels
enhancement (2)
question (1)
Pull Request Labels
dependencies (3)
github_actions (3)
Packages
- Total packages: 2
-
Total downloads:
- pypi 288 last-month
-
Total dependent packages: 1
(may contain duplicates) -
Total dependent repositories: 4
(may contain duplicates) - Total versions: 59
- Total maintainers: 2
pypi.org: dtool
dtool command line client for managing data
- Documentation: https://dtool.readthedocs.io
- License: mit
-
Latest release: 3.27.0
published about 2 years ago
Rankings
Dependent repos count: 7.5%
Dependent packages count: 10.0%
Stargazers count: 15.2%
Average: 15.9%
Downloads: 17.0%
Forks count: 29.8%
Maintainers (2)
Last synced:
11 months ago
conda-forge.org: dtool
The dtool command line interface allows one to organise files into datasets and to move datasets between different storage solutions, for example from local disk to remote object storage. Importantly it also provides methods to verify that the transfer has been successful. The Python API gives complete access to the data and metadata in a dataset. It makes it easy to create scripts for processing the items, or a subset of items, in a dataset. The Python API also allows datasets to be constructed programatically.
- Homepage: https://github.com/jic-dtool/dtool
- License: MIT
-
Latest release: 3.26.2
published over 4 years ago
Rankings
Dependent repos count: 34.0%
Stargazers count: 48.3%
Average: 48.6%
Dependent packages count: 51.2%
Forks count: 61.1%
Last synced:
11 months ago
Dependencies
setup.py
pypi
- dtool-annotation ==0.1.1
- dtool-cli ==0.7.1
- dtool-config ==0.4.1
- dtool-create ==0.23.4
- dtool-http ==0.5.1
- dtool-info ==0.16.2
- dtool-overlay ==0.3.1
- dtool-symlink ==0.3.1
- dtool-tag ==0.1.1
- dtoolcore ==3.18.2