xbatcher

Batch generation from xarray datasets

https://github.com/xarray-contrib/xbatcher

Science Score: 59.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 3 DOI reference(s) in README
✓
Academic publication links
Links to: zenodo.org
✓
Committers with academic emails
3 of 17 committers (17.6%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (14.8%) to scientific vocabulary

Keywords

keras pytorch tensorflow xarray xarray-accessor

Keywords from Contributors

energy-system mesh geoscience geophysics open-science earth-science energy-system-model hydrology medical-imaging climate-model

Last synced: 6 months ago · JSON representation

Repository

Batch generation from xarray datasets

Basic Info

Host: GitHub
Owner: xarray-contrib
License: apache-2.0
Language: Python
Default Branch: main
Homepage: https://xbatcher.readthedocs.io
Size: 460 KB

Statistics

Stars: 182
Watchers: 11
Forks: 28
Open Issues: 38
Releases: 3

Topics

keras pytorch tensorflow xarray xarray-accessor

Created over 7 years ago · Last pushed 6 months ago

Metadata Files

Readme Contributing License Citation

README.rst

xbatcher: Batch Generation from Xarray Datasets
===============================================

|Build Status| |codecov| |docs| |pypi| |conda-forge| |license| |zenodo|


Xbatcher is a small library for iterating Xarray DataArrays and Datasets in
batches. The goal is to make it easy to feed Xarray objects to machine
learning libraries such as PyTorch_ or TensorFlow_. View the |docs| for more
info.

.. _TensorFlow: https://www.tensorflow.org/

.. _PyTorch: https://pytorch.org/


.. |Build Status| image:: https://github.com/xarray-contrib/xbatcher/workflows/CI/badge.svg
   :target: https://github.com/xarray-contrib/xbatcher/actions
   :alt: github actions build status
.. |codecov| image:: https://codecov.io/gh/xarray-contrib/xbatcher/branch/main/graph/badge.svg
   :target: https://codecov.io/gh/xarray-contrib/xbatcher
   :alt: code coverage
.. |docs| image:: http://readthedocs.org/projects/xbatcher/badge/?version=latest
   :target: http://xbatcher.readthedocs.org/en/latest/?badge=latest
   :alt: docs
.. |pypi| image:: https://img.shields.io/pypi/v/xbatcher.svg
   :target: https://pypi.python.org/pypi/xbatcher
   :alt: pypi
.. |conda-forge| image:: https://img.shields.io/conda/vn/conda-forge/xbatcher.svg
   :target: https://anaconda.org/conda-forge/xbatcher
   :alt: conda-forge
.. |license| image:: https://img.shields.io/github/license/xarray-contrib/xbatcher.svg
   :target: https://github.com/xarray-contrib/xbatcher
   :alt: license
.. |zenodo| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.13776824.svg
   :target: https://doi.org/10.5281/zenodo.13776824
   :alt: zenodo

Installation
------------

Xbatcher can be installed from PyPI as::

    python -m pip install xbatcher

Or via Conda as::

    conda install -c conda-forge xbatcher

Or from source as::

    python -m pip install git+https://github.com/xarray-contrib/xbatcher.git

.. note::
   The required dependencies installed with Xbatcher are `Xarray `_,
   `Dask `_, and `NumPy `_.
   You will need to separately install `TensorFlow `_
   or `PyTorch `_ to use those data loaders or
   Xarray accessors. `Review the installation instructions `_
   for more details.

Documentation
-------------

Documentation is hosted on ReadTheDocs: https://xbatcher.readthedocs.org

License
------------

Apache License 2.0, see LICENSE file.

Acknowledgements
----------------

This work was funded in part by:

NASA ACCESS19-0049: Pangeo ML: Open Source Tools and Pipelines for Scalable Machine Learning Using NASA Earth Observation Data

This work was motivated by many conversations in the Pangeo community and Pangeo ML working group

Owner

Name: xarray-contrib
Login: xarray-contrib
Kind: organization

Website: https://xarray.dev
Twitter: xarray_dev
Repositories: 23
Profile: https://github.com/xarray-contrib

xarray compatible projects

GitHub Events

Total

Issues event: 3
Watch event: 14
Delete event: 23
Issue comment event: 25
Push event: 15
Pull request review comment event: 1
Pull request event: 50
Pull request review event: 3
Fork event: 2
Create event: 24

Last Year

Issues event: 3
Watch event: 14
Delete event: 23
Issue comment event: 25
Push event: 15
Pull request review comment event: 1
Pull request event: 50
Pull request review event: 3
Fork event: 2
Create event: 24

Committers

Last synced: 9 months ago

All Time

Total Commits: 198
Total Committers: 17
Avg Commits per committer: 11.647
Development Distribution Score (DDS): 0.722

Past Year

Commits: 25
Committers: 6
Avg Commits per committer: 4.167
Development Distribution Score (DDS): 0.56

Top Committers

Name	Email	Commits
dependabot[bot]	4****]	55
Max Jones	m**j@a**u	52
Joseph Hamman	j**n@u**u	29
Ryan Abernathey	r**y@g**m	14
pre-commit-ci[bot]	6****]	12
Anderson Banihirwe	1****5	12
tcchiao	t**o@g**m	6
Ray Bell	r**0@g**m	4
Raphael Hagen	n**n@g**m	3
Joe Hamman	j**e@e**o	2
RichardScottOZ	7****Z	2
Wei Ji	2****4	2
Bobby Jackson	r**n@a**v	1
James Bourbeau	j****u	1
Joe Hamman	j**e@c**g	1
Cindy Chiao	c**o@l**m	1
TJ Vandal	t**0@g**m	1

Committer Domains (Top 20 + Academic)

lyft.com: 1 carbonplan.org: 1 anl.gov: 1 earthmover.io: 1 ucar.edu: 1 alum.mit.edu: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 42
Total pull requests: 211
Average time to close issues: 5 months
Average time to close pull requests: 30 days
Total issue authors: 13
Total pull request authors: 15
Average comments per issue: 3.45
Average comments per pull request: 1.18
Merged pull requests: 131
Bot issues: 0
Bot pull requests: 134

Past Year

Issues: 3
Pull requests: 63
Average time to close issues: 24 days
Average time to close pull requests: 7 days
Issue authors: 3
Pull request authors: 4
Average comments per issue: 1.33
Average comments per pull request: 0.76
Merged pull requests: 21
Bot issues: 0
Bot pull requests: 49

View more stats

Top Authors

Issue Authors

maxrjones (16)
jhamman (6)
cmdupuis3 (5)
weiji14 (3)
andersy005 (2)
djhoese (2)
arbennett (2)
rbavery (1)
dhruvbalwada (1)
tcchiao (1)
ljstrnadiii (1)
agirnow (1)
s-kganz (1)

Pull Request Authors

dependabot[bot] (120)
maxrjones (35)
andersy005 (22)
pre-commit-ci[bot] (14)
jhamman (5)
ThomasMGeo (3)
rcjackson (2)
arbennett (2)
weiji14 (2)
norlandrhagen (1)
tjvandal (1)
ljstrnadiii (1)
cmdupuis3 (1)
djhoese (1)
s-kganz (1)

Top Labels

Issue Labels

documentation (7) question (6) bug (6) feature (5) maintenance (5) enhancement (3) wontfix (1) community (1) duplicate (1) use case (1)

Pull Request Labels

dependencies (120) github_actions (120) maintenance (34) skip-changelog (23) documentation (16) feature (8) enhancement (3) bug (2) python (2)

Packages

Total packages: 2
Total downloads:
- pypi 27,571 last-month
Total docker downloads: 70

Total dependent packages: 3
(may contain duplicates)
Total dependent repositories: 4
(may contain duplicates)
Total versions: 6
Total maintainers: 3

pypi.org: xbatcher

Batch generation from Xarray objects

Documentation: https://xbatcher.readthedocs.io/
License: Apache
Latest release: 0.4.0
published over 1 year ago

Versions: 4
Dependent Packages: 3
Dependent Repositories: 2
Downloads: 27,571 Last month
Docker Downloads: 70

Rankings

Dependent packages count: 4.7%

Docker downloads count: 4.8%

Average: 8.8%

Dependent repos count: 11.6%

Downloads: 14.3%

Maintainers (3)

rabernat jhamman maxrjones

Last synced: 6 months ago

conda-forge.org: xbatcher

Xbatcher is a small library for iterating xarray DataArrays in batches. The goal is to make it easy to feed xarray datasets to machine learning libraries such as PyTorch or TensorFlow.

Homepage: https://github.com/xarray-contrib/xbatcher
License: Apache-2.0
Latest release: 0.2.0
published over 3 years ago

Versions: 2
Dependent Packages: 0
Dependent Repositories: 2

Rankings

Dependent repos count: 20.2%

Stargazers count: 34.0%

Forks count: 34.7%

Average: 35.1%

Dependent packages count: 51.6%

Last synced: 6 months ago

Dependencies

.github/workflows/main.yaml actions

actions/cache v3 composite
actions/checkout v3 composite
actions/setup-python v4.5.0 composite
codecov/codecov-action v3.1.1 composite

.github/workflows/pypi-release.yaml actions

actions/checkout v3 composite
actions/download-artifact v3 composite
actions/setup-python v4.5.0 composite
actions/upload-artifact v3 composite
pypa/gh-action-pypi-publish v1.6.4 composite

.github/workflows/release-drafter.yml actions

release-drafter/release-drafter v5 composite

.github/workflows/testpypi-release.yaml actions

actions/checkout v3.2.0 composite
actions/setup-python v4.5.0 composite
pypa/gh-action-pypi-publish v1.6.4 composite

ci/requirements/environment.yml conda

adlfs
asv
black
blackdoc
dask
docformatter
flake8
isort >=5
numpy
pip
pre-commit
pylint
pytest
pytest-cov
python 3.9.*
pytorch
tensorflow
xarray
zarr

pyproject.toml pypi

dask *
numpy *
xarray *