https://github.com/conda/conda-package-streaming

An efficient library to read from new and old format .conda and .tar.bz2 conda packages.

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
○
Committers with academic emails
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (10.4%) to scientific vocabulary

Keywords from Contributors

conda package-management docs archival parallel pypi packaging energy-system-model projection generic

Last synced: 10 months ago · JSON representation

Repository

An efficient library to read from new and old format .conda and .tar.bz2 conda packages.

Basic Info

Host: GitHub
Owner: conda
License: other
Language: Python
Default Branch: main
Homepage: https://conda.github.io/conda-package-streaming/
Size: 589 KB

Statistics

Stars: 13
Watchers: 7
Forks: 12
Open Issues: 8
Releases: 11

Created about 4 years ago · Last pushed 10 months ago

Metadata Files

Readme Changelog License Code of conduct Codeowners

conda-package-streaming

An efficient library to read from new and old format .conda and .tar.bz2 conda packages.

Download conda metadata from packages without transferring entire file. Get metadata from local .tar.bz2 packages without reading entire files.

Uses enhanced pip lazy_wheel to fetch a file out of .conda with no more than 3 range requests, but usually 2.

Uses tar = tarfile.open(fileobj=...) to stream remote .tar.bz2. Closes the HTTP request once desired files have been seen.

Quickstart

The basic API yields (tarfile, member) tuples from conda files as tarfile is needed to extract member. Note the .tar.bz2 format yields all members, not just info/, from stream_conda_info / stream_conda_component, while the .conda format yields members from the requested inner archive — allowing the caller to decide when to stop reading.

From a url, ```python from condapackagestreaming.url import streamcondainfo

url = (ends with .conda or .tar.bz2)

for tar, member in streamcondainfo(url): if member.name == "info/index.json": index_json = json.load(tar.extractfile(member)) break ```

From s3, ```python client = boto3.client("s3") from condapackagestreaming.s3 import streamcondainfo

key = (ends with .conda or .tar.bz2)

for tar, member in streamcondainfo(client, bucket, key): if member.name == "info/index.json": index_json = json.load(tar.extractfile(member)) break ```

From a filename, ```python from condapackagestreaming import package_streaming

filename = (ends with .conda or .tar.bz2)

for tar, member in packagestreaming.streamcondainfo(filename): if member.name == "info/index.json": indexjson = json.load(tar.extractfile(member)) break ```

From a file-like object, ```python from contextlib import closing

from condapackagestreaming.url import condareaderforurl from condapackagestreaming.packagestreaming import streamcondacomponent filename, conda = condareaderfor_url(url)

file object must be seekable for `.conda` format, but merely readable for `.tar.bz2`

with closing(conda): for tar, member in streamcondacomponent(filename, conda, component="info"): if member.name == "info/index.json": index_json = json.load(tar.extractfile(member)) break ```

If you need the entire package, download it first and use the file-based APIs. The URL-based APIs are more efficient if you only need to access package metadata.

Package goals

Extract conda packages (both formats)
Easy to install from pypi or conda
Do the least amount of I/O possible (no temporary files, transfer partial packages)
Open files from the network / standard HTTP / s3
Continue using conda-package-handling to create .conda packages

Generating documentation

Uses markdown, furo theme. Requires newer mdit-py-plugins.

pip install conda-package-streaming[docs]

One time: sphinx-apidoc -o docs .

Owner

Name: conda
Login: conda
Kind: organization

Website: https://conda.org
Twitter: condaproject
Repositories: 29
Profile: https://github.com/conda

conda is system-level, binary package and environment manager running on all major operating systems and platforms.

GitHub Events

Total

Create event: 18
Release event: 1
Issues event: 5
Watch event: 2
Delete event: 14
Issue comment event: 11
Push event: 38
Pull request review comment event: 16
Pull request review event: 32
Pull request event: 43
Fork event: 3

Last Year

Create event: 18
Release event: 1
Issues event: 5
Watch event: 2
Delete event: 14
Issue comment event: 11
Push event: 38
Pull request review comment event: 16
Pull request review event: 32
Pull request event: 43
Fork event: 3

Committers

Last synced: about 1 year ago

All Time

Total Commits: 106
Total Committers: 10
Avg Commits per committer: 10.6
Development Distribution Score (DDS): 0.509

Past Year

Commits: 37
Committers: 7
Avg Commits per committer: 5.286
Development Distribution Score (DDS): 0.649

Top Committers

Name	Email	Commits
Daniel Holth	d**h@a**m	52
conda-bot	1****t	20
pre-commit-ci[bot]	6****]	17
Daniel Bast	2****t	4
Anaconda Renovate Bot	a**t@a**m	4
dependabot[bot]	4****]	3
Jannis Leidel	j**s@l**o	2
Yannik Tausch	y**h@q**m	2
Orion Poplawski	o**n@n**m	1
Ken Odegard	k**d@a**m	1

Committer Domains (Top 20 + Academic)

anaconda.com: 3 nwra.com: 1 quantco.com: 1 leidel.info: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time

Total issues: 42
Total pull requests: 162
Average time to close issues: 4 months
Average time to close pull requests: 26 days
Total issue authors: 9
Total pull request authors: 13
Average comments per issue: 1.31
Average comments per pull request: 0.4
Merged pull requests: 118
Bot issues: 0
Bot pull requests: 45

Past Year

Issues: 5
Pull requests: 69
Average time to close issues: 10 days
Average time to close pull requests: 11 days
Issue authors: 2
Pull request authors: 7
Average comments per issue: 0.0
Average comments per pull request: 0.28
Merged pull requests: 48
Bot issues: 0
Bot pull requests: 28

View more stats

Top Authors

Issue Authors

dholth (33)
jaimergp (2)
kenodegard (1)
bernt-matthias (1)
anaconda-renovate-bot (1)
marcoesters (1)
asford (1)
dbast (1)
remkade (1)

Pull Request Authors

dholth (47)
conda-bot (46)
pre-commit-ci[bot] (32)
dependabot[bot] (13)
anaconda-renovate-bot (5)
ytausch (4)
jezdez (3)
marcoesters (3)
kenodegard (2)
scw (2)
dbast (2)
msarahan (2)
opoplawski (1)

Top Labels

Issue Labels

locked (26) stale (4) stale::closed (4) vote (1) sprint (1)

Pull Request Labels

cla-signed (99) locked (48) dependencies (12) github_actions (12) renovate (5) enhancement (2) stale (2) stale::closed (2)

Packages

Total packages: 3
Total downloads:
- pypi 54,494 last-month
Total docker downloads: 502,776,845

Total dependent packages: 2
(may contain duplicates)
Total dependent repositories: 36
(may contain duplicates)
Total versions: 20
Total maintainers: 1

pypi.org: conda_package_streaming

An efficient library to read from new and old format .conda and .tar.bz2 conda packages.

Homepage: https://github.com/conda/conda-package-streaming
Documentation: https://conda.github.io/conda-package-streaming/
License: BSD License
Latest release: 0.12.0
published about 1 year ago

Versions: 11
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 54,494 Last month
Docker Downloads: 502,776,845

Rankings

Downloads: 1.8%

Dependent packages count: 10.2%

Average: 23.2%

Dependent repos count: 57.5%

Maintainers (1)

joeforker

Last synced: 10 months ago

anaconda.org: conda-package-streaming

An efficient library to read from new and old format .conda and .tar.bz2 conda packages.

Homepage: https://github.com/conda/conda-package-streaming
License: BSD-3-Clause
Latest release: 0.12.0
published about 1 year ago

Versions: 6
Dependent Packages: 2
Dependent Repositories: 18

Rankings

Dependent packages count: 20.4%

Dependent repos count: 33.0%

Average: 41.8%

Stargazers count: 56.1%

Forks count: 57.5%

Last synced: 10 months ago

conda-forge.org: conda-package-streaming

Homepage: https://github.com/conda/conda-package-streaming
License: BSD-3-Clause
Latest release: 0.6.0
published over 3 years ago

Versions: 3
Dependent Packages: 0
Dependent Repositories: 18

Rankings

Dependent repos count: 8.4%

Average: 44.4%

Dependent packages count: 51.6%

Stargazers count: 58.5%

Forks count: 59.3%

Last synced: 10 months ago

Dependencies

.github/workflows/sphinx.yml actions

actions/checkout v3 composite
actions/deploy-pages v1 composite
actions/setup-python v4 composite
actions/upload-pages-artifact v1 composite

pyproject.toml pypi

requests *
zstandard >=0.15

requirements.txt pypi

boto3 *
boto3-stubs *
bottle *
furo *
mdit-py-plugins >=0.3.0
myst-parser *
pytest >=7
pytest-cov *
pytest-mock *
requests *
sphinx *
zstandard >=0.15

https://github.com/conda/conda-package-streaming

Science Score: 26.0%

Keywords from Contributors

Repository

Basic Info

Statistics

Metadata Files

README.md

conda-package-streaming

Quickstart

url = (ends with .conda or .tar.bz2)

key = (ends with .conda or .tar.bz2)

filename = (ends with .conda or .tar.bz2)

file object must be seekable for .conda format, but merely readable for .tar.bz2

Package goals

Generating documentation

Owner

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: conda_package_streaming

Rankings

Maintainers (1)

anaconda.org: conda-package-streaming

Rankings

conda-forge.org: conda-package-streaming

Rankings

Dependencies

file object must be seekable for `.conda` format, but merely readable for `.tar.bz2`