Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/conda/conda-index

conda index, formerly part of conda-build. Create channels from collections of packages.

https://github.com/conda/conda-index

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.8%) to scientific vocabulary

Keywords from Contributors

conda package-management docs
Last synced: 10 months ago · JSON representation

Repository

conda index, formerly part of conda-build. Create channels from collections of packages.

Basic Info
Statistics
  • Stars: 12
  • Watchers: 8
  • Forks: 9
  • Open Issues: 10
  • Releases: 8
Created about 4 years ago · Last pushed about 1 year ago
Metadata Files
Readme Changelog License Code of conduct Authors

README.md

conda-index

conda index, formerly part of conda-build. Create repodata.json for collections of conda packages.

The conda_index command operates on a channel directory. A channel directory contains a noarch subdirectory at a minimum and will almost always contain other subdirectories named for conda's supported platforms linux-64, win-64, osx-64, etc. A channel directory cannot have the same name as a supported platform. Place packages into the same platform subdirectory each archive was built for. Conda-index extracts metadata from these packages to generate index.html, repodata.json etc. with summaries of the packages' metadata. Then conda uses the metadata to solve dependencies before doing an install.

By default, the metadata is output to the same directory tree as the channel directory, but it can be output to a separate tree with the --output <output> parameter. The metadata cache is always placed with the packages, in .cache folders under each platform subdirectory.

After conda-index has finished, its output can be used as a channel conda install -c file:///path/to/output ... or it would typically be placed on a web server.

Run normally

sh python -m conda_index <path to channel directory>

Note conda index (instead of python -m conda_index) may find legacy conda-build index.

Run for debugging

sh python -m conda_index --verbose --threads=1 <path to channel directory>

Contributing

```sh conda create -n conda-index "python >=3.9" conda conda-build "pip >=22"

git clone https://github.com/conda/conda-index.git pip install -e conda-index[test]

cd conda-index pytest ```

Summary of changes from the previous conda-build index version

  • Approximately 2.2x faster conda package extraction, by extracting just the metadata to streams instead of extracting packages to a temporary directory; closes the package early if all metadata has been found.

  • No longer read existing repodata.json. Always load from cache.

  • Uses a sqlite metadata cache that is orders of magnitude faster than the old many-tiny-files cache.

  • The first time conda index runs, it will convert the existing file-based .cache to a sqlite3 database .cache/cache.db. This takes about ten minutes per subdir for conda-forge. (If this is interrupted, delete cache.db to start over, or packages will be re-extracted into the cache.) sqlite3 must be compiled with the JSON1 extension. JSON1 is built into SQLite by default as of SQLite version 3.38.0 (2022-02-22).

  • Each subdir osx-64, linux-64 etc. has its own cache.db; conda-forge’s 1.2T osx-64 subdir has a single 2.4GB cache.db. Storing the cache in fewer files saves time since there is a per-file wait to open each of the many tiny .json files in old-style .cache/.

  • cache.db is highly compressible, like the text metadata. 2.4G → zstd → 88M

  • No longer cache paths.json (only used to create post_install.json and not referenced later in the indexing process). Saves 90% disk space in .cache.

  • Updated Python and dependency requirements.

  • Mercilessly cull less-used features.

  • Format with black

Parallelism

This version of conda-index continues indexing packages from other subdirs while the main thread is writing a repodata.json.

All current_repodata.json are generated in parallel. This may use a lot of ram if repodata.json has tens of thousands of entries.

Owner

  • Name: conda
  • Login: conda
  • Kind: organization

conda is system-level, binary package and environment manager running on all major operating systems and platforms.

GitHub Events

Total
  • Issues event: 20
  • Watch event: 2
  • Delete event: 5
  • Issue comment event: 23
  • Push event: 54
  • Pull request review comment event: 36
  • Pull request review event: 39
  • Pull request event: 18
  • Create event: 9
Last Year
  • Issues event: 20
  • Watch event: 2
  • Delete event: 5
  • Issue comment event: 23
  • Push event: 54
  • Pull request review comment event: 36
  • Pull request review event: 39
  • Pull request event: 18
  • Create event: 9

Committers

Last synced: over 1 year ago

All Time
  • Total Commits: 130
  • Total Committers: 8
  • Avg Commits per committer: 16.25
  • Development Distribution Score (DDS): 0.331
Past Year
  • Commits: 38
  • Committers: 5
  • Avg Commits per committer: 7.6
  • Development Distribution Score (DDS): 0.474
Top Committers
Name Email Commits
Daniel Holth d****h@a****m 87
conda-bot 1****t 33
Jannis Leidel j****s@l****o 5
jaimergp j****p 1
Klaus Zimmermann k****n@q****m 1
Ken Odegard k****d@a****m 1
Carl Anderson b****o 1
Andrew Vallette a****e@g****m 1
Committer Domains (Top 20 + Academic)
anaconda.com: 2 quansight.com: 1 leidel.info: 1

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 84
  • Total pull requests: 131
  • Average time to close issues: 4 months
  • Average time to close pull requests: 20 days
  • Total issue authors: 13
  • Total pull request authors: 7
  • Average comments per issue: 1.6
  • Average comments per pull request: 0.5
  • Merged pull requests: 117
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 12
  • Pull requests: 31
  • Average time to close issues: about 2 months
  • Average time to close pull requests: 11 days
  • Issue authors: 5
  • Pull request authors: 5
  • Average comments per issue: 0.42
  • Average comments per pull request: 0.35
  • Merged pull requests: 25
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • dholth (69)
  • jezdez (3)
  • kenodegard (2)
  • barabo (1)
  • jaimergp (1)
  • scdub (1)
  • JeanChristopheMorinPerso (1)
  • ialb97 (1)
  • meesepyter (1)
  • opoplawski (1)
  • clement-chaneching (1)
  • jpmaterial (1)
Pull Request Authors
  • dholth (99)
  • conda-bot (51)
  • jezdez (11)
  • barabo (5)
  • jaimergp (2)
  • jtroe (2)
  • kenodegard (1)
  • zklaus (1)
  • AndrewVallette (1)
Top Labels
Issue Labels
locked (31) type::bug (25) type::feature (18) stale (9) source::anaconda (8) stale::closed (8) in-progress (4) severity::2 (2) type::documentation (2) type::task (2) good first issue (1) backlog (1) pending::discussion (1) source::community (1)
Pull Request Labels
cla-signed (131) locked (47) in-progress (8) stale (3) stale::closed (2) stale::recovered (1)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 940 last-month
  • Total dependent packages: 1
    (may contain duplicates)
  • Total dependent repositories: 2
    (may contain duplicates)
  • Total versions: 8
pypi.org: conda-index
  • Homepage: https://github.com/conda/conda-index
  • Documentation: https://conda-index.readthedocs.io/
  • License: BSD 3-Clause License Copyright (c) 2022, conda All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  • Latest release: 0.6.1
    published about 1 year ago
  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 940 Last month
Rankings
Dependent packages count: 9.1%
Average: 30.2%
Dependent repos count: 51.3%
Last synced: 11 months ago
anaconda.org: conda-index

Create repodata.json for collections of conda packages.

  • Versions: 6
  • Dependent Packages: 1
  • Dependent Repositories: 2
Rankings
Dependent packages count: 41.0%
Dependent repos count: 48.4%
Average: 50.1%
Forks count: 55.4%
Stargazers count: 55.5%
Last synced: 11 months ago

Dependencies

.github/workflows/cla.yml actions
  • conda/actions/check-cla v22.9.0 composite
.github/workflows/issues.yml actions
  • actions-ecosystem/action-add-labels v1.1.0 composite
  • actions-ecosystem/action-remove-labels v1.3.0 composite
.github/workflows/labels.yml actions
  • EndBug/label-sync v2.3.0 composite
  • actions/checkout v3 composite
  • andstor/file-existence-action v1.0.1 composite
.github/workflows/project.yml actions
  • actions/add-to-project v0.3.0 composite
.github/workflows/sphinx.yml actions
  • actions/checkout v3 composite
  • actions/deploy-pages v1 composite
  • actions/setup-python v4 composite
  • actions/upload-pages-artifact v1 composite
.github/workflows/stale.yml actions
  • actions/stale v4 composite
  • conda/actions/read-yaml v22.9.0 composite
.github/workflows/tests.yml actions
  • actions/checkout v2 composite
  • actions/checkout v3 composite
  • actions/download-artifact v3 composite
  • actions/upload-artifact v3 composite
  • conda-incubator/setup-miniconda v2 composite
  • conda/actions/canary-release v22.10.0 composite
  • re-actors/alls-green release/v1 composite
  • test-summary/action v2 composite
docs/requirements.txt pypi
  • furo *
  • mdit-py-plugins >=0.3.0
  • myst-parser *
  • sphinx *
pyproject.toml pypi
  • click >=8
  • conda >=4.14.0
  • conda-package-streaming >=0.7.0
  • filelock *
  • jinja2 *
  • more-itertools *
  • ruamel.yaml *
.github/workflows/canary-weekly.yml actions
  • actions/checkout v2 composite
  • actions/download-artifact v3 composite
  • actions/upload-artifact v3 composite
  • conda-incubator/setup-miniconda v2 composite
  • re-actors/alls-green release/v1 composite
  • test-summary/action v2 composite
.github/workflows/lock.yml actions
  • dessant/lock-threads v4 composite