Data

Tools

Indexes

Applications

Experiments

Open Source Science

joblib

Computing with Python functions.

https://github.com/joblib/joblib

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    14 of 146 committers (9.6%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (16.8%) to scientific vocabulary

Keywords

caching memoization multiprocessing parallel-computing python threading

Keywords from Contributors

closember wx tk qt gtk prediction econometrics timeseries-analysis robust-estimation regression-models
Last synced: 6 months ago · JSON representation ·

Repository

Computing with Python functions.

Basic Info
Statistics
  • Stars: 4,192
  • Watchers: 62
  • Forks: 434
  • Open Issues: 418
  • Releases: 0
Topics
caching memoization multiprocessing parallel-computing python threading
Created almost 16 years ago · Last pushed 6 months ago
Metadata Files
Readme Changelog License Citation Security

README.rst 

|PyPi| |CIStatus| |ReadTheDocs| |Codecov|

.. |PyPi| image:: https://badge.fury.io/py/joblib.svg
   :target: https://badge.fury.io/py/joblib
   :alt: Joblib version

.. |CIStatus| image:: https://github.com/joblib/joblib/actions/workflows/test.yml/badge.svg
   :target: https://github.com/joblib/joblib/actions/workflows/test.yml?query=branch%3Amain
   :alt: CI status

.. |ReadTheDocs| image:: https://readthedocs.org/projects/joblib/badge/?version=latest
    :target: https://joblib.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation Status

.. |Codecov| image:: https://codecov.io/gh/joblib/joblib/branch/main/graph/badge.svg
   :target: https://codecov.io/gh/joblib/joblib
   :alt: Codecov coverage


The homepage of joblib with user documentation is located on:

https://joblib.readthedocs.io

Getting the latest code
=======================

To get the latest code using git, simply type::

    git clone https://github.com/joblib/joblib.git

If you don't have git installed, you can download a zip
of the latest code: https://github.com/joblib/joblib/archive/refs/heads/main.zip

Installing
==========

You can use `pip` to install joblib from any directory::

    pip install joblib

or install it in editable mode from the source directory::

    pip install -e .

Dependencies
============

- Joblib has no mandatory dependencies besides Python (supported versions are
  3.9+).
- Joblib has an optional dependency on Numpy (at least version 1.6.1) for array
  manipulation.
- Joblib includes its own vendored copy of
  `loky `_ for process management.
- Joblib can efficiently dump and load numpy arrays but does not require numpy
  to be installed.
- Joblib has an optional dependency on
  `python-lz4 `_ as a faster alternative to
  zlib and gzip for compressed serialization.
- Joblib has an optional dependency on psutil to mitigate memory leaks in
  parallel worker processes.
- Some examples require external dependencies such as pandas. See the
  instructions in the `Building the docs`_ section for details.

Workflow to contribute
======================

To contribute to joblib, first create an account on `github
`_. Once this is done, fork the `joblib repository
`_ to have your own repository,
clone it using ``git clone``. Make your changes in a branch of your clone, push
them to your github account, test them locally, and when you are happy with
them, send a pull request to the main repository.

You can use `pre-commit `_ to run code style checks
before each commit::

    pip install pre-commit
    pre-commit install

pre-commit checks can be disabled for a single commit with::

    git commit -n

Running the test suite
======================

To run the test suite, you need the pytest (version >= 3) and coverage modules.
Run the test suite using::

    pytest joblib

from the root of the project.

Building the docs
=================

To build the docs you need to have sphinx (>=1.4) and some dependencies
installed::

    pip install -U -r .readthedocs-requirements.txt

The docs can then be built with the following command::

    make doc

The html docs are located in the ``doc/_build/html`` directory.


Making a source tarball
=======================

To create a source tarball, eg for packaging or distributing, run the
following command::

    pip install build
    python -m build --sdist

The tarball will be created in the `dist` directory. This command will create
the resulting tarball that can be installed with no extra dependencies than the
Python standard library.

Making a release and uploading it to PyPI
=========================================

This command is only run by project manager, to make a release, and
upload in to PyPI::

    pip install build
    python -m build --sdist --wheel
    twine upload dist/*


Note that the documentation should automatically get updated at each git
push. If that is not the case, try building th doc locally and resolve
any doc build error (in particular when running the examples).

Updating the changelog
======================

Changes are listed in the CHANGES.rst file. They must be manually updated
but, the following git command may be used to generate the lines::

    git log --abbrev-commit --date=short --no-merges --sparse

Owner

  • Name: joblib
  • Login: joblib
  • Kind: organization

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - name: "The joblib developers"
title: "joblib"
version: latest
doi: https://doi.org/10.5281/zenodo.14915601
url: "https://github.com/joblib/joblib"

GitHub Events

Total
  • Create event: 3
  • Commit comment event: 1
  • Release event: 1
  • Issues event: 111
  • Watch event: 311
  • Delete event: 1
  • Issue comment event: 404
  • Push event: 62
  • Pull request event: 123
  • Pull request review event: 140
  • Pull request review comment event: 111
  • Fork event: 26
Last Year
  • Create event: 3
  • Commit comment event: 1
  • Release event: 1
  • Issues event: 111
  • Watch event: 311
  • Delete event: 1
  • Issue comment event: 404
  • Push event: 62
  • Pull request event: 123
  • Pull request review event: 140
  • Pull request review comment event: 111
  • Fork event: 26

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 1,470
  • Total Committers: 146
  • Avg Commits per committer: 10.068
  • Development Distribution Score (DDS): 0.73
Past Year
  • Commits: 61
  • Committers: 19
  • Avg Commits per committer: 3.211
  • Development Distribution Score (DDS): 0.689
Top Committers
Name Email Commits
Gael Varoquaux g****x@n****g 397
Olivier Grisel o****l@e****g 339
Loïc Estève l****e@y****m 231
Alexandre Abadie a****e@i****r 76
Thomas Moreau t****0@g****m 74
Pierre Glaser p****r@m****m 25
Philippe Gervais p****s@i****r 23
Karan Desai k****6@l****m 22
Yaroslav Halchenko d****n@o****m 16
Franck Charras 2****s 15
Pietro Berkes p****s@g****m 10
Valentin Haenel v****l@g****e 10
myint m****t@m****l 10
Guillaume Lemaitre g****8@g****m 9
Lars Buitinck L****k@u****l 9
Adrin Jalali a****i@g****m 8
Gael Varoquaux v****u@r****g 7
Yannick Schwartz y****z@g****m 6
Eric Larson l****d@g****m 5
Jérémie du Boisberranger 3****b 4
Stefanie Senger 9****r 4
Julien Jerphanion g****t@j****z 3
Juan Manuel Caicedo Carvajal j****n@c****m 3
Fabian Pedregosa f****a@i****r 3
Arne Neumann g****b@a****l 3
Andreas Olsson a****s@a****e 3
Kenneth C. Arnold k****d@a****u 3
Matthew Rocklin m****n@g****m 3
Maxime Weyl 3****l 3
Nicolas Hug c****t@n****m 3
and 116 more...
Committer Domains (Top 20 + Academic)
inria.fr: 5 berkeley.edu: 3 crans.org: 2 hackerspace.pl: 1 thomaslevine.com: 1 bluewin.ch: 1 mit.edu: 1 hroncok.cz: 1 rachum.com: 1 fseoane.net: 1 nmr.mgh.harvard.edu: 1 50mail.com: 1 vene.ro: 1 nicolas-hug.com: 1 alum.mit.edu: 1 arrakis.se: 1 arne.cl: 1 cavorite.com: 1 jjerphan.xyz: 1 uva.nl: 1 uw.edu: 1 cs.ubc.ca: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 253
  • Total pull requests: 315
  • Average time to close issues: over 1 year
  • Average time to close pull requests: 4 months
  • Total issue authors: 209
  • Total pull request authors: 93
  • Average comments per issue: 3.11
  • Average comments per pull request: 3.02
  • Merged pull requests: 190
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 59
  • Pull requests: 124
  • Average time to close issues: 19 days
  • Average time to close pull requests: 5 days
  • Issue authors: 48
  • Pull request authors: 21
  • Average comments per issue: 1.41
  • Average comments per pull request: 1.89
  • Merged pull requests: 91
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • lesteve (13)
  • ogrisel (7)
  • larsoner (4)
  • Schefflera-Arboricola (4)
  • jeremiedbb (3)
  • yarikoptic (3)
  • tomMoral (3)
  • winer-lch (2)
  • StefanieSenger (2)
  • holmescharles (2)
  • thiswillbeyourgithub (2)
  • stefan6419846 (2)
  • amueller (2)
  • tinducvo (2)
  • JohannesBuchner (2)
Pull Request Authors
  • tomMoral (56)
  • lesteve (46)
  • fcharras (38)
  • ogrisel (16)
  • adrinjalali (12)
  • StefanieSenger (11)
  • androids-electric-sheep (4)
  • judahrand (4)
  • larsoner (3)
  • japhba (3)
  • andreaso (3)
  • thiswillbeyourgithub (3)
  • jeremiedbb (3)
  • jjerphan (3)
  • jdanbrown (3)
Top Labels
Issue Labels
bug (8) enhancement (4) windows (3) documentation (2) packaging (1) regression (1) dask backend (1) help wanted (1) need reproducer (1)
Pull Request Labels
enhancement (4) need Review (3) documentation (1) bug (1)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 105,596,834 last-month
  • Total docker downloads: 1,420,820,634
  • Total dependent packages: 2,289
    (may contain duplicates)
  • Total dependent repositories: 79,200
    (may contain duplicates)
  • Total versions: 128
  • Total maintainers: 4
  • Total advisories: 1
pypi.org: joblib

Lightweight pipelining with Python functions

  • Versions: 107
  • Dependent Packages: 2,273
  • Dependent Repositories: 77,243
  • Downloads: 105,596,834 Last month
  • Docker Downloads: 1,420,820,634
Rankings
Dependent packages count: 0.0%
Dependent repos count: 0.0%
Downloads: 0.0%
Docker downloads count: 0.0%
Average: 0.9%
Stargazers count: 1.8%
Forks count: 3.3%
Maintainers (4)
tommoral ogrisel GaelVaroquaux lesteve
Advisories (1)
Last synced: 6 months ago
anaconda.org: joblib

"A set of tools to provide lightweight pipelining in Python. In particular: transparent disk-caching of functions and lazy re-evaluation (memoize pattern), easy, simple parallel computing. Joblib is optimized to be fast and robust on large data in particular and has specific optimizations for numpy arrays."

  • Versions: 21
  • Dependent Packages: 16
  • Dependent Repositories: 1,957
Rankings
Dependent repos count: 1.8%
Dependent packages count: 5.1%
Average: 9.6%
Stargazers count: 14.3%
Forks count: 17.1%
Last synced: 6 months ago

Dependencies

.readthedocs-requirements.txt pypi
  • distributed *
  • docutils <0.18
  • lz4 *
  • matplotlib *
  • numpy *
  • numpydoc *
  • pandas *
  • pillow *
  • sphinx *
  • sphinx-gallery *