Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of Pooch

Pooch - v1.8.2

Released on: 2024/06/06

DOI: https://doi.org/10.5281/zenodo.11493461

Bug fixes:

  • Use a variable to set the default request timeout (#418)

Documentation:

  • Add HyperSpy, RosettaSciIO, eXSpy to projects using pooch (#408)
  • Add more packages using Pooch (#403)

Maintenance:

  • Add optional dependencies to environment.yml (#413)
  • Run tests with oldest dependencies on x86 macos (#414)
  • Mark additional tests requiring network (#412)
  • Fix package description in pyproject.toml (#407)
  • Setup Trusted Publisher deployment to PyPI (#406)
  • Use Burocrata to check and add license notices (#402)
  • Use pyproject.toml instead of setup.cfg (#401)

This release contains contributions from:

  • Sandro
  • Jonas Lähnemann
  • Santiago Soler
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda almost 2 years ago

Pooch - v1.8.1

Released on: 2024/02/19

DOI: https://doi.org/10.5281/zenodo.10680982

Bug fixes:

  • Use the ID instead of persistentID for Dataverse downloads since some repositories don\'t issue persistentIDs but all issue normal IDs (#355)
  • Ensure all archive members are unpacked in subsequent uses of Untar/Unzip if the first call only asked for a few members (#365)

Documentation:

  • Move \"Projects using Pooch\" further up the README (#386)
  • Update the versions of sphinx and its plugins (#385)

Maintenance:

  • Remove many deprecated pylint options (#329)
  • Use Dependabot to manage GitHub Actions (#387)
  • Simplify the test GitHub Actions workflow (#384)
  • Update format for Black 24.1.1 (#383)

This release contains contributions from:

  • Mark Harfouche
  • Juan Nunez-Iglesias
  • Santiago Soler
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda over 2 years ago

Pooch - v1.8.0

Released on: 2023/10/24

doi: 10.5281/zenodo.10037888

Bug fixes:

  • Fix bug: add support for old and new Zenodo APIs (#375)

New features:

  • Only create local data directories if necessary (#370)
  • Speed up import time by lazy loading requests (#328)

Maintenance:

  • Add support for Python 3.11 (#348)
  • Only run CI cron job for the upstream repository (#361)

Documentation:

  • Add GemGIS to list of projects using Pooch (#349)
  • Fix spelling of Dataverse (#353)
  • Fix grammar on retrieve documentation (#359)

This release contains contributions from:

  • Hugo van Kemenade
  • AlexanderJuestel
  • Mark Harfouche
  • Philip Durbin
  • Rob Luke
  • Santiago Soler
  • Stephan Hoyer

Scientific Software - Peer-reviewed - Python
Published by santisoler over 2 years ago

Pooch - v1.7.0

Released on: 2023/02/27

doi: 10.5281/zenodo.7678844

Bug fixes:

  • Make archive extraction always take members into account (#316)
  • Figshare downloaders fetch the correct version, instead of always the latest one. (#343)

New features:

  • Allow spaces in filenames in registry files (#315)
  • Refactor Pooch.is_available to use downloaders (#322)
  • Add support for downloading files from Dataverse DOIs (#318)
  • Add a new Pooch.load_registry_from_doi method that populates the Pooch registry using DOI-based data repositories (#325)
  • Support urls for Zenodo repositories created through the GitHub integration service, which include slashes in the filename of the main zip files (#340)
  • Automatically add a trailing slash to base_url on pooch.create (#344)

Maintenance:

  • Drop support for Python 3.6 (#299)
  • Port from deprecated appdirs to platformdirs (#339)
  • Update version of Codecov\'s Action to v3 (#345)

Documentation:

  • Update sphinx, theme, and sphinx-panels (#300)
  • Add CITATION.cff for the JOSS article (#308)
  • Use Markdown for the README (#311)
  • Improve docstring of known_hash in retrieve function (#333)
  • Replace link to Pooch\'s citation with a BibTeX code snippet (#335)

Projects that started using Pooch:

This release contains contributions from:

  • Alex Fikl
  • Anirudh Dagar
  • Björn Ludwig
  • Brian Rose
  • Dominic Kempf
  • Florian Wellmann
  • Gabriel Fu
  • Kyle I S Harrington
  • Leonardo Uieda
  • myd7349
  • Rowan Cockett
  • Santiago Soler

Scientific Software - Peer-reviewed - Python
Published by santisoler about 3 years ago

Pooch - v1.6.0

Released on: 2022/01/24

doi: 10.5281/zenodo.5793074

Important:

  • This is the last release with support for Python 3.6. The next major/minor release of Pooch will require Python >= 3.7.
  • Pooch now specifies version bounds for our required dependencies and a plan for dropping support for older versions. Please revise it if you depend on Pooch.

Enhancements:

  • Add option to disable updates on hash mismatch (#291 and #292)
  • Allow enabling progress bars with an argument in Pooch.fetch and retrieve (#277)

Documentation:

Maintenance:

  • Replace Google Analytics for Plausible to make our docs more privacy-friendly (#293)
  • Use Dependente to capture dependencies on CI (#289)
  • Use build instead of setup.py (#287)
  • Run the tests weekly on GitHub Actions (#286)
  • Set minimum required version of dependencies (#280)
  • Rename \"master\" to \"main\" throughout the project (#278)
  • Remove trailing slash from GitHub handle in AUTHORS.md (#279)

This release contains contributions from:

  • Santiago Soler
  • Genevieve Buckley
  • Ryan Abernathey
  • Ryan May
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda over 4 years ago

Pooch - v1.5.2

Released on: 2021/10/11

doi:10.5281/zenodo.5560923

Bug fixes:

  • Fix bug when unpacking an entire subfolder from an archive. Now both unpacking processors (Untar and Unzip) handle members that are folders (not files) correctly. (#266)

Enhancements:

  • Add support for Python 3.10 (#260)
  • Point to the user\'s code for the file_hash warning instead of our internal code (which isn\'t very useful) (#259)

Documentation:

  • Fix typo in a variable name of the examples in the documentation (#268)
  • Fix typo when specifying the SFTP protocol in the about page (#267)

Maintenance:

  • Remove old testing checks if running on TravisCI (#265)

This release contains contributions from:

  • Santiago Soler
  • Hugo van Kemenade
  • Mark Harfouche
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda over 4 years ago

Pooch - v1.5.1

Released on: 2021/08/24

doi:10.5281/zenodo.5242882

WARNING: Please use from pooch import file_hash instead of from pooch.utils import file_hash. This is backwards compatible with all previous versions of Pooch. We recommend importing all functions and classes from the top-level namespace.

Bug fixes:

  • Make file_hash accessible from the pooch.utils module again. Moving this function to pooch.hashes caused crashes downstream. To prevent these crashes, add a wrapper back to utils that issues a warning that users should import from the top-level namespace instead. (#257)
  • Use a mirror of the test data directory in tests that write to it. (#255)
  • Add a pytest mark for tests accessing the network so that they can easily excluded when testing offline. (#254)

This release contains contributions from:

  • Antonio Valentino
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda almost 5 years ago

Pooch - v1.5.0

Released on: 2021/08/23

Digital Object Identifier for the Zenodo archive

New features:

  • Add support for non-cryptographic hashes from the xxhash package. They aren\'t as safe (but safe enough) and compute in fractions of the time from SHA or MD5. This makes it feasible to use hash checking on large datasets. (#242)
  • Add support for using figshare and Zenodo DOIs as URLs (with the protocol doi:{DOI}/{file name}, which works out-of-the-box with Pooch.fetch and retrieve). Can only download 1 file from the archive (not the full archive) and the file name must be specified in the URL. (#241)

Maintenance:

  • Move hash functions to their own private module. No changes to the public API. (#244)
  • Run CI jobs on Python version extremes instead of all supported versions (#243)

This release contains contributions from:

  • Mark Harfouche
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda almost 5 years ago

Pooch - v1.4.0

Released on: 2021/06/08

Digital Object Identifier for the Zenodo archive

Bug fixes:

  • Fix bug in Untar and Unzip when the archive contains subfolders (#224)

Documentation:

  • New theme (sphinx-book-theme) and layout of the documentation (#236 #237 #238)

Enhancements:

  • Add support for non-tqdm progress bars on HTTPDownloader (#228)
  • Allow custom unpack locations in Untar and Unzip (#224)

Maintenance:

  • Replace versioneer with setuptools-scm (#235)
  • Automatically check license notice on code files (#231)
  • Don't store documentation HTML as CI build artifacts (#221)

This release contains contributions from:

  • Leonardo Uieda
  • Agustina Pesce
  • Clément Robert
  • Daniel McCloy

Scientific Software - Peer-reviewed - Python
Published by leouieda almost 5 years ago

Pooch - v1.3.0

Released on: 2020/11/27

Digital Object Identifier for the Zenodo archive

Bug fixes:

  • Properly handle capitalized hashes. On Windows, users might sometimes get capitalized hashes from the system. To avoid false hash mismatches, convert stored and computed hashes to lowercase before doing comparisons. Convert hashes to lowercase when reading from the registry to make sure stored hashes are always lowercase. (#214)

New features:

  • Add option to retry downloads if they fail. The new retry_if_failed option to pooch.create and pooch.Pooch allows retrying the download the specified number of times in case of failures due to hash mismatches (coming from Pooch) or network issues (coming from requests). This is useful for running downloads on CI that tend to fail sporadically. Waits a period of time between consecutive downloads starting with 1s and increasing up to 10s in 1s increments. (#215)
  • Allow user defined decompressed file names. Introduce new name argument to pooch.Decompress to allow user defined file names. Defaults to the previous naming convention for backward compatibility. (#203)

Documentation:

  • Add seaborn-image to list of packages using Pooch (#218)

Maintenance:

  • Add support for Python 3.9. (#220)
  • Drop support for Python 3.5. (#204)
  • Use pip instead of conda to speed up Actions (#216)
  • Add license and copyright notice to every .py file (#213)

This release contains contributions from:

  • Leonardo Uieda
  • Danilo Horta
  • Hugo van Kemenade
  • SarthakJariwala

Scientific Software - Peer-reviewed - Python
Published by leouieda over 5 years ago

Pooch - v1.2.0

Released on: 2020/09/10

Digital Object Identifier for the Zenodo archive

Bug fixes:

  • Fix FTP availability check when the file is in a directory. If the data file is not in the base directory, the Pooch.is_available test was broken since we were checking for the full path in ftp.nlst instead of just the file name. (#191)

New features:

  • Add the SFTPDownloader class for secure FTP downloads (#165)
  • Expose Pooch version as pooch.__version__ (#179)
  • Allow line comments in registry files with # (#180)

Enhancements:

  • Point to Unzip/tar from Decompress docs and errors (#200)

Documentation:

  • Re-factor the documentation into separate pages (#202)
  • Add warning to the docs about dropping Python 3.5 (#201)
  • Add histolab to the Pooch-powered projects (#189)

Maintenance:

  • Push documentation to GitHub Pages using Actions (#198)
  • Add GitHub Actions workflow for publishing to PyPI (#196)
  • Set up GitHub Actions for testing and linting (#194)
  • Test FTP downloads using a local test server (#192)

This release contains contributions from:

  • Leonardo Uieda
  • Hugo van Kemenade
  • Alessia Marcolini
  • Luke Gregor
  • Mathias Hauser

Scientific Software - Peer-reviewed - Python
Published by leouieda over 5 years ago

Pooch - v1.1.1

Bug fixes:

  • Delay data cache folder creation until the first download is attempted. As seen in recent issues in scikit-image, creating the data folder in pooch.create can cause problems since this function is called at import time. This means that importing the package in parallel can cause race conditions and crashes. To prevent that from happening, delay the creation of the cache folder until Pooch.fetch or retrieve are called. (#173)
  • Allow the data folder to already exist when creating it. This is can help cope with parallel execution as well. (#171)

Documentation:

  • Added scikit-image to list of Pooch users. (#168)
  • Fix typo in README and front page contributing section. (#166)

This release contains contributions from:

  • Leonardo Uieda
  • Egor Panfilov
  • Rowan Cockett

Scientific Software - Peer-reviewed - Python
Published by leouieda about 6 years ago

Pooch - v1.1.0

Digital Object Identifier for the Zenodo archive

New features:

  • New function pooch.retrieve to fetch single files This is much more convenient than setting up a Pooch while retaining the hash checks and use of downloaders and processors. It automatically selects a unique file name and saves files to a cache folder. (#152)
  • Allow to use of different hashing algorithms (other than SHA256). Optionally specify the hash as alg:hash and allow pooch.Pooch to recognize the algorithm when comparing hashes. Setting an algorithsm is optional and omiting it defaults to SHA256. This is particularly useful when data are coming from external sources and published hashes are already available. (#133)

Documentation:

  • Add example for fetching datasets that change on the server, for which the hash check would always fail. (#144)
  • Fix path examples in docstring of pooch.os_cache. The docstring mentioned the data path as examples instead of the cache path. (#140)
  • Add example of creating a registry when you don\'t have the data files locally and would have to download them manually. The example uses the pooch.retrieve function to automate the process. The example covers two cases: when all remote files share the same base URL and when every file has its own URL. (#161)

Maintenance:

  • A lot of general refactoring of the internals of Pooch to facilitate development of the new pooch.retrieve function (#159 #157 #156 #151 #149)

This release contains contributions from:

  • Leonardo Uieda
  • Santiago Soler
  • Kacper Kowalik
  • Lucas Martin-King
  • Zac Flamig

Scientific Software - Peer-reviewed - Python
Published by leouieda about 6 years ago

Pooch - v1.0.0

DOI: https://doi.org/10.5281/zenodo.3629329

This release marks the stabilization of the Pooch API. Further changes to the 1.* line will be fully backwards compatible (meaning that updating Pooch should not break existing code). If there is great need to make backwards incompatible changes, we will release a 2.* line. In that case, bug fixes will still be ported to the 1.* line for a period of time.

Improvements:

  • Allow blank lines in registry files. Previously, they would cause an error. (#138)

Backwards incompatible changes:

  • Using Python’s logging module to instead of warnings to inform users of download, update, and decompression/unpacking actions. This allows messages to be logged with different priorities and the user filter out log messages or silence Pooch entirely. Introduces the function pooch.get_logger to access the logging object used by Pooch. Users who relied on Pooch issuing warnings will need to update to capturing logs instead. All other parts of the API remain unchanged. (#115)

This release contains contributions from:

  • Daniel Shapero

Scientific Software - Peer-reviewed - Python
Published by leouieda over 6 years ago

Pooch - v0.7.2

🚨 Announcement: 🚨 We now have a JOSS paper about Pooch! Please cite it when you use Pooch for your research. (#116 with reviews in #132 and #134)

This is minor release which only updates the citation information to the new JOSS paper. No DOI was issued for this release since there are no code or documentation changes.

Scientific Software - Peer-reviewed - Python
Published by leouieda over 6 years ago

Pooch - v0.7.1

DOI: https://doi.org/10.5281/zenodo.3611376

Improvements:

  • Better error messages when hashes don't match. Include the file name in the exception for a hash mismatch between a downloaded file and the registry. Before, we included the name of temporary file, which wasn't very informative. (#128)
  • Better error message for malformed registry files. When loading a registry file, inform the name of the file and include the offending content in the error message instead of just the line number. (#129)

Maintenance:

  • Change development status flag in setup.py to "stable" instead of "alpha". (#127)

This release was reviewed at the Journal of Open Source Software. The code and software paper contain contributions from:

  • Anderson Banihirwe
  • Martin Durant
  • Mark Harfouche
  • Hugo van Kemenade
  • John Leeman
  • Rémi Rampin
  • Daniel Shapero
  • Santiago Rubén Soler
  • Matthew Turk
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda over 6 years ago

Pooch - v0.7.0

DOI: https://doi.org/10.5281/zenodo.3547640

New features:

  • New pooch.FTPDownloader class for downloading files over FTP. Uses the standard library ftplib. The appropriate downloader is automatically selected by pooch.Pooch.fetch based on the URL (for anonymous FTP only), so no configuration is required. If authentication is required, pooch.FTPDownloader provides the need support. Ported from NCAR/aletheia-data by the author. (#118)
  • Support for file-like objects to Pooch.load_registry (opened either in binary or text mode). (#117)

Maintenance:

  • Testing and official support for Python 3.8. (#113)
  • 🚨 Drop support for Python 2.7. 🚨 Remove conditional dependencies and CI jobs. (#100)

Documentation:

  • In the tutorial, use pkgresources.resourcestream() from setuptools to load the registry.txt file. It’s less error-prone than using os.path and __file__ and allows the package to work from zip files. (#120)
  • Docstrings formatted to 79 characters (instead of 88) for better rendering in Jupyter notebooks and IPython. These displays are limited to 80 chars so the longer lines made the docstring unreadable. (#123)

This release contains contributions from:

  • Anderson Banihirwe
  • Hugo van Kemenade
  • Remi Rampin
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda over 6 years ago

Pooch - v0.6.0

🚨 Pooch v0.6.0 is the last release to support Python 2.7 🚨

DOI: https://doi.org/10.5281/zenodo.3515031

New features:

  • Add optional download progress bar to pooch.HTTPDownloader (#97)

Maintenance:

  • Warn that 0.6.0 is the last version to support Python 2.7 (#108)

Documentation:

  • Update contact information to point to our Slack channel (#107)
  • Add icepack to list of projects using Pooch (#98)

This release contains contributions from:

  • Daniel Shapero
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda over 6 years ago

Pooch - v0.5.2

Maintenance:

  • Add back support for Python 3.5 with continuous integration tests. No code changes were needed, only removing the restriction from setup.py. (#93)

This release contains contributions from:

  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda almost 7 years ago

Pooch - v0.5.1

Documentation fixes:

  • Fix formatting error in pooch.Decompress docstring. (#81)
  • Fix wrong imports in the usage guide for post-processing hooks. (#84)
  • Add section to the usage guide explaining when to use pooch.Decompress. (#85)

This release contains contributions from:

  • Santiago Soler
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda about 7 years ago

Pooch - v0.5.0

New features:

  • New processor pooch.Decompress saves a decompressed version of the downloaded file. Supports gzip, lzma/xz, and bzip2 compression. Note: Under Python 2.7, lzma and bzip2 require the backports.lzma and bz2file packages as well. These are soft dependencies and not required to use Pooch. See Installing. (#78)
  • New processor pooch.Untar unpacks files contained in a downloaded tar archive (with or without compression). (#77)

This release contains contributions from:

  • Matthew Turk
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda about 7 years ago

Pooch - v0.4.0

New features:

  • Add customizable downloaders. Delegate file download into separate classes that can be passed to Pooch.fetch. Created the HTTPDownloader class (used by default) which can also be used to download files that require authentication/login. (#66)
  • Add post-download processor hooks to Pooch.fetch. Allows users to pass in a function that is executed right before returning and can overwrite the file path that is returned by fetch. Use this, for example, to perform unpacking/decompression operations on larger files that can be time consuming and we only want to do once. (#59)
  • Add the Unzip post-download processor to extract files from a downloaded zip archive. Unpacks files into a directory in the local store and returns a list of all unzipped files. (#72)
  • Make the check_version function public. It’s used internally but will be useful in examples that want to download things from the pooch repository. (#69)

Maintenance:

  • Pin sphinx to version 1.8.5. New versions of Sphinx (2.0.*) are messing up the numpydoc style docstrings. (#64)

This release contains contributions from:

  • Santiago Soler
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda about 7 years ago

Pooch - v0.3.1

Minor patches:

  • Add a project logo (#57)
  • Replace http with https in the README.rst to avoid mixed content warnings in some browsers (#56)

Scientific Software - Peer-reviewed - Python
Published by leouieda about 7 years ago

Pooch - v0.3.0

New features:

  • Use the appdirs library to get the cache directory. Could change the default data location on all platforms. Locations are compatible with the XDG Base Directory Specification (#45)
  • Add method Pooch.is_available to check remote file availability (#50)
  • Add Pooch.registry_files property to get a name of all files in the registry (#42)
  • Make Pooch.get_url a public method to get the download URL for a given file (#55)

Maintenance:

  • Drop support for Python 3.5. Pooch now requires Python >= 3.6. (#52)
  • Add a private method to check if a file is in the registry (#49)
  • Fix typo in the Pooch.load_registry docstring (#41)

This release contains contributions from:

  • Santiago Soler
  • Rémi Rampin
  • Leonardo Uieda

Scientific Software - Peer-reviewed - Python
Published by leouieda about 7 years ago

Pooch - v0.2.1

Bug fixes:

  • Fix unwanted ~ directory creation when not using a version in pooch.create (#37)

Scientific Software - Peer-reviewed - Python
Published by leouieda over 7 years ago

Pooch - v0.2.0

Bug fixes:

  • Avoid copying of files across the file system (#33)
  • Correctly delete temporary downloads on error (#32)

New features:

  • Allow custom download URLs for individual files (#30)
  • Allow dataset versioning to be optional (#29)

Maintenance:

  • Move URLs building to a dedicated method for easy subclassing (#31)
  • Add testing and support for Python 3.7 (#25)

Scientific Software - Peer-reviewed - Python
Published by leouieda over 7 years ago

Pooch - v0.1.1

Bug fixes:

  • Check if the local data folder is writable and warn the user instead of crashing (#23)

Scientific Software - Peer-reviewed - Python
Published by leouieda over 7 years ago

Pooch - v0.1

First release of Pooch :tada:

Pooch will manage downloading your sample data files over HTTP from a server and storing them in a local directory:

  • Download a file only if it's not in the local storage.
  • Check the SHA256 hash to make sure the file is not corrupted or needs updating.
  • If the hash is different from the registry, Pooch will download a new version of the file.
  • If the hash still doesn't match, Pooch will raise an exception warning of possible data corruption.

Scientific Software - Peer-reviewed - Python
Published by leouieda almost 8 years ago