Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/fanchengyan/data-downloader

Make downloading scientific data much easier

https://github.com/fanchengyan/data-downloader

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 3 committers (33.3%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (15.0%) to scientific vocabulary

Keywords

downloader hyp3 licsar sentinel-1
Last synced: 6 months ago · JSON representation

Repository

Make downloading scientific data much easier

Basic Info
Statistics
  • Stars: 11
  • Watchers: 1
  • Forks: 5
  • Open Issues: 1
  • Releases: 7
Topics
downloader hyp3 licsar sentinel-1
Created almost 6 years ago · Last pushed 7 months ago
Metadata Files
Readme License

README.md


Downloads PyPI Documentation Status

Make downloading scientific data much easier

Introduction

DataDownloader is a user-friendly package for downloading files using HTTP/HTTPS. It currently includes a downloader module for downloading files, a parse_urls module for parsing URLs, and a services module for managing well-known online services.

Highlight Features

DataDownloader has several features to make retrieving files easy, including:

  • Resumable: You can resume aborted downloads automatically when you re-execute the code if website support resuming (status code is 216 or 416 when send a HEAD request to the server supplying a Range header)
  • Asynchronous: Can download multiple files at the same time when download a single file very slow.
  • Convenient: Provide a easy way to manage your username and password and parse urls from different sources:
    • netrc: Provide a convenient way to manage your username and password via .netrc file, avoiding providing your login information over and over again.
    • parse_urls: Provide various methods to parse urls from different sources. See sections :ref:parse_urls for more details
    • services: Provide a convenient way to manage well-known online services, currently support: HyP3, LiCSAR, Sentinel-1 orbit.

Installation

You can install DataDownloader via pip from PyPI:

bash pip install data_downloader

or you can install the latest version from GitHub:

bash pip install git+hhttps://github.com/Fanchengyan/data-downloader.git

Usage

The detailed documentation is available at: https://data-downloader.readthedocs.io/en/latest/.

[!TIP] We provide some examples to help you get started quickly : https://data-downloader.readthedocs.io/en/latest/Tutorials/index.html

Owner

  • Name: Fanchengyan
  • Login: Fanchengyan
  • Kind: user

GitHub Events

Total
  • Watch event: 1
  • Push event: 14
  • Pull request event: 5
  • Fork event: 1
  • Create event: 1
Last Year
  • Watch event: 1
  • Push event: 14
  • Pull request event: 5
  • Fork event: 1
  • Create event: 1

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 147
  • Total Committers: 3
  • Avg Commits per committer: 49.0
  • Development Distribution Score (DDS): 0.184
Past Year
  • Commits: 2
  • Committers: 1
  • Avg Commits per committer: 2.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
fanchengyan f****4@l****n 120
fanchengyan 1****8@q****m 26
MrNightSky 3****y@u****m 1
Committer Domains (Top 20 + Academic)
qq.com: 1 lzu.edu.cn: 1

Issues and Pull Requests

Last synced: 7 months ago

All Time
  • Total issues: 0
  • Total pull requests: 4
  • Average time to close issues: N/A
  • Average time to close pull requests: 5 days
  • Total issue authors: 0
  • Total pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 3
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 3
  • Average time to close issues: N/A
  • Average time to close pull requests: about 3 hours
  • Issue authors: 0
  • Pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • Fanchengyan (3)
  • MrNightSky (1)
Top Labels
Issue Labels
Pull Request Labels

Dependencies

setup.py pypi
  • beautifulsoup4 *
  • browser-cookie3 *
  • httpx *
  • nest_asyncio *
  • python-dateutil *
  • setuptools *
  • tqdm *