Data

Tools

Indexes

Applications

Experiments

Open Source Science

datalad-fuse

DataLad extension to provide FUSE file system access

https://github.com/datalad/datalad-fuse

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.0%) to scientific vocabulary

Keywords

datalad

Keywords from Contributors

closember data-storage git-annex usable rdm
Last synced: 10 months ago · JSON representation

Repository

DataLad extension to provide FUSE file system access

Basic Info
  • Host: GitHub
  • Owner: datalad
  • License: other
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 257 KB
Statistics
  • Stars: 2
  • Watchers: 2
  • Forks: 4
  • Open Issues: 22
  • Releases: 9
Topics
datalad
Created over 5 years ago · Last pushed 10 months ago
Metadata Files
Readme Changelog License Zenodo

README.md

DataLad FUSE extension package

codecov.io tests docs

datalad-fuse provides commands for reading files in a DataLad dataset from their remote web URLs without having to download them in their entirety first. Instead, fsspec is used to sparsely download and locally cache the files as needed.

Installation

Current version of datalad-fuse requires Python 3.7 or higher. Just use pip for Python 3 (You have pip, right?) to install it:

python3 -m pip install datalad-fuse

In addition, use of the datalad fusefs command requires FUSE to be installed; on Debian-based systems, this can be done with:

sudo apt-get install fuse

Commands

datalad fsspec-cache-clear [<options>]

Clears the local download cache for a dataset.

Options

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -r, --recursive — Clear the caches of subdatasets as well.

datalad fsspec-head [<options>] <path>

Shows leading lines/bytes of an annexed file by fetching its data from a remote URL.

Options

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -n <INT>, --lines <INT> — How many lines to show (default: 10)

  • -c <INT>, --bytes <INT> — How many bytes to show

datalad fusefs [<options>] <mount-path>

Create a read-only FUSE mount at <mount-path> that exposes the files in the given dataset. Opening a file under the mount that is not locally present in the dataset will cause its contents to be downloaded from the file's web URL as needed.

When the command finishes, fsspec-cache-clear may be run depending on the value of the datalad.fusefs.cache-clear configuration option. If it is set to "visited", then any (sub)datasets that were accessed in the FUSE mount will have their caches cleared; if it is instead set to "recursive", then all (sub)datasets in the dataset being operated on will have their caches cleared.

Options

  • --allow-other — Allow all users to access files in the mount. This requires setting user_allow_other in /etc/fuse.conf.

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -f, --foreground — Run the FUSE process in the foreground; use Ctrl-C to exit. This option is currently required.

  • --mode-transparent — Expose the dataset's .git directory in the mount

Owner

  • Name: DataLad
  • Login: datalad
  • Kind: organization
  • Email: team@datalad.org
  • Location: USA&Germany

Data distribution and management platform

GitHub Events

Total
  • Release event: 1
  • Watch event: 1
  • Delete event: 3
  • Issue comment event: 2
  • Push event: 3
  • Pull request event: 5
  • Create event: 4
Last Year
  • Release event: 1
  • Watch event: 1
  • Delete event: 3
  • Issue comment event: 2
  • Push event: 3
  • Pull request event: 5
  • Create event: 4

Committers

Last synced: over 3 years ago

All Time
  • Total Commits: 151
  • Total Committers: 4
  • Avg Commits per committer: 37.75
  • Development Distribution Score (DDS): 0.192
Top Committers
Name Email Commits
John T. Wodder II g****t@v****g 122
Yaroslav Halchenko d****n@o****m 22
auto a****o@n****l 6
Adina Wagner a****r@t****e 1
Committer Domains (Top 20 + Academic)
t-online.de: 1 onerussian.com: 1 varonathe.org: 1

Issues and Pull Requests

Last synced: 10 months ago

All Time
  • Total issues: 59
  • Total pull requests: 61
  • Average time to close issues: 9 days
  • Average time to close pull requests: 14 days
  • Total issue authors: 4
  • Total pull request authors: 5
  • Average comments per issue: 3.0
  • Average comments per pull request: 3.69
  • Merged pull requests: 58
  • Bot issues: 0
  • Bot pull requests: 4
Past Year
  • Issues: 0
  • Pull requests: 2
  • Average time to close issues: N/A
  • Average time to close pull requests: 3 months
  • Issue authors: 0
  • Pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 0.5
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 1
View more stats
Top Authors
Issue Authors
  • yarikoptic (53)
  • TheChymera (3)
  • jwodder (2)
  • Myrskyta (1)
Pull Request Authors
  • jwodder (48)
  • yarikoptic (15)
  • dependabot[bot] (8)
  • TheChymera (2)
  • adswa (1)
Top Labels
Issue Labels
easy (1) priority-medium (1) blocked (1)
Pull Request Labels
internal (22) tests (14) release (10) patch (9) minor (5) documentation (2) UX (1) performance (1) dependencies (1) major (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 736 last-month
  • Total dependent packages: 1
  • Total dependent repositories: 0
  • Total versions: 8
  • Total maintainers: 1
pypi.org: datalad-fuse

DataLad FUSE extension package

  • Versions: 8
  • Dependent Packages: 1
  • Dependent Repositories: 0
  • Downloads: 736 Last month
Rankings
Dependent packages count: 10.1%
Downloads: 10.4%
Forks count: 19.1%
Average: 27.8%
Stargazers count: 31.9%
Dependent repos count: 67.4%
Maintainers (1)
yarikoptic
Last synced: 10 months ago

Dependencies

requirements-devel.txt pypi
  • sphinx *
  • sphinx_rtd_theme *
.github/workflows/benchmark.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/docbuild.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/lint.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/release.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/test-libfuse.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • codecov/codecov-action v3 composite
.github/workflows/test.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • codecov/codecov-action v3 composite
.github/workflows/typing.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v4 composite
_datalad_buildsupport/setup.py pypi
pyproject.toml pypi
setup.py pypi