pescador
Stochastic multi-stream sampling for iterative learning
Science Score: 46.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
✓DOI references
Found 3 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
✓Committers with academic emails
4 of 12 committers (33.3%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (14.1%) to scientific vocabulary
Keywords
Keywords from Contributors
Repository
Stochastic multi-stream sampling for iterative learning
Basic Info
- Host: GitHub
- Owner: pescadores
- License: isc
- Language: Python
- Default Branch: main
- Homepage: https://pescador.readthedocs.io
- Size: 588 KB
Statistics
- Stars: 81
- Watchers: 7
- Forks: 11
- Open Issues: 14
- Releases: 12
Topics
Metadata Files
README.md
pescador
Pescador is a library for streaming (numerical) data, primarily for use in machine learning applications.
Pescador addresses the following use cases:
- Hierarchical sampling
- Out-of-core learning
- Parallel streaming
These use cases arise in the following common scenarios:
- Say you have three data sources
(A, B, C)that you want to sample. For example, each data source could contain all the examples of a particular category.
Pescador can dynamically interleave these sources to provide a randomized stream D <- (A, B, C).
The distribution over (A, B, C) need not be uniform: you can specify any distribution you like!
- Now, say you have 3000 data sources, each of which may contain a large number of samples. Maybe that's too much data to fit in RAM at once.
Pescador makes it easy to interleave these sources while maintaining a small working set.
Not all sources are simultaneously active, but Pescador manages the working set so you don't have to.
- If loading data incurs substantial latency (e.g., due to accessing on-disk storage or pre-processing), this can be a problem.
Pescador can seamlessly move data generation into a background process, so that your main thread can continue working.
Want to learn more? Read the docs!
Installation
Pescador can be installed from PyPI through pip:
pip install pescador
or with conda using the conda-forge channel:
conda install -c conda-forge pescador
Owner
- Name: pescadores
- Login: pescadores
- Kind: organization
- Repositories: 4
- Profile: https://github.com/pescadores
An organization for pescador development
GitHub Events
Total
- Watch event: 5
Last Year
- Watch event: 5
Committers
Last synced: over 2 years ago
Top Committers
| Name | Commits | |
|---|---|---|
| Brian McFee | b****e@n****u | 203 |
| Christopher Jacoby | c****y@g****m | 70 |
| Brian McFee | b****2@c****u | 20 |
| Christopher Jacoby | c****y@m****m | 14 |
| Eric J. Humphrey | e****y@n****u | 8 |
| Vincent Lostanlen | v****n@n****u | 5 |
| Eric Humphrey | e****y@s****m | 4 |
| Hugo | h****k@u****m | 4 |
| Brian McFee | b****e@u****m | 3 |
| Eric Humphrey | h****c@g****m | 3 |
| Hendrik Schreiber | hs@t****m | 1 |
| Waldir Pimenta | w****s@g****m | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 60
- Total pull requests: 50
- Average time to close issues: 12 months
- Average time to close pull requests: 2 months
- Total issue authors: 11
- Total pull request authors: 7
- Average comments per issue: 6.3
- Average comments per pull request: 3.98
- Merged pull requests: 45
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- bmcfee (20)
- ejhumphrey (17)
- cjacoby (9)
- faroit (5)
- lostanlen (2)
- stefan-balke (2)
- jongwook (1)
- rabitt (1)
- beasteers (1)
- IshwaryaAnant (1)
Pull Request Authors
- bmcfee (36)
- cjacoby (9)
- hugovk (5)
- ejhumphrey (5)
- lostanlen (1)
- hendriks73 (1)
- faroit (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 2
-
Total downloads:
- pypi 2,651 last-month
- Total docker downloads: 63
-
Total dependent packages: 1
(may contain duplicates) -
Total dependent repositories: 34
(may contain duplicates) - Total versions: 18
- Total maintainers: 1
pypi.org: pescador
Multiplex generators for incremental learning
- Homepage: https://github.com/pescadores/pescador
- Documentation: https://pescador.readthedocs.io/
- License: ISC
-
Latest release: 3.0.0
published almost 2 years ago
Rankings
Maintainers (1)
conda-forge.org: pescador
- Homepage: http://github.com/pescadores/pescador
- License: ISC
-
Latest release: 2.1.0
published over 6 years ago
Rankings
Dependencies
- matplotlib *
- numpydoc >=0.6
- six *
- sphinx *
- sphinx-gallery *
- sphinx_rtd_theme *
- decorator >=4.0
- joblib >=0.9
- numpy >=1.9
- pyzmq >=15.0
- six >=1.8
- actions/cache v4 composite
- actions/checkout v4 composite
- codecov/codecov-action v3 composite
- conda-incubator/setup-miniconda v3 composite
- actions/cache v4 composite
- actions/checkout v4 composite
- conda-incubator/setup-miniconda v3 composite