Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
1 of 4 committers (25.0%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.8%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: janezd
- License: mit
- Language: Python
- Default Branch: master
- Size: 340 KB
Statistics
- Stars: 74
- Watchers: 10
- Forks: 15
- Open Issues: 2
- Releases: 1
Metadata Files
README.md
baycomp
Baycomp is a library for Bayesian comparison of classifiers.
Functions compare two classifiers on one or on multiple data sets. They
compute three probabilities: the probability that the first classifier has
higher scores than the second, the probability that differences are within
the region of practical equivalence (rope), or that the second classifier
has higher scores. We will refer to this probabilities as p_left, p_rope
and p_right. If the argument rope is omitted (or set to zero), functions
return only p_left and p_right.
The region of practical equivalence (rope) is specified by the caller and should correspond to what is "equivalent" in practice; for instance, classification accuracies that differ by less than 0.5 may be called equivalent.
Similarly, whether higher scores are better or worse depends upon the type of the score.
The library can also plot the posterior distributions.
The library can be used in three ways.
Two shortcut functions can be used for comparison on single and on multiple data sets. If
nbcandj48contain a list of average classification accuracies of naive Bayesian classifier and J48 on a collection of data sets, we can call>>> two_on_multiple(nbc, j48, rope=1) (0.23124, 0.00666, 0.7621)
(Actual results may differ due to Monte Carlo sampling.)
With some additional arguments, the function can also plot the posterior distribution from which these probabilities came.
Tests are packed into test classes. The above call is equivalent to
>>> SignedRankTest.probs(nbc, j48, rope=1) (0.23124, 0.00666, 0.7621)
and to get a plot, we call
>>> SignedRankTest.plot(nbc, j48, rope=1, names=("nbc", "j48"))
To switch to another test, use another class::
>>> SignTest.probs(nbc, j48, rope=1)
(0.26508, 0.13274, 0.60218)
Finally, we can construct and query sampled posterior distributions.
>>> posterior = SignedRankTest(nbc, j48, rope=0.5) >>> posterior.probs() (0.23124, 0.00666, 0.7621) >>> posterior.plot(names=("nbc", "j48"))
Installation
Install from PyPI:
pip install baycomp
Documentation
User documentation is available on https://baycomp.readthedocs.io/.
A detailed description of the implemented methods is available in Time for a Change: a Tutorial for Comparing Multiple Classifiers Through Bayesian Analysis, Alessio Benavoli, Giorgio Corani, Janez Demšar, Marco Zaffalon. Journal of Machine Learning Research, 18 (2017) 1-36.
Owner
- Name: Janez Demšar
- Login: janezd
- Kind: user
- Location: Ljubljana, Slovenia
- Company: University of Ljubljana
- Repositories: 41
- Profile: https://github.com/janezd
GitHub Events
Total
- Watch event: 5
Last Year
- Watch event: 5
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| janezd | j****r@f****i | 25 |
| luccaportes | l****4@g****m | 4 |
| David Pätzel | d****l@p****e | 2 |
| Matt Dirks | m****t@s****a | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 11 months ago
All Time
- Total issues: 14
- Total pull requests: 4
- Average time to close issues: about 2 months
- Average time to close pull requests: 3 days
- Total issue authors: 11
- Total pull request authors: 3
- Average comments per issue: 2.79
- Average comments per pull request: 1.75
- Merged pull requests: 4
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- dpaetzel (2)
- luccaportes (2)
- jmhessel (2)
- davidshumway (1)
- henrique-voni (1)
- Arturus (1)
- usptact (1)
- jorjasso (1)
- KCrux (1)
- sherbold (1)
- gcelano (1)
Pull Request Authors
- dpaetzel (2)
- luccaportes (1)
- skylogic004 (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 3
-
Total downloads:
- pypi 12,526 last-month
- Total docker downloads: 50
-
Total dependent packages: 6
(may contain duplicates) -
Total dependent repositories: 37
(may contain duplicates) - Total versions: 8
- Total maintainers: 1
pypi.org: baycomp
Bayesian tests for comparison of classifiers
- Homepage: https://github.com/janezd/baycomp.git
- Documentation: https://baycomp.readthedocs.io/
- License: MIT License
-
Latest release: 1.0.3
published almost 3 years ago
Rankings
Maintainers (1)
conda-forge.org: baycomp
A library for comparing results of predictive models on single or multiple data sets using Bayesian approaches. The libary is in pure Python, and depends on numpy, scipy and matplotlib. The more advanced hierarchical tests require pystan, which needs to be installed separately.
- Homepage: https://github.com/janezd/baycomp
- License: MIT
-
Latest release: 1.0.2
published over 6 years ago
Rankings
anaconda.org: baycomp
A library for comparing results of predictive models on single or multiple data sets using Bayesian approaches. The libary is in pure Python, and depends on numpy, scipy and matplotlib. The more advanced hierarchical tests require pystan, which needs to be installed separately.
- Homepage: https://github.com/janezd/baycomp
- License: MIT
-
Latest release: 1.0.3
published about 1 year ago
Rankings
Dependencies
- matplotlib *
- numpy *
- scipy *