pygamma-agreement
pygamma-agreement: Gamma γ measure for inter/intra-annotator agreement in Python - Published in JOSS (2021)
Science Score: 93.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 6 DOI reference(s) in README and JOSS metadata -
✓Academic publication links
Links to: joss.theoj.org -
○Committers with academic emails
-
○Institutional organization owner
-
✓JOSS paper metadata
Published in Journal of Open Source Software
Keywords
Scientific Fields
Repository
Gamma Agreement in Python
Basic Info
- Host: GitHub
- Owner: bootphon
- License: mit
- Language: Jupyter Notebook
- Default Branch: master
- Size: 1.92 MB
Statistics
- Stars: 45
- Watchers: 7
- Forks: 8
- Open Issues: 4
- Releases: 0
Topics
Metadata Files
README.md
Pygamma-agreement
pygamma-agreement is an open-source package to measure Inter/Intra-annotator[^1] agreement for sequences of annotations with the γ measure[^2]. It is written in Python3 and based mostly on NumPy, Numba and pyannote.core. For a full list of available functions, please refer to the package documentation.
Dependencies
The main dependencies of pygamma-agreement are NumPy, CVXPY, Numba and pyannote.core.
Optionally, to allow pygamma-agreement to display visual representations of
our API's objects in Jupyter Notebooks, Matplotlib
is needed.
pygamma-agreement is a Python 3 package and is currently tested for Python 3.7 to 3.11.
pygamma-agreement does not work with Python 2.7.
Installation
pygamma-agreement can be easily installed using pip :
$ pip install pygamma-agreement
Pygamma-agreement uses the GNU Linear Programming Kit as its default Mixed Integer
Programming solver (critical step of the gamma-agreement algorithm). Since it is quite slow, you can install the
CBC solver and its official python API.
To use those in pygamma-agreement, simply install them:
- Ubuntu/Debian :
$ sudo apt install coinor-libcbc-dev - Fedora :
$ sudo yum install coin-or-Cbc-devel - Arch Linux :
$ sudo pacman -S coin-or-cbc - Mac OS X :
$ brew tap coin-or-tools/coinor$ brew install coin-or-tools/coinor/cbc pkg-config
then:
$ pip install "pygamma-agreement[CBC]"
If you have trouble during the two last steps, pygamma-agreement should work anyway, although significantly slower for larger input.
WARNING: A bug in GLPK causes the standard output to be polluted by non-deactivable messages. It is strongly advised to use the CBC solver.
Tests
The package comes with a unit-tests suite. To run it, first install the CBC solver, then the testing dependencies on your Python environment:
$ sudo apt install coinor-libcbc-dev
$ pip install "pygamma-agreement[testing]"
After that, tests are ready to be launched (they take ~2m on a recent laptop) :
$ pytest tests/
Documentation
The documentation for this package has been written using Sphinx. To build the documentation locally, run:
$ pip install "pygamma-agreement[docs]"
$ cd docs/
$ make html
After that, you can view the documentation by running
$ firefox build/html/index.html
Submitting and issue or contributing
Please read CONTRIBUTING.md before submitting and issue or writing some contribution
to this package.
Citing Pygamma
If you're using pygamma in your work, please cite our package using the following bibtex entry:
``` @article{Titeux2021, doi = {10.21105/joss.02989}, url = {https://doi.org/10.21105/joss.02989}, year = {2021}, publisher = {The Open Journal}, volume = {6}, number = {62}, pages = {2989}, author = {Hadrien Titeux and Rachid Riad}, title = {pygamma-agreement: Gamma $\gamma$ measure for inter/intra-annotator agreement in Python}, journal = {Journal of Open Source Software} }
```
References
[^1]: Titeux H., Riad R. pygamma-agreement: Gamma γ measure for inter/intra-annotator agreement in Python.
Owner
- Name: CoML
- Login: bootphon
- Kind: organization
- Email: syntheticlearner@gmail.com
- Location: Paris, France
- Website: https://cognitive-ml.fr
- Repositories: 55
- Profile: https://github.com/bootphon
JOSS Publication
pygamma-agreement: Gamma γ measure for inter/intra-annotator agreement in Python
Authors
LSCP/ENS/CNRS/EHESS/INRIA/PSL Research University, Paris, France
LSCP/ENS/CNRS/EHESS/INRIA/PSL Research University, Paris, France, NPI/ENS/INSERM/UPEC/PSL Research University, Créteil, France
Editor
Arfon Smith
Tags
linguistics annotation statisticsGitHub Events
Total
- Watch event: 2
Last Year
- Watch event: 2
Committers
Last synced: 5 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| lfavre | f****d@g****m | 219 |
| hadware | h****z@g****m | 121 |
| Rachine | r****3@g****m | 60 |
| Valentin Oliver Loftsson | v****n@g****m | 4 |
| leopold0favre | 7****e | 3 |
| tim_sockel | t****l@u****e | 2 |
| Fabian-Robert Stöter | m****l@f****m | 1 |
| Daniel Göller | 6****p | 1 |
| Arfon Smith | a****n | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 4 months ago
All Time
- Total issues: 30
- Total pull requests: 17
- Average time to close issues: 3 months
- Average time to close pull requests: 21 days
- Total issue authors: 14
- Total pull request authors: 8
- Average comments per issue: 1.7
- Average comments per pull request: 1.06
- Merged pull requests: 15
- Bot issues: 0
- Bot pull requests: 1
Past Year
- Issues: 1
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 1
- Pull request authors: 0
- Average comments per issue: 0.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- Rachine (10)
- hadware (5)
- valentinoli (3)
- faroit (2)
- lucasgautheron (1)
- TiMauzi (1)
- vestedinterests (1)
- sangwonme (1)
- zlindaz2411 (1)
- michaelvauth (1)
- apiad (1)
- phlobo (1)
- hbredin (1)
- azehe (1)
Pull Request Authors
- hadware (6)
- valentinoli (4)
- TiMauzi (2)
- ghost (2)
- arfon (1)
- dependabot[bot] (1)
- faroit (1)
- LelViLamp (1)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- pypi 264 last-month
- Total dependent packages: 1
- Total dependent repositories: 2
- Total versions: 20
- Total maintainers: 2
pypi.org: pygamma-agreement
Inter-annotator agreement measure and alignment written in python
- Homepage: https://pygamma-agreement.readthedocs.io/en/latest/
- Documentation: https://pygamma-agreement.readthedocs.io/en/latest/
- License: MIT
-
Latest release: 0.5.9
published almost 2 years ago
Rankings
Maintainers (2)
Dependencies
- TextGrid >=1.5
- cvxopt ==1.2.7
- cvxpy >=1.0.25
- dataclasses >=0.7
- numba >=0.54.0
- numpy >=1.10.4
- pyannote.core >=4.1
- pyannote.database >=4.1
- pympi-ling >=1.69
- sortedcontainers >=2.0.4
- tqdm >=4.46.0
- typing_extensions >=3.7.4.3
- actions/checkout v2 composite
- actions/upload-artifact v1 composite
- openjournals/openjournals-draft-action master composite
- actions/checkout v4 composite
- actions/setup-python v5 composite