https://github.com/beacon-biosignals/lighthouse.jl
Performance evaluation tools for multiclass, multirater classification models
Science Score: 26.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.4%) to scientific vocabulary
Keywords from Contributors
Repository
Performance evaluation tools for multiclass, multirater classification models
Basic Info
- Host: GitHub
- Owner: beacon-biosignals
- License: mit
- Language: Julia
- Default Branch: main
- Size: 19.5 MB
Statistics
- Stars: 18
- Watchers: 22
- Forks: 2
- Open Issues: 28
- Releases: 40
Metadata Files
README.md
Lighthouse.jl
Lighthouse.jl is a Julia package that standardizes and automates performance evaluation for multiclass, multirater classification models. By implementing a minimal interface, your classifier automagically gains a thoroughly instrumented training/testing harness (Lighthouse.learn!) that computes and logs tons of meaningful performance metrics to TensorBoard in real-time, including:
- test set loss
- inter-rater agreement (e.g. Cohen's Kappa)
- PR curves
- ROC curves
- calibration curves
Lighthouse itself is framework-agnostic; end-users should use whichever extension package matches their desired framework (e.g. https://github.com/beacon-biosignals/LighthouseFlux.jl).
This package follows the YASGuide.
Installation
To install Lighthouse for development, run:
julia -e 'using Pkg; Pkg.develop(PackageSpec(url="https://github.com/beacon-biosignals/Lighthouse.jl"))'
This will install Lighthouse to the default package development directory, ~/.julia/dev/Lighthouse.
TensorBoard
Note that Lighthouse's LearnLogger logs metrics to a user-specified path in TensorBoard's logdir format. TensorBoard can be installed via python3 -m pip install tensorboard (note: if you have tensorflow>=1.14, you should already have tensorboard). Once TensorBoard is installed, you can view Lighthouse-generated metrics via tensorboard --logdir path where path is the path specified by Lighthouse.LearnLogger. From there, TensorBoard itself can be used/configured however you like; see https://github.com/tensorflow/tensorboard for more information.
You can use alternative loggers, as long as they comply with the logging interface.
Owner
- Name: Beacon Biosignals
- Login: beacon-biosignals
- Kind: organization
- Website: https://beacon.bio
- Repositories: 30
- Profile: https://github.com/beacon-biosignals
GitHub Events
Total
- Create event: 2
- Commit comment event: 2
- Release event: 1
- Issues event: 1
- Delete event: 1
- Issue comment event: 1
- Push event: 10
- Pull request event: 1
- Pull request review event: 8
- Pull request review comment event: 7
Last Year
- Create event: 2
- Commit comment event: 2
- Release event: 1
- Issues event: 1
- Delete event: 1
- Issue comment event: 1
- Push event: 10
- Pull request event: 1
- Pull request review event: 8
- Pull request review comment event: 7
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| Hannah Robertson | h****a | 27 |
| Eric Hanson | 5****n | 11 |
| Simon | s****h@p****m | 7 |
| Alex Chan | a****n | 5 |
| Phillip Alday | p****y | 4 |
| Chris de Graaf | me@c****v | 3 |
| David Josephs | 4****d | 2 |
| a-cakir | 1****r | 1 |
| McCoy R. Becker | 3****c | 1 |
| Kim Laberinto | g****t@k****m | 1 |
| Dave Kleinschmidt | d****t@g****m | 1 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 12 months ago
All Time
- Total issues: 35
- Total pull requests: 72
- Average time to close issues: 4 months
- Average time to close pull requests: 6 days
- Total issue authors: 10
- Total pull request authors: 12
- Average comments per issue: 2.34
- Average comments per pull request: 1.26
- Merged pull requests: 63
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 2
- Average time to close issues: N/A
- Average time to close pull requests: about 16 hours
- Issue authors: 0
- Pull request authors: 1
- Average comments per issue: 0
- Average comments per pull request: 1.5
- Merged pull requests: 2
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- ericphanson (13)
- hannahilea (11)
- femtomc (3)
- alexmchan (2)
- abhinav3398 (1)
- jrevels (1)
- JuliaTagBot (1)
- a-cakir (1)
- nbagherzadeh (1)
Pull Request Authors
- hannahilea (23)
- ericphanson (15)
- palday (11)
- SimonDanisch (9)
- alexmchan (5)
- josephsdavid (4)
- christopher-dG (3)
- soulshake (2)
- kimlaberinto (1)
- kleinschmidt (1)
- a-cakir (1)
- femtomc (1)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- actions/cache v2 composite
- actions/checkout v2 composite
- codecov/codecov-action v2 composite
- julia-actions/julia-buildpkg v1 composite
- julia-actions/julia-processcoverage v1 composite
- julia-actions/julia-runtest v1 composite
- julia-actions/setup-julia v1 composite
- percy/exec-action v0.3.1 composite
- JuliaRegistries/TagBot v1 composite
- actions/checkout v2 composite
- actions/checkout v1 composite
- julia-actions/setup-julia latest composite
- reviewdog/action-suggester v1 composite