uncertainty-bench
Code repository for **Benchmarking Scalable Predictive Uncertainty in Text Classification**, by Jordy Van Landeghem, Matthew Blaschko, Bertrand Anckaert and Marie-Francine Moens.
Science Score: 44.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.1%) to scientific vocabulary
Keywords
Repository
Code repository for **Benchmarking Scalable Predictive Uncertainty in Text Classification**, by Jordy Van Landeghem, Matthew Blaschko, Bertrand Anckaert and Marie-Francine Moens.
Basic Info
Statistics
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 0
- Releases: 1
Topics
Metadata Files
README.md
uncertainty-bench
Repository for Benchmarking Scalable Predictive Uncertainty in Text Classification, by Jordy Van Landeghem, Matthew Blaschko, Bertrand Anckaert and Marie-Francine Moens.
It contains the source code, the experiments, and datasets used
in src, experiments, and datasets respectively.
Motivation
Perfect predictive accuracy is unattainable for most text classification problems, explaining the need for reliable ML solutions that can communicate predictive uncertainty when dealing with noisy or unknown inputs. In the quest for a simple, principled and scalable uncertainty method, which one to choose, when and why?
Our survey on Bayesian Deep Learning methods and benchmarking on 6 different text classification datasets aims to help practicioners make this decision and have future researchers spurred to continue investigations into hybrid uncertainty methods.
Methods
Installation
Usage
Training a model
main file: experiment.py
Example command:
python3 experiment.py with CONFIG_NAME identifier=DATASET
Citation
@inproceedings{VanLandeghem2021,
TITLE = {Benchmarking Scalable Predictive Uncertainty in Text Classification},
AUTHOR = {Van Landeghem, Jordy and Blaschko, Matthew B. and Anckaert, Bertrand and Moens, Marie-Francine},
BOOKTITLE = {Submitted to ...},
YEAR = {2021}
}
Results
Disclaimer
The code was originally run in a corporate environment*, now reimplemented and open-sourced for aiding the research community. There will be small changes between the current output & results presented in the paper.
<!---
Changelog
- [x] boilerplate repo
- [x] raw evaluation data
- [x] link or host datasets
- [x] update experiment instructions
- [x] assign proper LICENSE
- [x] re-implementation, see Disclaimer ---!>
Owner
- Name: Jordy Van Landeghem
- Login: Jordy-VL
- Kind: user
- Location: Brussels
- Company: Contract.fit
- Website: https://www.linkedin.com/in/jordy-van-landeghem-3b1166b3/
- Twitter: JordyLandeghem
- Repositories: 41
- Profile: https://github.com/Jordy-VL
PhD-student in Artificial Intelligence @ KU Leuven / NLP Research Engineer @Contractfit Trying to understand why computers do not understand documents.
Citation (CITATION.cff)
cff-version: 1.1.0
message: If you use this software, next to the paper, please cite as below.
authors:
- family-names: Van Landeghem
given-names: Jordy
title: uncertainty-bench
version: 0.0.1
date-released: 2021-07-01