https://github.com/bagustris/exvo2022
Repo for ExVo 2022 Challenge, forked from humaAI
Science Score: 36.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (9.9%) to scientific vocabulary
Repository
Repo for ExVo 2022 Challenge, forked from humaAI
Basic Info
Statistics
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
ICML ExVo 2022 Workshop & Competition
Baseline code for the three tracks of ExVo 2022 competition. Full details and results can be found in the lastest release of the ExVo White Paper.
The Multi-task High-Dimensional Emotion, Age & Country Task (ExVo-MultiTask)
In ExVo MultiTask, participants will be challenged with predicting the average intensity of each of 10 emotions perceived in vocal bursts, as well as the speaker's Age and native-country, as a multi-task process. For emotion and age, the participants will perform a regression task, and for native-country, a 4-class classification. Participants will report the Concordance Correlation Coefficient (CCC), for the emotion regression task, Mean Absolute Error (MAE) for Age (in years), and Unweighted Average Recall (UAR) for the native-country classification task. The baseline for this track is based on a combined score computed by the harmonic mean between CCC, (inverted) MAE, and UAR.
The Baseline score to beat on the test set for ExVo-MultiTask is: 0.335 SMTL
The Generative Emotional Vocal Burst Task (ExVo-Generate)
In the ExVo Generate task, participants are tasked with applying generative modeling approaches to produce vocal bursts that are associated with 10 distinct emotions. Each team should submit 1000 machine-generated vocalizations (100 for each class) that differentially convey each of the 10 emotions—“awe,” “fear,” etc.—with maximal intensity and fidelity. Participants can submit samples for either one or all classes. The ExVo organization team will provide code for computing the Fréchet Inception Score, this score should also be provided to the organisers. The final evaluation will incorporate human ratings gathered by Hume AI of a random subset of 5/100 samples per targeted emotion. These ratings of the generated vocal bursts will be gathered using the same methodology used to collect the training data, with each vocal burst judged in terms of the perceived intensity of each target emotion. Generated samples will be evaluated based on the Pearsons between normed (0-1) average intensity ratings for the 10 classes and the identity matrix consisting of dummy variables for each class.
The Baseline score to beat for ExVo-Generate is: 0.174 SGEN
The Few-Shot Emotion Recognition task (ExVo-FewShot)
In the ExVo Few-Shot task, the participants will predict the same 10 emotions as a multi-output regression task, using a model or multiple models. Participants will be provided with at least 2 samples per speaker in all splits (train, validation, test) and will be tasked with performing two-shot personalized emotion recognition. The subject IDs and corresponding emotion labels with two samples per speaker in the test set will be withheld until a week before the deadline for final evaluation of ExVo Few-Shot models on the test data. Participants will report the Concordance Correlation Coefficient (CCC) across all 10 emotion outputs as an evaluation metric.
The Baseline score to beat on the test set for ExVo-FewShot is: 0.444 CCC
More info on competitions guidelines found competitions.hume.ai.
Any questions: competitions@hume.ai
© 2022 Creative Commons Attribution-Non Commercial-No Derivatives 4.0 International Licence (CC BY-NC-ND)
Owner
- Name: Bagus Tris Atmaja
- Login: bagustris
- Kind: user
- Location: Tsukuba
- Company: AIST
- Website: http://www.bagustris.blogspot.com
- Twitter: btatmaja
- Repositories: 221
- Profile: https://github.com/bagustris
Researcher @aistairc @VibrasticLab
GitHub Events
Total
Last Year
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| B Atmaja | b****a@a****p | 8 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0