Improving Valence Prediction in Dimensional Speech Emotion Recognition using Linguistic Information

by Bagus Tris Atmaja and Masato Akagi

This paper has been accepted for publication in OCOCOSDA 2020.

Abstract

In dimensional emotion recognition, a model called valence, arousal, and dominance is widely used. The current research has shown that the performance of valence prediction is lower than arousal and dominance in dimensional speech emotion recognition. This paper presents an approach to improve the low score on valence to approach or surpass arousal and dominance scores. Our approach combines acoustic features with text features, which is a conversion from words to vectors. The results showed significant improvements on both single-task learning single-output (predicting valence only) and multitask learning multi-output (predicting valence, arousal, and dominance). Using a proper combination of acoustic and text features not only improved valence prediction but also improved dominance prediction in multitask learning.

Software implementation

Briefly describe the software that was written to produce the results of this paper.

All source code used to generate the results and figures in the paper are in the code folder. The calculations and figure generation are all run inside Jupyter notebooks. The data used in this study is provided in data and the sources for the manuscript text and figures are in manuscript. Results generated by the code are saved in results. See the README.md files in each directory for a full description.

Getting the code

You can download a copy of all the files in this repository by cloning the git repository:

git clone https://github.com/bagustris/dser_with_text.git

or download a zip archive.

A copy of the repository is also archived at insert DOI here

Dependencies

You'll need a working Python environment to run the code. The recommended way to set up your environment is through the Anaconda Python distribution which provides the conda package manager. Anaconda can be installed in your user directory and does not interfere with the system Python installation. The required dependencies are specified in the file requirements.txt.

We use pip virtual environments to manage the project dependencies in isolation. Thus, you can install our dependencies without causing conflicts with your setup (even with different Python versions).

Run the following command in the repository folder (where environment.yml is located) to create a separate environment and install all required dependencies in it:

pip3.6 venv REPO_NAME

Reproducing the results

Before running any code you must activate the conda environment:

source activate REPO_NAME

To reproduce result in , run the following in order:
bash

License

All source code is made available under a BSD 3-clause license. You can freely use and modify the code, without warranty, so long as you provide attribution to the authors. See LICENSE.md for the full license text.

The manuscript text is not open source. The authors reserve the rights to the article content, which is published in oCOCOSDA 2020.

Citation

B.T. Atmaja, and M. Akagi. "Improving Valence Prediction in Dimensional Speech Emotion Recognition Using Linguistic Information." In 2020 23rd Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), pp. 166-171. IEEE, 2020.