# Deep Semi-Supervised Learning with Holistic methods (SSLH) Unofficial PyTorch and PyTorch-Lightning implementations of Deep Semi-Supervised Learning methods for audio tagging.

There is 4 SSL methods : - FixMatch (FM) [1] - MixMatch (MM) [2] - ReMixMatch (RMM) [3] - Unsupervised Data Augmentation (UDA) [4]

For the following datasets : - CIFAR-10 (CIFAR10) - ESC-10 (ESC10) - Google Speech Commands (GSC) - Primate Vocalization Corpus (PVC) - UrbanSound8k (UBS8K)

With 3 models : - WideResNet28 (WRN28) - MobileNetV1 (MNV1) - MobileNetV2 (MNV2)

If you meet problems to run experiments, you can contact me at lpriyanshjainofficial@gmail.com.

Installation

Download & setup

bash git clone https://github.com/priyanshjain3/SSLH1 conda env create -n env_sslh -f environment.yaml conda activate env_sslh pip install -e SSLH --no-dependencies

Alternatives

• As python package : bash pip install https://github.com/priyanshjain3/SSLH1 The dependencies will be automatically installed with pip instead of conda, which means the the build versions can be slightly different.

The project contains also a environment.yaml and requirements.txt for installing the packages respectively with conda or pip. - With conda environment file : bash conda env create -n env_sslh -f environment.yaml conda activate env_sslh pip install -e . --no-dependencies

• With pip requirements file : bash pip install -r requirements.txt pip install -e . --no-dependencies

Datasets

CIFAR10, ESC10, GoogleSpeechCommands and FSD50K can be downloaded and installed. For UrbanSound8k, please read the README of leocances, in section "Prepare the dataset". AudioSet (ADS) and Primate Vocalize Corpus (PVC) cannot be installed automatically by now.

To download a dataset, you can use the data.download=true option.

[comment]: <> (TODO : For Audioset install !) [comment]: <> (TODO : For PVC install !)

Usage

This code use Hydra for parsing args. The syntax of setting an argument is "name=value" instead of "--name value".

Example 1 : Itermix on ESC10 bash python -m sslh.mixmatch data=ssl_esc10 pl=itermix data.dm.download=true

Example 2 : MixMatch on ESC10 bash python -m sslh.mixmatch data=ssl_esc10 data.dm.download=true

Example 3 : Supervised+Weak on GSC bash python -m sslh.supervised data=sup_gsc aug@train_aug=weak data.dm.bsize=256 epochs=300 data.dm.download=true

Example 4 : FixMatch+IterMixUp on UBS8K bash python -m sslh.fixmatch data=ssl_ubs8K pl=fixmatch_itermixup data.dm.bsize_s=128 data.dm.bsize_u=128 epochs=300 data.dm.download=true

Example 5 : ReMixMatch on CIFAR-10 bash python -m sslh.remixmatch data=ssl_cifar10 model.n_input_channels=3 aug@weak_aug=img_weak aug@strong_aug=img_strong data.dm.download=true

List of main arguments

| Name | Description | Values | Default | | --- | --- | --- | --- | | data | Dataset used | (sup|ssl)(ads|cifar10|esc10|fsd50k|gsc|pvc|ubs8k) | (sup|ssl)esc10 | | pl | Pytorch Lightning training method (experiment) used | (depends of the python script, see the filenames in config/pl/ folder) | (depends of the python script) | | model | Pytorch model to use | mobilenetv1, mobilenetv2, vgg, wideresnet28 | wideresnet28 | | optim | Optimizer used | adam, sgd | adam | | sched | Learning rate scheduler | cosine, softcosine, none | softcosine | | epochs | Number of training epochs | int | 1 | | bsize | Batch size in SUP methods | int | 60 | | ratio | Ratio of the training data used in SUP methods | float in [0, 1] | 1.0 | | bsizes | Batch size of supervised part in SSL methods | int | 30 | | bsizeu | Batch size of unsupervised part in SSL methods | int | 30 | | ratios | Ratio of the supervised training data used in SSL methods | float in [0, 1] | 0.1 | | ratiou | Ratio of the unsupervised training data used in SSL methods | float in [0, 1] | 0.9 |

SSLH Package overview

sslh ├── callbacks ├── datamodules │ ├── supervised │ └── semi_supervised ├── datasets ├── pl_modules │ ├── deep_co_training │ ├── fixmatch │ ├── mean_teacher │ ├── mixmatch │ ├── mixup │ ├── pseudo_labeling │ ├── remixmatch │ ├── supervised │ └── uda ├── metrics ├── models ├── transforms │ ├── get │ ├── image │ ├── other │ ├── pools │ ├── self_transforms │ ├── spectrogram │ └── waveform └── utils

Authors

This repository has been created by Etienne Labbé (Labbeti on Github).

It contains also some code from the following authors : - Léo Cancès (leocances on github) - For AudioSet, ESC10, GSC, PVC and UBS8K datasets base code. - Qiuqiang Kong (qiuqiangkong on Github) - For MobileNetV1 & V2 model implementation from PANN.

Additional notes

• This project has been made with Ubuntu 20.04 and Python 3.8.5.

Glossary

| Acronym | Description | | --- | --- | | activation | Activation function | | ADS | AudioSet | | aug, augm, augment | Augmentation | | ce | Cross-Entropy | | expt | Experiment | | fm | FixMatch | | fn, func | Function | | GSC | Google Speech Commands dataset (with 35 classes) | | GSC12 | Google Speech Commands dataset (with 10 classes from GSC, 1 unknown class and 1 silence class) | | hparams | Hyperparameters | | js | Jensen-Shannon | | kl | Kullback-Leibler | | loc | Localisation | | lr | Learning Rate | | mm | MixMatch | | mse | Mean Squared Error | | pred | Prediction | | PVC | Primate Vocalize Corpus dataset | | rmm | ReMixMatch | | _s | Supervised | | sched | Scheduler | | SSL | Semi-Supervised Learning | | SUP | Supervised Learning | | _u | Unsupervised | | UBS8K | UrbanSound8K dataset |

References

[1] K. Sohn, D. Berthelot, C.-L. Li, Z. Zhang, N. Carlini, E. D. Cubuk, A. Ku- rakin, H. Zhang, and C. Raffel, “FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence,” p. 21.

[2] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. Raffel, “MixMatch: A Holistic Approach to Semi-Supervised Learning,” Oct. 2019, number: arXiv:1905.02249 arXiv:1905.02249 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1905.02249

[3] D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel, “ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring,” Feb. 2020, number: arXiv:1911.09785 arXiv:1911.09785 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1911.09785

[4] Q. Xie, Z. Dai, E. Hovy, M.-T. Luong, and Q. V. Le, “Unsu- pervised Data Augmentation for Consistency Training,” Nov. 2020, number: arXiv:1904.12848 arXiv:1904.12848 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1904.12848