Improving generalization via style transfer-based data augmentation: Novel regularization method

Introduction

Currently, deep learning algorithms are considered as state-of-the-art in many classification tasks, and yet the problem of weak generalization is very common, widely mentioned, and still up-to-date.

The present paper focuses most on the data augmentation. In our method, new images are synthetized with neural style transfer (NST), and the generated images are then used to train the convolutional neural network (CNN) in order to improve its generalization abilities.
The main contributions of this paper are: * The proposition of using neural style transfer for the data augmentation (ST-DA). This approach is presented on the skin lesion case study by transforming a benign skin lesion to a malignant lesion, and tested with dataset enrichment evaluation; * Incorporating unlabeled, synthesized data into training by adding pseudo-labels generated by another CNN; * Limiting the problem of noisy pseudo-labels in synthetic images used as a CNN training set by using only real images in validation and test sets; * Evaluating the ability to enrich the training dataset with artificially generated data with Deep Taylor Decomposition, * Proving that the ST-DA method significantly improves the performance and repeatability of training for deep neural networks.

ST-DA

How-to

Short and friendly how-to tutorial will be soon available here

Details

The result and details of the method will be able to be find soon in the original paper here: soon You can check instead our previous papers about data augmentation: * Data augmentation for improving deep learning in image classification problem, 2018 * Style transfer-based image synthesis as an efficient regularization technique in deep learning, 2019

Database

Download

The total databse size is 248 489 unalabeled generated dermoscopic images of skin lesions (224x224 px). * Few full-size examples can be found here * Database can be download soon here (soon)

If you use this database please star the repository and cite the following paper (soon):

"Improving generalization via style transfer-based data augmentation: Novel regularization method", by Agnieszka Mikołajczyk , Michał Grochowski, Arkadiusz Kwasigroch

Sources

The database was generated using following sources:

• Image generation:
  • Style transfer original paper: A Neural Algorithm of Artistic Style is a first paper that presented Neural Style Transfer.
  • Style transfer implementation: Implementation of Neural Style Transfer & Neural Doodles from the paper A Neural Algorithm of Artistic Style in Keras 2.0+
• Explainability method:
  • Deep Taylor decomposition: DeepTaylor computes for each neuron a rootpoint, that is close to the input, but which's output value is 0, and uses this difference to estimate the attribution of each neuron recursively.
  • Repository: iNNvestigate library contains implementations for the SmoothGrad, DeConvNet, Guided BackProp, PatternNet, DeepTaylor, PatternAttribution, IntegratedGradients and DeepLIFT.
    
• Source database:
  • ISIC Archive: The ISIC Archive contains over 23k images of skin lesions, labeled as 'benign' or 'malignant'. Those images were used to generate our database.
  • ISIC Archive Downloader: A script to download the ISIC Archive of lesion images
• Previous papers about data augmentation:
  • Data augmentation for improving deep learning in image classification problem, 2018
  • Style transfer-based image synthesis as an efficient regularization technique in deep learning, 2019
• Similar projects:
  • Generating skin lesions with GANs - Beating Melanoma with Deep Learning: letting the data speak
• Other:
  • VGG8 Selected Technical Issues of Deep Neural Networks for Image Classification Purposes prestents the details of VGG8 architecture.