https://github.com/ai-forever/easyportrait

EasyPortrait - Face Parsing and Portrait Segmentation Dataset

Keywords

arxiv-papers dataset deep-learning face-parsing image-dataset image-segmentation mmdetection opensource portrait-segmentation pytorch

Last synced: 6 months ago · JSON representation

Repository

EasyPortrait - Face Parsing and Portrait Segmentation Dataset

Basic Info

Host: GitHub
Owner: ai-forever
Language: Python
Default Branch: main
Homepage: https://arxiv.org/abs/2304.13509
Size: 2.95 MB

Statistics

Stars: 28
Watchers: 1
Forks: 0
Open Issues: 0
Releases: 0

Fork of hukenovs/easyportrait

Topics

arxiv-papers dataset deep-learning face-parsing image-dataset image-segmentation mmdetection opensource portrait-segmentation pytorch

Created almost 3 years ago · Last pushed almost 3 years ago

https://github.com/ai-forever/easyportrait/blob/main/

![easyportrait](images/main.jpg)
# EasyPortrait - Face Parsing and Portrait Segmentation Dataset
We introduce a large-scale image dataset **EasyPortrait** for portrait segmentation and face parsing. Proposed dataset can be used in several tasks, such as background removal in conference applications, teeth whitening, face skin enhancement, red eye removal or eye colorization, and so on.

EasyPortrait dataset size is about **91.78GB**, and it contains **40,000** RGB images (~38.3K FullHD images) with high quality annotated masks. This dataset is divided into training set, validation set and test set by subject `user_id`. The training set includes 30,000 images, the validation set includes 4,000 images, and the test set includes 6,000 images.

For more information see our paper [EasyPortrait  Face Parsing and Portrait Segmentation Dataset](https://arxiv.org/abs/2304.13509).

##  Changelog
 - **`2023/11/13`**: We release EasyPortrait 2.0. 
   - **40,000** RGB images (~38.3K FullHD images)
   - Added diversity by region, race, human emotions and lighting conditions
   - The data was further cleared and new ones were added
   - Train/val/test split: (30,000) **75%** / (4,000) **10%** / (6,000) **15%** by subject `user_id`
   - Multi-gpu training and testing
   - Added new models for face parsing and portrait segmentation
   - Dataset size is **91.78GB**
   - **13,705** unique persons
 - **`2023/02/23`**: EasyPortrait (Initial Dataset) 
   - Dataset size is **26GB**
   - **20,000** RGB images (~17.5K FullHD images) with **9** classes annotated
   - Train/val/test split: (14,000) **70%** / (2,000) **10%** / (4,000) **20%** by subject `user_id`
   - **8,377** unique persons
   

 Old EasyPortrait dataset is also available into branch `EasyPortrait_v1`!
## Downloads

| Link                                                                                                          | Size  |
|---------------------------------------------------------------------------------------------------------------|-------|
| [`images`](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/v2/images.zip)           | 91.8 GB  |
| [`annotations`](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/v2/annotations.zip) | 657.1 MB  |
| [`meta`](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/v2/meta.zip) | 1.9 MB  |
| [`train set`](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/v2/train.zip)         | 68.3 GB |
| [`validation set`](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/v2/val.zip)      | 10.7 GB  |
| [`test set`](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/v2/test.zip)           | 12.8 GB  |

Also, you can download EasyPortrait dataset from [Kaggle](https://www.kaggle.com/datasets/kapitanov/easyportrait).

### Structure
```
.
 images.zip
    train/         # Train set: 30k
    val/           # Validation set: 4k
    test/          # Test set: 6k
 annotations.zip
    train/
    val/
    test/
 meta.zip       # Meta-information (width, height, brightness, imhash, user_id)
...
```

## Models
We provide some pre-trained models as the baseline for portrait segmentation and face parsing. We use mean Intersection over Union (mIoU) as the main metric.

#### Portrait segmentation:
| Model Name                                     | Parameters (M) | Input shape | mIoU      |
|------------------------------------------------|----------------|-------------|-----------|
| [BiSeNet-V2](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/bisenet-ps.pth)                       | 56.5          | 384 x 384 | 97.95     |
| [DANet](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/danet-ps.pth)                    | 190.2           | 384 x 384   | 98.63    |
| [DeepLabv3](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/deeplabv3-ps.pth)                    | 260           | 384 x 384   | 98.63    |
| [ExtremeC3Net](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/extremenet-ps.pth)                    | 0.15           | 384 x 384   | 96.54    |
| [Fast SCNN](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fast_scnn-ps.pth)               | 6.13          | 384 x 384 | 97.64 |
| [FCN + MobileNetv2](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fcn-ps.pth)               | 31.17           | 384 x 384 | 98.19 |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-ps-1024.pth)                 | 108.91           | 1024  1024   | 98.54    |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-ps-512.pth)               | 108.91            | 512  512 | 98.64 |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-ps.pth)               | 108.91          | 384 x 384 | 98.64 |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-ps-224.pth)               | 108.91          | 224  224 | 98.31 |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-ps-1024.pth)  | 14.9          | 1024  1024 |98.74     |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-ps-512.pth)            | 14.9           | 512  512   | 98.66      |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-ps.pth)          | 14.9           | 384 x 384 | 98.61     |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-ps-224.pth)            | 14.9           | 224  224   | 98.17     |
| [SINet](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/sinet-ps.pth)                    | 0.13          | 384 x 384   | 93.32  |


#### Face parsing:
| Model Name                                     | Parameters (M) | Input shape | mIoU      |
|------------------------------------------------|----------------|-------------|-----------|
| [BiSeNet-V2](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/bisenet-fp.pth)                       | 56.5          | 384 x 384 | 76.72     |
| [DANet](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/danet-fp.pth)                    | 190.2           | 384 x 384   | 79.3    |
| [DeepLabv3](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/deeplabv3-fp.pth)                    | 260           | 384 x 384   | 79.11    |
| [EHANet](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/ehanet-fp.pth)                    | 44.81          | 384 x 384   | 72.56    |
| [Fast SCNN](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fast_scnn-fp.pth)               | 6.13          | 384 x 384 | 67.56|
| [FCN + MobileNetv2](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fcn-fp.pth)               | 31.17           | 384 x 384 | 75.23 |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-fp-1024.pth)                 | 108.91           | 1024  1024   | 85.37   |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-fp-512.pth)               | 108.91            | 512  512 | 83.33 |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-fp.pth)               | 108.91          | 384 x 384 | 81.83  |
| [FPN + ResNet50](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/fpn-fp-224.pth)               | 108.91          | 224  224 | 75.6 |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-fp-1024.pth)  | 14.9          | 1024  1024 |85.42     |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-fp-512.pth)            | 14.9           | 512  512   | 83.19      |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-fp.pth)          | 14.9           | 384 x 384 | 81.38    |
| [SegFormer-B0](https://rndml-team-cv.obs.ru-moscow-1.hc.sbercloud.ru/datasets/easyportrait/experiments/models/segformer-fp-224.pth)            | 14.9           | 224  224   | 74.83     |


## Annotations

Annotations are presented as 2D-arrays, images in `*.png` format with several classes:

| Index | Class      |
|------:|:-----------|
|     0 | BACKGROUND |
|     1 | PERSON     |
|     2 | SKIN       |
|     3 | LEFT_BROW  |
|     4 | RIGHT_BROW |
|     5 | LEFT_EYE   |
|     6 | RIGHT_EYE  |
|     7 | LIPS       |
|     8 | TEETH      |

Also, we provide some additional meta-information for dataset in `annotations/meta.zip` file:

|    | image_name | user_id | height | width | set | brightness |
|---:|:--------------|:--------|:----------|------:|-------:|-----------:|
|  0 | a753e021-...  | 56...   | 720    |  960 |   train |        126 |
|  1 | 4ff04492-...  | ba...   | 1920    |  1440 |   test |        173 |
|  2 | e8934c99-...  | 1d...   | 1920    |  1440 |   val |        187 |

where:
- `image_name` - image file name without extension
- `user_id` - unique anonymized user ID
- `height` - image height
- `width` - image width
- `brightness` - image brightness
- `set` - "train", "test" or "val" for train / test / val subsets respectively

## Images
![easyportrait](images/data.jpg)


## Training, Evaluation and Testing on EasyPortrait

>The code is based on [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) with 0.30.0 version.

Models were trained and evaluated on 8 NVIDIA V100 GPUs with CUDA 11.2.

For installation process follow the instructions [here](https://github.com/open-mmlab/mmsegmentation/blob/v0.30.0/docs/en/get_started.md#installation) and use the **requirements.txt** file in our repository.


  Training

  For single GPU mode:
  ```console
  python ./pipelines/tools/train.py ./pipelines/local_configs/easy_portrait_experiments//.py --gpu-id 
  ```

  For distributed training mode:
  ```console
  ./pipelines/tools/dist_train.sh ./pipelines/local_configs/easy_portrait_experiments//.py 
  ```



  Evaluation

  For single GPU mode:
  ```console
  python ./pipelines/tools/test.py    --gpu-id  --eval mIoU
  ```

  For distributed evaluation mode:
  ```console
  ./pipelines/tools/dist_test.sh     --eval mIoU
  ```



  Run demo

  ```console
  python ./pipelines/demo/image_demo.py    --palette=easy_portrait --out-file=
  ```


## Authors and Credits
- [Alexander Kapitanov](https://www.linkedin.com/in/hukenovs)
- [Karina Kvanchiani](https://www.linkedin.com/in/kvanchiani)
- [Elizaveta Petrova](https://www.linkedin.com/in/kleinsbotle)
- [Karen Efremyan](https://www.linkedin.com/in/befozg)
- [Alexander Sautin](https://www.linkedin.com/in/befozg/alexander-sautin-b5039623b)

## Links
- [arXiv](https://arxiv.org/abs/2304.13509)
- [Paperswithcode](https://paperswithcode.com/dataset/easyportrait)
- [Kaggle](https://www.kaggle.com/datasets/kapitanov/easyportrait)
- [Habr](https://habr.com/ru/companies/sberdevices/articles/731794/)
- [Gitlab](https://gitlab.aicloud.sbercloud.ru/rndcv/easyportrait)

## Citation
You can cite the paper using the following BibTeX entry:

    @article{EasyPortrait,
        title={EasyPortrait - Face Parsing and Portrait Segmentation Dataset},
        author={Kapitanov, Alexander and Kvanchiani, Karina and Kirillova Sofia},
        journal={arXiv preprint arXiv:2304.13509},
        year={2023}
    }

## License

This work is licensed under a variant of Creative Commons Attribution-ShareAlike 4.0 International License.

Please see the specific [license](https://github.com/hukenovs/easyportrait/blob/master/license/en_us.pdf).

Owner

Name: AI Forever
Login: ai-forever
Kind: organization
Location: Armenia

Repositories: 60
Profile: https://github.com/ai-forever

Creating ML for the future. AI projects you already know. We are non-profit organization with members from all over the world.

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/ai-forever/easyportrait

Science Score: 10.0%

Keywords

Repository

Basic Info

Statistics

Topics

https://github.com/ai-forever/easyportrait/blob/main/

Owner

GitHub Events

Total

Last Year