MobileNetV3 real-time semantic segmentation

This repository contains the implementation for a dual-path network with mobilenetv3-small backbone. I have used PSP module as the context aggregation block.

Requirements

The Cityscapes dataset, which can be downloaded here.

NOTE: The code has been tested in Ubuntu 18.04, and requirements.txt contains all the nessary packages.

Usage

Train

To train the model, we run train.py python3 train.py --root Cityscapes_root_directory --model_path optional_param, to resume training from a checkpoint.

Evaluate

The trainer, also evaluates the model for every save and logs the results, but if evaluation needs to be done for a particular model, we run evaluate.py

python3 evaluate.py --root Cityscapes_root_directory --model_path saved_model_path_to_evaluate.

Evaluate Server

The evaluateserver.py evaluates the model, and store the segmentation masks in *cityscapesresults* folder created in the root path of the script. This is used for submiting the results to Cityscapes evaluation server.

python3 evaluate_server.py --root Cityscapes_root_directory --model_path saved_model_path_to_evaluate.

Demo

To visulaize the results, we run demo.py.

python3 demo.py --root Cityscapes_root_directory --model_path saved_model_path_to_run_demo.

Demo Single Image

To run inference on a single image, we run demosingle.py. Can run inference to any image given by imgpath.

python3 demo_single.py --model_path saved_model_path_to_run_demo. --img_path optional_param, default is images/demo.png.

Result

The FPS metrics are evaluated on a RTX2070. And evaluation was done by single scale input images.

• Cityscapes

| Config | Params(M) | RES | FLOPS (G) | FP32(fps) | FP16(fps)| train-split | mIoU - val | mIoU - test | model | | :-------: | :--: | :----: | :----: | :---: | :-------:| :------: | :------: | :------: | :------: |
| MV3-Small + PSP + FFM | 1.74 |2048x1024 | 11.63 | 40.85 | 54.50 | train | 0.662 | 0.6388 | file (6.86MB) | | MV3-Small + PSP + FFM | 1.74 |1024x512 | 2.91 | 78.79 | 71.74 | train | 0.615 | - | file (6.86MB) | | MV3-Small + PSP + FFM | 1.74 |2048x1024 | 11.63 | 40.85 | 54.50 | train + val | 0.717 | 0.6559 | file (6.86MB) | | MV3-Small + PSP + FFM | 1.74 |1024x512 | 2.91 | 78.79 | 71.74 | train + val | 0.646 | - | file (6.86MB) |

Note: Params and FLOPS are got using torchstat.

To Do

• [ ] Add mobilenetv3 large
• [ ] Improve performance.
• [ ] Add more configurations support.