Loss functions for image segmentation

@article{LossOdyssey, title = {Loss Odyssey in Medical Image Segmentation}, journal = {Medical Image Analysis}, volume = {71}, pages = {102035}, year = {2021}, author = {Jun Ma and Jianan Chen and Matthew Ng and Rui Huang and Yu Li and Chen Li and Xiaoping Yang and Anne L. Martel} doi = {https://doi.org/10.1016/j.media.2021.102035}, url = {https://www.sciencedirect.com/science/article/pii/S1361841521000815} }

Take-home message: compound loss functions are the most robust losses, especially for the highly imbalanced segmentation tasks.

Some recent side evidence: the winner in MICCAI 2020 HECKTOR Challenge used DiceFocal loss; the winner and runner-up in MICCAI 2020 ADAM Challenge used DiceTopK loss.

|Date|First Author|Title|Conference/Journal| |---|---|---|---| |20211109|Litao Yu|Distribution-Aware Margin Calibration for Semantic Segmentation in Images (pytorch) | IJCV| |20211013|Pei Wang|Relax and Focus on Brain Tumor Segmentation | MedIA| |20210418|Bingyuan Liu|The hidden label-marginal biases of segmentation losses (pytorch) | arxiv| |20210330|Suprosanna Shit and Johannes C. Paetzold|clDice - a Novel Topology-Preserving Loss Function for Tubular Structure Segmentation (keras and pytorch)|CVPR 2021| |20210325|Attila Szabo, Hadi Jamali-Rad|Tilted Cross Entropy (TCE): Promoting Fairness in Semantic Segmentation|arxiv| |20210318|Xiaoling Hu|Topology-Aware Segmentation Using Discrete Morse Theory arxiv|ICLR 2021| |20210211|Hoel Kervadec|Beyond pixel-wise supervision: semantic segmentation with higher-order shape descriptors|Submitted to MIDL 2021| |20210210|Rosana EL Jurdi|A Surprisingly Effective Perimeter-based Loss for Medical Image Segmentation|Submitted to MIDL 2021| |20201222|Zeju Li|Analyzing Overfitting Under Class Imbalance in Neural Networks for Image Segmentation|TMI| |20210129|Nick Byrne|A Persistent Homology-Based Topological Loss Function for Multi-class CNN Segmentation of Cardiac MRI arxiv| STACOM 2020| |20201019|Hyunseok Seo|Closing the Gap Between Deep Neural Network Modeling and Biomedical Decision-Making Metrics in Segmentation via Adaptive Loss Functions|TMI| |20200929|Stefan Gerl|A Distance-Based Loss for Smooth and Continuous Skin Layer Segmentation in Optoacoustic Images|MICCAI 2020| |20200821|Nick Byrne|A persistent homology-based topological loss function for multi-class CNN segmentation of cardiac MRI arxiv|STACOM| |20200720|Boris Shirokikh|Universal Loss Reweighting to Balance Lesion Size Inequality in 3D Medical Image Segmentation arxiv (pytorch)|MICCAI 2020| |20200708|Gonglei Shi|Marginal loss and exclusion loss for partially supervised multi-organ segmentation (arXiv)|MedIA| |20200706|Yuan Lan|An Elastic Interaction-Based Loss Function for Medical Image Segmentation (pytorch) (arXiv)|MICCAI 2020| |20200615|Tom Eelbode|Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index|TMI| |20200605|Guotai Wang|Noise-robust Dice loss: A Noise-robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions from CT Images (pytorch)|TMI| |202004|J. H. Moltz|Contour Dice coefficient (CDC) Loss: Learning a Loss Function for Segmentation: A Feasibility Study|ISBI| |201912|Yuan Xue|Shape-Aware Organ Segmentation by Predicting Signed Distance Maps (arxiv) (pytorch)|AAAI 2020| |201912|Xiaoling Hu|Topology-Preserving Deep Image Segmentation (paper) (pytorch)|NeurIPS| |201910|Shuai Zhao|Region Mutual Information Loss for Semantic Segmentation (paper) (pytorch)|NeurIPS 2019| |201910|Shuai Zhao|Correlation Maximized Structural Similarity Loss for Semantic Segmentation (paper)|arxiv| |201908|Pierre-AntoineGanaye|Removing Segmentation Inconsistencies with Semi-Supervised Non-Adjacency Constraint (paper) (official pytorch)|Medical Image Analysis| |201906|Xu Chen|Learning Active Contour Models for Medical Image Segmentation (paper) (official-keras)|CVPR 2019| |20190422|Davood Karimi|Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks (paper) (pytorch)|TMI 201907| |20190417|Francesco Caliva|Distance Map Loss Penalty Term for Semantic Segmentation (paper)|MIDL 2019| |20190411|Su Yang|Major Vessel Segmentation on X-ray Coronary Angiography using Deep Networks with a Novel Penalty Loss Function (paper)|MIDL 2019| |20190405|Boah Kim|Multiphase Level-Set Loss for Semi-Supervised and Unsupervised Segmentation with Deep Learning (paper)|arxiv| |201901|Seyed Raein Hashemi|Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection (paper)|IEEE Access| |201812|Hoel Kervadec|Boundary loss for highly unbalanced segmentation (paper), (pytorch 1.0)|MIDL 2019| |201810|Nabila Abraham|A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation (paper) (keras)|ISBI 2019| |201809|Fabian Isensee|CE+Dice: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation (paper)|arxiv| |20180831|Ken C. L. Wong|3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes (paper)|MICCAI 2018| |20180815|Wentao Zhu|Dice+Focal: AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy (arxiv) (pytorch)|Medical Physics| |201806|Javier Ribera|Weighted Hausdorff Distance: Locating Objects Without Bounding Boxes (paper), (pytorch)|CVPR 2019| |201805|Saeid Asgari Taghanaki|Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation (arxiv) (keras)|Computerized Medical Imaging and Graphics| |201709|S M Masudur Rahman AL ARIF|Shape-aware deep convolutional neural network for vertebrae segmentation (paper)|MICCAI 2017 Workshop| |201708|Tsung-Yi Lin|Focal Loss for Dense Object Detection (paper), (code)|ICCV, TPAMI| |20170711|Carole Sudre|Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations (paper)|DLMIA 2017| |20170703|Lucas Fidon|Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks (paper)|MICCAI 2017 BrainLes| |201705|Maxim Berman|The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks (paper), (code)|CVPR 2018| |201701|Seyed Sadegh Mohseni Salehi|Tversky loss function for image segmentation using 3D fully convolutional deep networks (paper)|MICCAI 2017 MLMI| |201612|Md Atiqur Rahman|Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation (paper)|2016 International Symposium on Visual Computing| |201608|Michal Drozdzal|"Dice Loss (without square)" The Importance of Skip Connections in Biomedical Image Segmentation (arxiv)|DLMIA 2016| |201606|Fausto Milletari|"Dice Loss (with square)" V-net: Fully convolutional neural networks for volumetric medical image segmentation (arxiv), (caffe code)|International Conference on 3D Vision| |201605|Zifeng Wu|TopK loss Bridging Category-level and Instance-level Semantic Image Segmentation (paper)|arxiv| |201511|Tom Brosch|"Sensitivity-Specifity loss" Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation (paper) (code)|MICCAI 2015| |201505|Olaf Ronneberger|"Weighted cross entropy" U-Net: Convolutional Networks for Biomedical Image Segmentation (paper)|MICCAI 2015| |201309|Gabriela Csurka|What is a good evaluation measure for semantic segmentation? (paper)|BMVA 2013|

Most of the corresponding tensorflow code can be found here.