Image Inpainting

This repository is a paper list of image inpainting. Feel free to contact me (yutao666@mail.ustc.edu.cn) if you find any interesting paper about inpainting that I missed. I would greatly appreciate it : )

Highlights

• 2023-04-10: We develop an image inapinting model called Inpaint Anything which can remove, fill and replace ANYTHING.
• 2022-10-12: Our Region Normalization inpainting work achieves 100 citations.

Traditional Methods

Year|Proceeding|Title|Tag --|:--:|:--:|:--: 2000|SIGGRAPH 2000|Image Inpainting [pdf] |Diffusion-based 2001|TIP 2001|Filling-in by joint interpolation of vector fields and gray levels [pdf]|Diffusion-based 2001|CVPR 2001|Navier-stokes, uid dynamics, and image and video inpainting [pdf]| 2001|SIGGRAPH 2001|Image Quilting for Texture Synthesis and Transfer [pdf] | 2001|SIGGRAPH 2001|Synthesizing Natural Textures [pdf]| 2002|EJAM 2002|Digital inpainting based on the mumfordshaheuler image model [pdf] |Diffusion-based 2003|CVPR 2003| Object removal by exemplar-based inpainting [pdf]| 2003|TIP 2003|Simultaneous structure and texture image inpainting [pdf]|Diffusion-based 2003|TIP 2003|Structure and Texture Filling-In of Missing Image Blocks in Wireless Transmission and Compression Applications [pdf]| 2003|ICCV 2003|Learning How to Inpaint from Global Image Statistics [pdf]|Diffusion-based 2003|TOG 2003|Fragment-based image completion [pdf] | Patch-based 2004|TIP 2004|Region Filling and Object Removal by Exemplar-Based Image Inpainting [pdf]|Patch-based; Inpainting order 2004|TPAMI 2004|Space-Time Video Completion [pdf] | 2005|SIGGRAPH 2005|Image Completion with Structure Propagation [pdf]|Patch-based 2006|ISCS 2006|Image Compression with Structure Aware Inpainting [pdf]| 2007|TOG 2007| Scene completion using millions of photographs [pdf]|
2007|CSVT 2007|Image Compression With Edge-Based Inpainting [pdf]|Diffusion-based 2008|CVPR 2008|Summarizing Visual Data Using Bidirectional Similarity [pdf]| 2009|SIGGRAPH 2009|PatchMatch: a randomized correspondence algorithm for structural image editing [pdf] |Patch-based 2010| TIP 2010|Image inpainting by patch propagation using patch sparsity [pdf] |Patch-based 2011|FTCGV 2011|Structured learning and prediction in computer vision [pdf]| 2011|ICIP 2011|Examplar-based inpainting based on local geometry [pdf]|Inpainting order 2012|TOG 2012|Combining inconsistent images using patch-based synthesis[pdf]|Patch-based 2014|TOG 2014|Image completion using Planar structure guidance [pdf]|Patch-based 2014|TVCG 2014|High-Quality Real-Time Video Inpainting with PixMix [pdf]|Video 2014|SIAM 2014|Video inpainting of complex scenes [pdf]|Video 2015|TIP 2015|Annihilating Filter-Based Low-Rank Hankel Matrix Approach for Image Inpainting [pdf]| 2015|TIP 2015|Exemplar-Based Inpainting: Technical Review and New Heuristics for Better Geometric Reconstructions [pdf]| 2016|TOG 2016|Temporally coherent completion of dynamic video [pdf] | Video

Deep-learning-based Methods

Year|Proceeding|Title|Tag --|:--:|:--:|:-- 2012|NIPS 2012| Image denoising and inpainting with deep neural networks [pdf]| 2014|GCPR 2014|Mask-specific inpainting with deep neural networks [pdf]| 2014|NIPS 2014|Deep Convolutional Neural Network for Image Deconvolution [pdf]| 2015|NIPS 2015|Shepard Convolutional Neural Networks [pdf] [code]| 2016|CVPR 2016|Context Encoders: Feature Learning by Inpainting [pdf] [code]| 2016|SIGGRAPH 2016|High-resolution multi-scale neural texture synthesis [pdf]| 2017|CVPR 2017|Semantic image inpainting with deep generative models [pdf] [code]| 2017|CVPR 2017|High-resolution image inpainting using multi-scale neural patch synthesis [pdf] [code]| 2017|CVPR 2017|Generative Face Completion [pdf] [code]| 2017|SIGGRAPH 2017|Globally and Locally Consistent Image Completion [pdf] [code]| 2018|CVPR 2018|Generative Image Inpainting with Contextual Attention [pdf] [code]| 2018|CVPR 2018|Natural and Effective Obfuscation by Head Inpainting [pdf]| 2018|CVPR 2018|Eye In-Painting With Exemplar Generative Adversarial Networks [pdf] [code]| 2018|CVPR 2018|UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition [pdf]| 2018|CVPR 2018|Disentangling Structure and Aesthetics for Style-aware Image Completion [pdf]| 2018|ECCV 2018|Image Inpainting for Irregular Holes Using Partial Convolutions [pdf] [code]| 2018|ECCV 2018| Contextual-based Image Inpainting: Infer, Match, and Translate [pdf]| 2018|ECCV 2018|Shift-Net: Image Inpainting via Deep Feature Rearrangement [pdf] [code]| 2018|NIPS 2018|Image Inpainting via Generative Multi-column Convolutional Neural Networks [pdf] [code]| 2018|TOG 2018|Faceshop: Deep sketch-based face image editing [pdf]| 2018|ACM MM 2018|Structural inpainting [pdf] | 2018|ACM MM 2018|Semantic Image Inpainting with Progressive Generative Networks [pdf] [code]| 2018|BMVC 2018|SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting [pdf]| 2018|BMVC 2018|MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesi [pdf] [code] | 2018|ACCV 2018|Face Completion iwht Semantic Knowledge and Collaborative Adversarial Learning [pdf]| 2018|ICASSP 2018|Edge-Aware Context Encoder for Image Inpainting [pdf]| 2018|ICPR 2018|Deep Structured Energy-Based Image Inpainting [pdf] [code]| 2018|AISTATS 2019|Probabilistic Semantic Inpainting with Pixel Constrained CNNs [pdf]| 2019|ICRA 2019| Empty Cities: Image Inpainting for a Dynamic-Object-Invariant Space [pdf] | 2019|AAAI 2019|Video Inpainting by Jointly Learning Temporal Structure and Spatial Details [pdf]| Video 2019|CVPR 2019| Pluralistic Image Completion [pdf] [code] [project]| 2019|CVPR 2019| Deep Reinforcement Learning of Volume-guided Progressive View Inpainting for 3D Point Scene Completion from a Single Depth Image [pdf]| 2019|CVPR 2019|Foreground-aware Image Inpainting [pdf] | 2019|CVPR 2019 |Privacy Protection in Street-View Panoramas using Depth and Multi-View Imagery [pdf] | 2019|CVPR 2019|Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting [pdf] [code] 2019|CVPR 2019|Deep Flow-Guided Video Inpainting [pdf] [project]| Video 2019|CVPR 2019|Deep video inapinting [pdf]|Video 2019|CVPRW 2019 |VORNet: Spatio-temporally Consistent Video Inpainting for Object Removal [pdf] | Video 2019|TNNLS 2019|PEPSI++: Fast and Lightweight Network for Image Inpainting [pdf]| 2019|IJCAI 2019|MUSICAL: Multi-Scale Image Contextual Attention Learning for Inpainting [pdf] | 2019|IJCAI 2019|Generative Image Inpainting with Submanifold Alignment [pdf] | 2019|ACM MM 2019| Progressive Image Inpainting with Full-Resolution Residual Network [pdf] [code]| 2019|ACM MM 2019| Deep Fusion Network for Image Completion [pdf] [code]| 2019|ACM MM 2019| GAIN: Gradient Augmented Inpainting Network for Irregular Holes [pdf] | 2019|ACM MM 2019| Single-shot Semantic Image Inpainting with Densely Connected Generative Networks [pdf] | 2019|ICCVW 2019|EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning [pdf] [code]| 2019|ICCV 2019|Coherent Semantic Attention for Image Inpainting [pdf] [code]| 2019|ICCV 2019|StructureFlow: Image Inpainting via Structure-aware Appearance Flow [pdf] [code] | 2019|ICCV 2019|Progressive Reconstruction of Visual Structure for Image Inpainting [pdf] [code] | 2019|ICCV 2019| Localization of Deep Inpainting Using High-Pass Fully Convolutional Network [pdf]| 2019|ICCV 2019| Image Inpainting with Learnable Bidirectional Attention Maps [pdf] [code] | 2019|ICCV 2019|Free-Form Image Inpainting with Gated Convolution [pdf] [project]| 2019|ICCV 2019| FiNet: Compatible and Diverse Fashion Image Inpainting [pdf] | Fashion 2019|ICCV 2019|SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color [pdf] [code] | Face 2019|ICCV 2019| Human Motion Prediction via Spatio-Temporal Inpainting [pdf] [code] | Motion 2019|ICCV 2019| Copy-and-Paste Networks for Deep Video Inpainting [pdf] [code]| Video 2019|ICCV 2019| Onion-Peel Networks for Deep Video Completion [pdf] [code]| Video 2019|ICCV 2019 |Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN [pdf] [code] | Video 2019|ICCV 2019| An Internal Learning Approach to Video Inpainting [pdf] | Video 2019|ICCV 2019| Vision-Infused Deep Audio Inpainting [pdf] [code] | Audio 2019|AAAI 2020| Region Normalization for Image Inpainting [pdf] [code] | 2019|AAAI 2020| Learning to Incorporate Structure Knowledge for Image Inpainting [pdf] [code] | 2020|CVPR 2020| Prior Guided GAN Based Semantic Inpainting [pdf] | 2020|CVPR 2020| UCTGAN: Diverse Image Inpainting based on Unsupervised Cross-Space Translation [pdf] | 2020|CVPR 2020| Recurrent Feature Reasoning for Image Inpainting [pdf] [code] | 2020|CVPR 2020| Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting [pdf] [code] | 2020|CVPR 2020| 3D Photography using Context-aware Layered Depth Inpainting [pdf] [code] | 2020|CVPR 2020| Learning Oracle Attention for High-fidelity Face Completion [pdf] | 2020|ECCV 2020| Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations [pdf] [code] | 2020|ECCV 2020| Short-Term and Long-Term Context Aggregation Network for Video Inpainting | Video 2020|ECCV 2020| Learning Object Placement by Inpainting for Compositional Data Augmentation | 2020|ECCV 2020| Learning Joint Spatial-Temporal Transformations for Video Inpainting [pdf] [code] | Video 2020|ECCV 2020| High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling [pdf] | 2020|ECCV 2020| DVI: Depth Guided Video Inpainting for Autonomous Driving [pdf] [code] | Video 2020|ECCV 2020| VCNet: A Robust Approach to Blind Image Inpainting [pdf] | 2020|ECCV 2020| Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes [pdf] | 2021|WACV 2021| Hyperrealistic Image Inpainting with Hypergraphs [pdf] [code]| 2021|CVPR 2021| Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE [pdf] [code]| 2021|CVPR 2021| Image Inpainting with External-internal Learning and Monochromic Bottleneck [pdf] [code]| 2021|CVPR 2021| PD-GAN: Probabilistic Diverse GAN for Image Inpainting [pdf] [code]| 2021|CVPR 2021| Image Inpainting Guided by Coherence Priors of Semantics and Textures [pdf] | 2021|CVPR 2021| FaceInpainter: High Fidelity Face Adaptation to Heterogeneous Domains [pdf] | 2021|CVPR 2021| TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations [pdf] [code]| 2021|CVPR 2021| Prior Based Human Completion [pdf] | Human completion 2021|IJCAI 2021| Context-Aware Image Inpainting with Learned Semantic Priors [pdf] [code]| 2021|IJCAI 2021| Noise Doesnt Lie: Towards Universal Detection of Deep Inpainting [pdf] | Inpainting detection 2021|TCSVT 2021| IID-Net: Image Inpainting Detection via Neural Architecture Search and Attention [pdf] [code] | Inpainting detection 2021|WWW 2021| Progressive Semantic Reasoning for Image Inpainting [pdf] [code]| 2021|ICCV 2021| Occlusion-Aware Video Object Inpainting [pdf] | Video 2021|ICCV 2021| Internal Video Inpainting by Implicit Long-range Propagation [pdf] [code]| Video 2021|ICCV 2021| Distillation-Guided Image Inpainting [pdf] | 2021|ICCV 2021| Frequency-Aware Spatiotemporal Transformers for Video Inpainting Detection [pdf] | Inpainting detection 2021|ICCV 2021| SLIDE: Single Image 3D Photography With Soft Layering and Depth-Aware Inpainting [pdf] [project] | 2021|ICCV 2021| FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting [pdf] [code] | Video 2021|ICCV 2021| WaveFill: A Wavelet-Based Generation Network for Image Inpainting [pdf] | 2021|ICCV 2021| CR-Fill: Generative Image Inpainting With Auxiliary Contextual Reconstruction [pdf] [code] | 2021|ICCV 2021| Learning a Sketch Tensor Space for Image Inpainting of Man-Made Scenes [pdf] [project] | 2021|ICCV 2021| Parallel Multi-Resolution Fusion Network for Image Inpainting [pdf] | 2021|ICCV 2021| Flow-Guided Video Inpainting With Scene Templates [pdf] | 2021|ICCV 2021| High-Fidelity Pluralistic Image Completion With Transformers [pdf] [project]| 2021|ICCV 2021| Learning High-Fidelity Face Texture Completion Without Complete Face Texture [pdf] | Face 2021|WACV 2022| Resolution-robust Large Mask Inpainting with Fourier Convolutions [pdf] [code] | 2022|CVPR 2022| Dual-path Image Inpainting with Auxiliary GAN Inversion | 2022|CVPR 2022| MAT: Mask-Aware Transformer for Large Hole Image Inpainting [pdf] [code] | 2022|CVPR 2022| Bridging Global Context Interactions for High-Fidelity Image Completion [pdf] [code] | 2022|CVPR 2022| Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding [pdf] [code] | 2022|CVPR 2022| Reduce Information Loss in Transformers for Pluralistic Image Inpainting | 2022|CVPR 2022| MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting [pdf] [code] | 2022|CVPR 2022| RePaint: Inpainting using Denoising Diffusion Probabilistic Models [pdf] [code] | 2022|CVPR 2022| DLFormer:Discrete Latent Transformer for Video Inpainting | Video 2022|CVPR 2022| The DEVIL is in the Details: A Diagnostic Evaluation Benchmark for Video Inpainting [pdf] [code] | Video 2022|CVPR 2022| Towards An End-to-End Framework for Flow-Guided Video Inpainting [pdf] [code] | Video 2022|CVPR 2022| Inertia-Guided Flow Completion and Style Fusion for Video Inpainting | Video 2022|CVPR 2022| DLFormer:Discrete Latent Transformer for Video Inpainting | Video

To be updated...