https://github.com/aim-uofa/oir
[ICLR 2024] Official PyTorch/Diffusers implementation of "Object-aware Inversion and Reassembly for Image Editing"
Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org, scholar.google -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (14.5%) to scientific vocabulary
Keywords
Repository
[ICLR 2024] Official PyTorch/Diffusers implementation of "Object-aware Inversion and Reassembly for Image Editing"
Basic Info
- Host: GitHub
- Owner: aim-uofa
- Language: Python
- Default Branch: main
- Homepage: https://aim-uofa.github.io/OIR-Diffusion/
- Size: 167 MB
Statistics
- Stars: 85
- Watchers: 9
- Forks: 2
- Open Issues: 0
- Releases: 0
Topics
Metadata Files
README.md
Object-aware Inversion and Reassembly for Image Editing
Zhen Yang*
·
Ganggui Ding*
·
Wen Wang*
·
Hao Chen*
·
Bohan Zhuang†
·
Chunhua Shen*
*Zhejiang University †Monash University
Setup
This code was tested with Python 3.9, Pytorch 2.0.1 using pre-trained models through huggingface / diffusers. Specifically, we implemented our method over Stable Diffusion 1.4. Additional required packages are listed in the requirements file. The code was tested on a NVIDIA GeForce RTX 3090 but should work on other cards.
Getting Started
- Download OIR-Bench.
- Create the environment and install the dependencies by running:
conda create -n oir python=3.9 conda activate oir pip install -r requirements.txt - Change the basic_config.py in configs/, change the model path and hyperparameters.
- Modify multiobjectedit.yaml or singleobjectedit.yaml in configs/ according to multi_object.yaml and single_object.yaml in OIR-Bench/.
- Run singleobjectedit.py (Search Metric in paper) or multiobjectedit.py (OIR in paper) to implement image editing.
- Option: Adjust reassembly_step and repeat the above process to get better results.
TODO
- Use prompt_change as dict's key may lead to error.
- Different editing pairs' masks mustn't have overlap.
- Search metric can be an ensemble learning tool. For example, we can use pnp, p2p, OIR ... method to edit an image and we can use search metric to select the optimal editing result.
- We can also use the method in TODO 3 to build a high quality dataset to train instruct-based image editing method.
- Deploy our method on different foundation model (SDXL, LCM ...)
Results
OIR results
|
Visualization of the search metric
|
Acknowlegment
Many thanks for the generous help in building the project website from Minghan Li.
License
For non-commercial academic use, this project is licensed under the 2-clause BSD License. For commercial use, please contact Chunhua Shen.
BibTeX
BibTeX
@inproceedings{yang2024objectaware,
title = {Object-Aware Inversion and Reassembly for Image Editing},
author = {Zhen Yang and Ganggui Ding and Wen Wang and Hao Chen and Bohan Zhuang and Chunhua Shen},
booktitle = {The Twelfth International Conference on Learning Representations},
year = {2024},
url = {https://openreview.net/forum?id=dpcVXiMlcv}
}
Owner
- Name: Advanced Intelligent Machines (AIM)
- Login: aim-uofa
- Kind: organization
- Location: China
- Repositories: 23
- Profile: https://github.com/aim-uofa
A research team at Zhejiang University, focusing on Computer Vision and broad AI research ...
GitHub Events
Total
- Issues event: 5
- Watch event: 11
- Issue comment event: 4
- Fork event: 1
Last Year
- Issues event: 5
- Watch event: 11
- Issue comment event: 4
- Fork event: 1
Committers
Last synced: 10 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| ZhenYangCS | z****s@g****m | 35 |
| C Shen | 1****n | 4 |
| yangzhen | y****n@y****l | 1 |
Issues and Pull Requests
Last synced: 5 months ago
All Time
- Total issues: 5
- Total pull requests: 0
- Average time to close issues: 13 days
- Average time to close pull requests: N/A
- Total issue authors: 5
- Total pull request authors: 0
- Average comments per issue: 1.4
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 3
- Pull requests: 0
- Average time to close issues: 12 days
- Average time to close pull requests: N/A
- Issue authors: 3
- Pull request authors: 0
- Average comments per issue: 1.33
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- SteveJMao (1)
- tian-2024 (1)
- XuwuChen443 (1)
- HUAFOR (1)
- RainbowZhuu (1)