objects-compositional-generalization
Official code for the paper "Provable Compositional Generalization for Object-Centric Learning" (ICLR 2024, oral)
https://github.com/brendel-group/objects-compositional-generalization
Science Score: 54.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (14.2%) to scientific vocabulary
Repository
Official code for the paper "Provable Compositional Generalization for Object-Centric Learning" (ICLR 2024, oral)
Basic Info
- Host: GitHub
- Owner: brendel-group
- License: mit
- Language: Jupyter Notebook
- Default Branch: submission
- Homepage: https://openreview.net/forum?id=7VPTUWkiDQ
- Size: 32.4 MB
Statistics
- Stars: 14
- Watchers: 4
- Forks: 2
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
Provable Compositional Generalization for Object-Centric Learning [ICLR 2024]
Official code for the paper Provable Compositional Generalization for Object-Centric Learning (openreview, arXiv).
We formalize compositional generalization as an identifiability problem for a latent variable model where objects are represented by latent slots. Here, compositional generalization requires two things:
- Identifying the ground-truth latent slots in-distribution (blue).
- Generalizing this behavior to unseen, out-of-distribution slot combinations (grey).
From an empirical side, the main contribution is the compositional consisteny loss outlined below:
Environment Setup
This code was tested for Python 3.10.
You can start by cloning the repository:
bash
git clone git@github.com:brendel-group/objects-compositional-generalization.git
cd objects-compositional-generalization
Then, set up your environment by choosing one of the following methods:
Option 1: Installing Dependencies Directly
```bash pip install -r requirements.txt ```Or, alternatively, you can use Docker:
Option 2: Building a Docker Image
Build and run a Docker container using the provided Dockerfile: ```bash docker build -t object_centric_ood . docker-compose up ```Data Generation
🔗 For understanding how the data looks and to play with the data generation, please refer to the notebooks/0. Sprite-World Dataset Example.ipynb notebook.
🔗 For the actual data generation, please refer to the notebooks/1. Data Generation.ipynb notebook. The folder used for saving the dataset at this point will be used for training and evaluation.
Training and Evaluation
Training
To train the model, run the following command:
bash
python main.py --dataset_path "/path/from/previous/step" --model_name "SlotAttention" --num_slots 2 --epochs 400 --use_consistency_loss True
For complete details on the parameters, please refer to the main.py file.
You can find some example commands for training below:
Different Training Setups
-Training SlotAttention
Training vanilla SlotAttention with 2 slots: ```bash python main.py --dataset_path "/path/from/previous/step" --model_name "SlotAttention" --num_slots 2 --use_consistency_loss False ``` Training vanilla SlotAttention with 2 slots and consistency loss: ```bash python main.py --dataset_path "/path/from/previous/step" --model_name "SlotAttention" --num_slots 2 --use_consistency_loss True --consistency_ignite_epoch 150 ``` Training SlotAttention with 2 slots, fixed SoftMax and sampling: ```bash python main.py --dataset_path "/path/from/previous/step" --model_name "SlotAttention" --num_slots 2 --use_consistency_loss True --consistency_ignite_epoch 150 --softmax False --sampling False ```Training AE Model
Training vanilla autoencoder with 2 slots:
bash
python main.py --dataset_path "/path/from/previous/step" --model_name "SlotMLPAdditive" --epochs 300 --num_slots 2 -n_slot_latents 6 --use_consistency_loss False
Training vanilla autoencoder with 2 slots and consistency loss:
bash
python main.py --dataset_path "/path/from/previous/step" --model_name "SlotMLPAdditive" --epochs 300 --num_slots 2 -n_slot_latents 6 --use_consistency_loss True --consistency_ignite_epoch 100
Evaluation
Evaluation can be done using the evaluate.py script and closely follows the procedure and metrics used in the training script. The main difference is in calculating the compositional contrast (note: it might cause OOM issues, thus is calculated only for the AE model).
Here is an example command for evaluation:
bash
python src/evaluation.py --dataset_path "/path/from/previous/step" --model_path "checkpoints/SlotMLPAdditive.pt" --model_name "SlotMLPAdditive" --n_slot_latents 6
Misc
🔗 `notebooks/2. Decoder Optimality.ipynb` shows how the experiment described in Appendix B.2 is conducted. Note that the paths to models' checkpoints and exact data splits are omitted, thus it serves more of an illustrative purpose.Citation
If you make use of this code in your own work, please cite our paper:
bibtex
@inproceedings{wiedemer2024provable,
title={Provable Compositional Generalization for Object-Centric Learning},
author={Thadd{\"a}us Wiedemer and Jack Brady and Alexander Panfilov and Attila Juhos and Matthias Bethge and Wieland Brendel},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=7VPTUWkiDQ}
}
Owner
- Name: brendel-group
- Login: brendel-group
- Kind: organization
- Repositories: 9
- Profile: https://github.com/brendel-group
Citation (CITATION.cff)
cff-version: 1.2.0
message: "If you use this repository in your research, please cite it as below."
authors:
- family-names: "Wiedemer"
given-names: "Thaddäus"
orcid: "https://orcid.org/0009-0003-6280-0804"
- family-names: "Brady"
given-names: "Jack"
- family-names: "Panfilov"
given-names: "Alexander"
- family-names: "Juhos"
given-names: "Attila"
- family-names: "Bethge"
given-names: "Matthias"
- family-names: "Brendel"
given-names: "Wieland"
title: "My Research Software"
version: 1.0.0
doi: 10.5281/zenodo.1234
date-released: 2024.02.19
url: "https://github.com/brendel-group/objects-compositional-generalization"
preferred-citation:
type: conference-paper
authors:
- family-names: "Wiedemer"
given-names: "Thaddäus"
orcid: "https://orcid.org/0009-0003-6280-0804"
- family-names: "Brady"
given-names: "Jack"
- family-names: "Panfilov"
given-names: "Alexander"
- family-names: "Juhos"
given-names: "Attila"
- family-names: "Bethge"
given-names: "Matthias"
- family-names: "Brendel"
given-names: "Wieland"
url: "https://openreview.net/forum?id=7VPTUWkiDQ"
collection-title: "The Twelfth International Conference on Learning Representations"
title: "Provable Compositional Generalization for Object-Centric Learning"
year: 2024
GitHub Events
Total
Last Year
Issues and Pull Requests
Last synced: about 2 years ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- nvidia/cuda 12.2.0-devel-ubuntu20.04 build
- object_centric_ood latest
- Pillow ==9.5.0
- dm_env *
- lion-pytorch *
- matplotlib *
- numpy *
- pandas *
- scikit-learn *
- scipy *
- seaborn *
- torch *
- torchaudio *
- torchmetrics *
- torchvision *
- tqdm *
- wandb *