Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/compvis/distilldift

[WACV 2025] DistillDIFT: Distillation of Diffusion Features for Semantic Correspondence

https://github.com/compvis/distilldift

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.5%) to scientific vocabulary
Last synced: 11 months ago · JSON representation

Repository

[WACV 2025] DistillDIFT: Distillation of Diffusion Features for Semantic Correspondence

Basic Info
  • Host: GitHub
  • Owner: CompVis
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 97.8 MB
Statistics
  • Stars: 24
  • Watchers: 2
  • Forks: 1
  • Open Issues: 0
  • Releases: 0
Created over 1 year ago · Last pushed about 1 year ago
Metadata Files
Readme

README.md

DistillDIFT: Distillation of Diffusion Features for Semantic Correspondence

Frank Fundel Johannes Schusterbauer Vincent Tao Hu Bjrn Ommer

CompVis @ LMU Munich, MCML

WACV 2025

Website Paper

TL;DR

We present DistillDIFT, a highly efficient approach to semantic correspondence that delivers state-of-the-art performance with significantly reduced computational cost. Unlike traditional methods that combine multiple large generative models, DistillDIFT uses a novel distillation technique to unify the strengths of two vision foundation models into a single, streamlined model. By integrating 3D data without requiring human annotations, DistillDIFT further improves accuracy.

Overall, our empirical results demonstrate that our distilled model with 3D data augmentation achieves superior performance to current state-of-the-art methods while significantly reducing computational load and enhancing practicality for real-world applications, such as semantic video correspondence.

Setup

This setup was tested with Ubuntu 22.04.4 LTS, CUDA Version: 12.2, and Python 3.9.20.

First, clone the github repo... bash git clone git@github.com:CompVis/distilldift.git cd DistillDIFT

Evaluation on SPair-71K

Our evaluation pipeline for SPair-71K is based on Telling-Left-From-Right for better comparability.

Follow their environment setup and data preparation, don't forget to first: bash cd eval

And then run the evaluation script via bash bash eval_distilldift.sh

Training

First use bash cd train

Then you have either the option to setup a virtual environment and install all required packages with pip via bash pip install -r requirements.txt or if you prefer to use conda create the conda environment via bash conda env create -f environment.yaml

Download the COCO dataset and embed the images (for unsupervised training) via bash bash datasets/download_coco.sh python embed.py --dataset_name COCO

And run the training via - Unsupervised Distillation bash accelerate launch --multi_gpu --num_processes 4 train.py distilled_us --dataset_name COCO --use_cache - Weakly Supervised Distillation bash accelerate launch --multigpu --numprocesses 4 train.py distilledws --datasetname SPair-71k --usecache - Supervised Training bash accelerate launch --multigpu --numprocesses 4 train.py distilleds --datasetname SPair-71k --usecache ```

Refinement using CO3D

Follow the official instructions to download the CO3D dataset and then prepare the CO3D dataset via bash python datasets/create_co3d.py

And run the training via bash accelerate launch --multi_gpu --num_processes 4 train.py distilled_s --dataset_name CO3D --use_cache

Citation

Please cite our paper:

bibtex @article{fundel2025distilldift, author = {Frank Fundel and Johannes Schusterbauer and Vincent Tao Hu and Bjrn Ommer}, title = {Distillation of Diffusion Features for Semantic Correspondence}, journal = {WACV}, year = {2025}, }

Owner

  • Name: CompVis - Computer Vision and Learning LMU Munich
  • Login: CompVis
  • Kind: organization
  • Email: assist.mvl@lrz.uni-muenchen.de
  • Location: Germany

Computer Vision and Learning research group at Ludwig Maximilian University of Munich (formerly Computer Vision Group at Heidelberg University)

GitHub Events

Total
  • Issues event: 3
  • Watch event: 25
  • Issue comment event: 2
  • Member event: 2
  • Push event: 6
  • Fork event: 2
  • Create event: 3
Last Year
  • Issues event: 3
  • Watch event: 25
  • Issue comment event: 2
  • Member event: 2
  • Push event: 6
  • Fork event: 2
  • Create event: 3

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 5
  • Total Committers: 2
  • Avg Commits per committer: 2.5
  • Development Distribution Score (DDS): 0.2
Past Year
  • Commits: 5
  • Committers: 2
  • Avg Commits per committer: 2.5
  • Development Distribution Score (DDS): 0.2
Top Committers
Name Email Commits
Johannes Fischer j****r@o****m 4
FrankFundel f****l@o****m 1

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 2
  • Total pull requests: 0
  • Average time to close issues: 20 days
  • Average time to close pull requests: N/A
  • Total issue authors: 2
  • Total pull request authors: 0
  • Average comments per issue: 0.5
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 2
  • Pull requests: 0
  • Average time to close issues: 20 days
  • Average time to close pull requests: N/A
  • Issue authors: 2
  • Pull request authors: 0
  • Average comments per issue: 0.5
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • abcp4 (1)
  • myunusseker (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels