Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/aim-uofa/segprompt

Official Implementation of ICCV 2023 Paper - SegPrompt: Boosting Open-World Segmentation via Category-level Prompt Learning

https://github.com/aim-uofa/segprompt

Science Score: 33.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: scholar.google
  • ✓
    Committers with academic emails
    1 of 3 committers (33.3%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.9%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Official Implementation of ICCV 2023 Paper - SegPrompt: Boosting Open-World Segmentation via Category-level Prompt Learning

Basic Info
Statistics
  • Stars: 110
  • Watchers: 9
  • Forks: 1
  • Open Issues: 1
  • Releases: 0
Created almost 3 years ago · Last pushed about 1 year ago
Metadata Files
Readme License

README.md

SegPrompt: Boosting Open-world Segmentation via Category-level Prompt Learning

Muzhi Zhu1,   Hengtao Li1,   [Hao Chen](https://stan-haochen.github.io/)1,   Chengxiang Fan1,   [Weian Mao](https://scholar.google.com/citations?user=Qu-QXTsAAAAJ)2,1,   [Chenchen Jing](https://jingchenchen.github.io/)1,   [Yifan Liu](https://irfanicmll.github.io/)2,   [Chunhua Shen](https://cshen.github.io/)1 1[Zhejiang University](https://github.com/aim-uofa),   2[The University of Adelaide](https://www.adelaide.edu.au/),  

News

  • [2023/07/14] Our work SegPrompt is accepted by Int. Conf. Computer Vision (ICCV) 2023! 🎉🎉🎉
  • [2023/08/30] We release our new benchmark LVIS-OW.

Installation

Please follow the instructions in Mask2Former

Other requirements

pip install torchshow pip install torch-scatter -f https://data.pyg.org/whl/torch-1.10.1+cu113.html pip install lvis pip install setuptools==59.5.0 pip install seaborn

LVIS-OW benchmark

Here we provide our proposed new benchmark LVIS-OW.

Dataset preparation

First prepare COCO and LVIS dataset, place them under $DETECTRON2_DATASETS following Detectron2

The dataset structure is as follows: datasets/ coco/ annotations/ instances_{train,val}2017.json {train,val}2017/ lvis/ lvis_v1_{train,val}.json

We reorganize the dataset and divide the categories into Known-Seen-Unseen to better evaluate the open-world model. The json files can be downloaded from here.

Or you can directly use the command to generate from the json file of COCO and LVIS. bash tools/prepare_lvisow.sh

After you successfully get lvisv1trainow.json and lvisv1valresplit_r.json, you can refer to here to register the training set and test set. Then you can use our benchmark for training and testing.

Training on LVIS

```bash

we use 4 A100 GPUs for training

python trainnet.py --num-gpus 4 --config-file configs/lvis/segpromptlvisfull.yaml ``` python trainnet.py --num-gpus 4 --config-file configs/coco/instance-segmentation/maskformer2examplefreezeonlycocoR50bs1650ep3xrepeat_full.yaml --resume

Training on LVIS-OW

```bash

first train a m2f model

python trainnet.py --num-gpus 4 --config-file configs/lvisow/maskformer2examplefreezeR50bs1650ep1xrepeatrmr_es.yaml

then train a segprompt model

python trainnet.py --num-gpus 4 --config-file configs/lvisow/segprompt_lvisow.yaml

```

Inference

```bash

use configs/coco/instance-segmentation/maskformer2R50bs1650ep1x_r.yaml, only use m2f model

CUDAVISIBLEDEVICES=1 python trainnet.py --config-file configs/coco/instance-segmentation/maskformer2R50bs1650ep1xr.yaml --eval-only MODEL.WEIGHTS output/m2fbinarylvisow/modelfinal.pth OUTPUTDIR output/m2fbinarylvisow/test ``````

Evaluation on LVIS-OW

bash python tools/eval_lvis_ow.py --dt-json-file output/m2f_binary_lvis_ow/lvis_r/inference/lvis_instances_results.json

Evaluation on LVIS in an original class-agnostic way

python tools/eval_orilvisapi.py --dt-json-file output/m2f_binary_lvis_ow/test/inference/lvis_instances_results.json

Acknowledgement

We thank the following repos for their great works: - Mask2Former - Detectron2

Cite our Paper

If you found this project useful for your paper, please kindly cite our paper.

🎫 License

For non-commercial academic use, this project is licensed under the 2-clause BSD License. For commercial use, please contact Chunhua Shen.

Owner

  • Name: Advanced Intelligent Machines (AIM)
  • Login: aim-uofa
  • Kind: organization
  • Location: China

A research team at Zhejiang University, focusing on Computer Vision and broad AI research ...

GitHub Events

Total
  • Watch event: 1
  • Push event: 1
Last Year
  • Watch event: 1
  • Push event: 1

Committers

Last synced: over 1 year ago

All Time
  • Total Commits: 9
  • Total Committers: 3
  • Avg Commits per committer: 3.0
  • Development Distribution Score (DDS): 0.556
Past Year
  • Commits: 3
  • Committers: 1
  • Avg Commits per committer: 3.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
chhshen c****n@y****m 4
Chunhua Shen 1****n 3
Z-MU-Z 7****Z 2

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 1
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 3.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • Alan-lab (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels