Data

Tools

Indexes

Applications

Experiments

Open Source Science

pix2seq-mmdetection

Unofficial implement of "Pix2seq: A Language Modeling Framework for Object Detection" on mmdetection

https://github.com/sharpiless/pix2seq-mmdetection

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.1%) to scientific vocabulary
Last synced: 10 months ago · JSON representation ·

Repository

Unofficial implement of "Pix2seq: A Language Modeling Framework for Object Detection" on mmdetection

Basic Info
  • Host: GitHub
  • Owner: Sharpiless
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 8.19 MB
Statistics
  • Stars: 33
  • Watchers: 3
  • Forks: 4
  • Open Issues: 1
  • Releases: 1
Created about 4 years ago · Last pushed about 4 years ago
Metadata Files
Readme License Citation

README.md

Introduction

This is an unofficial replication of "Pix2seq: A Language Modeling Framework for Object Detection" with pretrained model on mmdetection.

License

This project is released under the Apache 2.0 license.

Installation

Please refer to get_started.md for installation.

Train & Evaluation

Train by running (about 10 days with 8*V100 32GB) bash python -m torch.distributed.launch --nproc_per_node=8 --master_port=5003 \ tools/train.py configs/pix2seq/pix2seq_r50_8x4_50e_coco.py --work-dir pix2seq-output --gpus 8 --launcher pytorch

or

Download pretrained pix2seq weights.

Evaluate with single gpu: bash python tools/test.py configs/pix2seq/pix2seq_r50_8x4_300_coco.py \ weights/checkpoints.pth --work-dir pix2seq-output --eval bbox --show-dir pix2seq-vis

Evaluate with 8 gpus: bash python -m torch.distributed.launch --nproc_per_node=8 --master_port=5003 \ tools/test.py configs/pix2seq/pix2seq_r50_8x4_300_coco.py weights/checkpoints.pth \ --work-dir pix2seq-output --eval bbox --launcher pytorch

| Method | backbone | Epoch | Batch Size | AP | AP50 | AP75 | Weights | | :-----: | :------: | :----:| :---------:| :---:| :---: | :---: | :---: | | Ours | R50 | 300 | 32 | 36.4 | 52.8 | 38.5 | model | | Paper | R50 | 300 | 128 | 43.0 | 61.0 | 45.6 | - |

Visualization

TO-DO

  • [x] random shuffle targets
  • [x] training from scratch
  • [x] drop class token
  • [x] stochastic depth
  • [x] large scale jittering
  • [ ] support for custom dataset
  • [x] two independent augmentations for each image
  • [x] FrozenBatchNorm2d in backbones
  • [x] auto-argument
  • [x] nucleus sampling

Acknowledgement

https://github.com/gaopengcuhk/Pretrained-Pix2Seq

https://github.com/open-mmlab/mmdetection

Owner

  • Login: Sharpiless
  • Kind: user
  • Location: Beijing, China
  • Company: Beijing Institute of Technology

PhD student | 3D Vision | NERF | High Level Vision

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - name: "MMDetection Contributors"
title: "OpenMMLab Detection Toolbox and Benchmark"
date-released: 2018-08-22
url: "https://github.com/open-mmlab/mmdetection"
license: Apache-2.0

GitHub Events

Total
  • Watch event: 2
Last Year
  • Watch event: 2

Issues and Pull Requests

Last synced: about 1 year ago

All Time
  • Total issues: 1
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • vansin (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements/build.txt pypi
  • cython *
  • numpy *
requirements/docs.txt pypi
  • docutils ==0.16.0
  • recommonmark *
  • sphinx ==4.0.2
  • sphinx-copybutton *
  • sphinx_markdown_tables *
  • sphinx_rtd_theme ==0.5.2
requirements/mminstall.txt pypi
  • mmcv-full >=1.3.17
requirements/optional.txt pypi
  • cityscapesscripts *
  • imagecorruptions *
  • scipy *
  • sklearn *
requirements/readthedocs.txt pypi
  • mmcv *
  • torch *
  • torchvision *
requirements/runtime.txt pypi
  • matplotlib *
  • numpy *
  • pycocotools *
  • pycocotools-windows *
  • six *
  • terminaltables *
requirements/tests.txt pypi
  • asynctest * test
  • codecov * test
  • flake8 * test
  • interrogate * test
  • isort ==4.3.21 test
  • kwarray * test
  • mmtrack * test
  • onnx ==1.7.0 test
  • onnxruntime >=1.8.0 test
  • pytest * test
  • ubelt * test
  • xdoctest >=0.10.0 test
  • yapf * test