mask4d

Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences, RA-L, 2023

https://github.com/prbonn/mask4d

Science Score: 65.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 1 DOI reference(s) in README
○
Academic publication links
○
Academic email domains
✓
Institutional organization owner
Organization prbonn has institutional domain (www.ipb.uni-bonn.de)
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (11.1%) to scientific vocabulary

Last synced: 8 months ago · JSON representation ·

Repository

Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences, RA-L, 2023

Basic Info

Host: GitHub
Owner: PRBonn
License: mit
Language: Python
Default Branch: master
Homepage:
Size: 313 KB

Statistics

Stars: 65
Watchers: 3
Forks: 6
Open Issues: 0
Releases: 1

Created over 2 years ago · Last pushed almost 2 years ago

Metadata Files

Readme License Citation

Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences

This is the official implementation of Mask4D.

Overview

Mask4D is a method for 4D panoptic segmentation using masks. It builds on top of MaskPLS using SphereFormer as feature extractor.
We reuse the output queries of previous steps to decode and track the same instance over time.
It is an end-to-end approach without post-processing step like clustering.
We propose Position-aware mask attention to provide prior positional information to the cross-attention and improve the segmentation.

Get started

Install this package by running in the root directory of this repo: pip3 install -U -e .

Install pdependencies (we test on python=3.8.10, pytorch==1.12.0, cuda==11.3)

pip3 install torch==1.12.0+cu113 torchvision==0.13.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113 pip3 install -r requirements.txt

Install SparseTransformer.

Data preparation: SemanticKITTI

Download the SemanticKITTI dataset inside the directory data/kitti/. The directory structure should look like this: ./ └── data/ └── kitti └── sequences ├── 00/ │ ├── velodyne/ | | ├── 000000.bin | | ├── 000001.bin | | └── ... │ └── labels/ | ├── 000000.label | ├── 000001.label | └── ... ├── 08/ # for validation ├── 11/ # 11-21 for testing └── 21/ └── ...

Pretrained models

Reproducing results

python3 scripts/evaluate_model.py --w [path_to_4D_model]

Training

We leverage the weights of the 3D model MaskPLS with SphereFormer as backbone.

``` python3 scripts/trainmodel.py --w [pathto3Dmodel]

```

Citation

bibtex @article{marcuzzi2023ral-meem, author = {R. Marcuzzi and L. Nunes and L. Wiesmann and E. Marks and J. Behley and C. Stachniss}, title = {{Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences}}, journal = ral, year = 2023, volume = {8}, number = {11}, pages = {7487-7494}, doi = {10.1109/LRA.2023.3320020}, url = {https://www.ipb.uni-bonn.de/wp-content/papercite-data/pdf/marcuzzi2023ral-meem.pdf}, }

Owner

Name: Photogrammetry & Robotics Bonn
Login: PRBonn
Kind: organization
Email: cyrill.stachniss@igg.uni-bonn.de
Location: Bonn

Website: www.ipb.uni-bonn.de
Repositories: 41
Profile: https://github.com/PRBonn

Photogrammetry & Robotics Lab at the University of Bonn

Citation (CITATION.cff)

cff-version: 1.2.0
preferred-citation:
  title: "Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences"
  doi: "10.1109/LRA.2023.3320020"
  year: "2023"
  type: article
  journal: "IEEE Robotics and Automation Letters (RA-L)"
  url: https://www.ipb.uni-bonn.de/wp-content/papercite-data/pdf/marcuzzi2023ral-meem.pdf
  codeurl: https://github.com/PRBonn/Mask4D
  authors:
    - family-names: Marcuzzi
      given-names: Rodrigo
    - family-names: Nunes
      given-names: Lucas
    - family-names: Wiesmann
      given-names: Louis
    - family-names: Marks
      given-names: Elias
    - family-names: Behley
      given-names: Jens
    - family-names: Stachniss
      given-names: Cyrill

GitHub Events

Total

Watch event: 13
Fork event: 2

Last Year

Watch event: 13
Fork event: 2

Issues and Pull Requests

Last synced: over 1 year ago

All Time

Total issues: 6
Total pull requests: 0
Average time to close issues: 2 months
Average time to close pull requests: N/A
Total issue authors: 6
Total pull request authors: 0
Average comments per issue: 2.5
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 4
Pull requests: 0
Average time to close issues: 2 months
Average time to close pull requests: N/A
Issue authors: 4
Pull request authors: 0
Average comments per issue: 2.0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

scw0819 (1)
himadrir (1)
Zero-Yi (1)
G12311231 (1)
Xiangxu-0103 (1)
tigeriv (1)

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science