CaRtGS

Homepage | Paper

CaRtGS: Computational Alignment for Real-Time Gaussian Splatting SLAM
Dapeng Feng¹, Zhiqiang Chen², Yizhen Yin¹, Shipeng Zhong³, Yuhua Qi¹, and Hongbo Chen¹
Sun Yat-Sen University¹, The University of Hong Kong², WeRide Inc.³

Prerequisites

Dependencies

sudo apt install libeigen3-dev libboost-all-dev libjsoncpp-dev libopengl-dev mesa-utils libglfw3-dev libglm-dev

Installation of CaRtGS

bash git clone --recursive https://github.com/DapengFeng/cartgs.git cd cartgs/ ./build.sh

Splat-wise back-propagation for Depth Rendering (GSICPSLAM) are released at https://github.com/DapengFeng/splatwise-diff-gaussian-rasterization.

CaRtGS Examples on Some Benchmark Datasets

The benchmark datasets mentioned in our paper: Replica (NICE-SLAM Version), TUM RGB-D and VECtor.

1. (optional) Download the dataset. scripts/download_replica.sh scripts/download_tum.sh

2. For testing, you could use the below commands to run the system after specifying the PATH_TO_Replica and PATH_TO_SAVE_RESULTS. We would disable the viewer by adding no_viewer during the evaluation. ``` bash bin/replicargbd \ ORB-SLAM3/Vocabulary/ORBvoc.txt \ cfg/ORBSLAM3/RGB-D/Replica/office0.yaml \ cfg/gaussianmapper/RGB-D/Replica/replicargbd.yaml \ PATHTOReplica/office0 \ PATHTOSAVE_RESULTS

   no_viewer

   ```

3. We also provide scripts to conduct experiments on all benchmark datasets mentioned in our paper. We ran each sequence five times to lower the effect of the nondeterministic nature of the system. You need to change the dataset root lines in scripts/*.sh then run: ``` bash scripts/replicamono.sh scripts/replicargbd.sh scripts/tummono.sh scripts/tumrgbd.sh

   etc.

   ```

CaRtGS Evaluation

To use this toolkit, you have to ensure your results on each dataset are stored in the correct format. If you use our ./xxx.sh scripts to conduct experiments, the results are stored in results ├── replica_mono_0 │ ├── office0 │ ├── .... │ └── room2 ├── replica_rgbd_0 │ ├── office0 │ ├── .... │ └── room2 │ └── [replica/tum]_[mono/rgbd]_num .... ├── scene_1 ├── .... └── scene_n

Install required python package

bash conda create -n cartgs python=3.10.12 pytorch=2.3.1 torchvision pytorch-cuda=12.1 opencv -c pytorch -c nvidia -c conda-forge conda activate cartgs pip install -r python/requirement.txt

install submodel for rendering

bash pip install -e python/diff-gaussian-rasterization/

Convert Replica GT camera pose files to suitable pose files to run EVO package

bash python python/shapeReplicaGT.py --replica_dataset_path PATH_TO_REPLICA_DATASET

To get all metrics, you can run

bash python python/eval.py --dataset_center_path PATH_TO_ALL_DATASET --result_main_folder RESULTS_PATH Finally, you are supposed to get two files including RESULTS_PATH/log.txt and RESULTS_PATH/log.csv.

CaRtGS Examples with Real Cameras

We provide an example with the Intel RealSense D455 at examples/realsense_rgbd.cpp. Please see scripts/realsense_d455.sh for running it.

Acknowledgement

This work incorporates many open-source codes. We extend our gratitude to the authors of the software. - Photo-SLAM - Taming 3DGS

Citation

If you find this work useful in your research, consider citing it: @misc{feng2024CaRtGS, title={CaRtGS: Computational Alignment for Real-Time Gaussian Splatting SLAM}, author={Dapeng Feng and Zhiqiang Chen and Yizhen Yin and Shipeng Zhong and Yuhua Qi and Hongbo Chen}, year={2024}, eprint={2410.00486}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2410.00486}, }