Data

Tools

Indexes

Applications

Experiments

Open Source Science

deep_generative_models

Repository of the paper Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models at ML4AD @ NeurIPS 2021.

https://github.com/daniel-bogdoll/deep_generative_models

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (8.7%) to scientific vocabulary

Keywords

autonomous-driving generative-model machine-learning
Last synced: 6 months ago · JSON representation ·

Repository

Repository of the paper Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models at ML4AD @ NeurIPS 2021.

Basic Info
Statistics
  • Stars: 6
  • Watchers: 4
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Topics
autonomous-driving generative-model machine-learning
Created over 4 years ago · Last pushed over 3 years ago
Metadata Files
Readme Citation

README.md

Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models

Repository of the paper Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models, accepted at ML4AD@NeurIPS 2021.

Online Pipeline

The left side of the videos shows the ground truth data from CARLA. On the right you see the VAE based reconstructions. Videos are accelerated. For figure 6 in the paper the VAE model was trained with preprocessed lidar scans with a shape of 512x64 (same as for the images). This included some minor padding. After the submission we trained the VAE model with preprocessed lidar scans with a shape of 128x64 instead, which led to a much improved quality of the reconstructed pointclouds as you can see in the video.

https://user-images.githubusercontent.com/19552411/135649380-de1865eb-b1d5-4852-ad9f-807104dca5a9.mp4

https://user-images.githubusercontent.com/19552411/140888871-8678ced9-50f2-4da6-80ed-942d382bb239.mp4

Repository Structure

See the specific folders for additional information.

bash . catkin_ws # ROS workspace for running the online pipeline evaluation # Evaluation results gan # The GAN we use lidar # Contains the lidar preprocessing package and supplementary code paper-graphics # Code that generates some of our graphics vae # The VAE we use

Citation

If you find this code useful for your research, please cite our paper: @article{Bogdoll_Compressing_2021_NeurIPS, author = {Bogdoll, Daniel and Jestram, Johannes and Rauch, Jonas and Scheib, Christin and Wittig, Moritz and Z\"{o}llner, J. Marius}, title = {{Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models}}, journal = {NeurIPS Conference on Neural Information Processing Systems Workshop on Machine Learning for Autonomous Driving (ML4AD)}, year = {2021}, }

Owner

  • Name: Daniel Bogdoll
  • Login: daniel-bogdoll
  • Kind: user
  • Location: Karlsruhe, Germany
  • Company: FZI

Working on the Mobility of Tomorrow. Research Scientist at FZI and PhD student at KIT with a focus on deep learning for autonomous driving.

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Bogdoll"
  given-names: "Daniel"
  orcid: "https://orcid.org/0000-0003-0432-4937"
- family-names: "Jestram"
  given-names: "Johannes"
- family-names: "Rauch"
  given-names: "Jonas"
- family-names: "Scheib"
  given-names: "Christin"
- family-names: "Wittig"
  given-names: "Moritz"
- family-names: "Zöllner"
  given-names: "J. Marius"
title: "Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models"
url: "https://github.com/daniel-bogdoll/deep_generative_models"
preferred-citation:
  type: article
  authors:
  - family-names: "Bogdoll"
    given-names: "Daniel"
    orcid: "https://orcid.org/0000-0003-0432-4937"
  - family-names: "Jestram"
    given-names: "Johannes"
  - family-names: "Rauch"
    given-names: "Jonas"
  - family-names: "Scheib"
    given-names: "Christin"
  - family-names: "Wittig"
    given-names: "Moritz"
  - family-names: "Zöllner"
    given-names: "J. Marius"
  journal: "NeurIPS Conference on Neural Information Processing Systems Workshop on Machine Learning for Autonomous Driving (ML4AD)"
  title: "Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models"
  year: 2021

GitHub Events

Total
Last Year

Dependencies

catkin_ws/src/generative_models/requirements.txt pypi
  • pandas *
  • torch *
catkin_ws/src/generative_models/scripts/high_fidelity_generative_compression/requirements.txt pypi
  • Markdown ==3.2.2
  • Pillow ==8.1.1
  • PyWavelets ==1.1.1
  • Werkzeug ==1.0.1
  • absl-py ==0.9.0
  • autograd ==1.3
  • cachetools ==4.1.1
  • certifi ==2020.6.20
  • chardet ==3.0.4
  • cycler ==0.10.0
  • decorator ==4.4.2
  • future ==0.18.2
  • google-auth ==1.20.1
  • google-auth-oauthlib ==0.4.1
  • grpcio ==1.31.0
  • idna ==2.10
  • imageio ==2.9.0
  • importlib-metadata ==1.7.0
  • kiwisolver ==1.2.0
  • matplotlib ==3.3.0
  • networkx ==2.4
  • numpy ==1.19.1
  • oauthlib ==3.1.0
  • protobuf ==3.12.4
  • pyasn1 ==0.4.8
  • pyasn1-modules ==0.2.8
  • pyparsing ==2.4.7
  • python-dateutil ==2.8.1
  • requests ==2.24.0
  • requests-oauthlib ==1.3.0
  • rsa ==4.6
  • scikit-image ==0.17.2
  • scipy ==1.5.2
  • six ==1.15.0
  • tensorboard ==2.3.0
  • tensorboard-plugin-wit ==1.7.0
  • tifffile ==2020.7.24
  • torch ==1.6.0
  • torchvision ==0.7.0
  • tqdm ==4.48.2
  • urllib3 ==1.25.10
  • zipp ==3.1.0
gan/high-fidelity-generative-compression/requirements.txt pypi
  • Markdown ==3.2.2
  • Pillow ==8.1.1
  • PyWavelets ==1.1.1
  • Werkzeug ==1.0.1
  • absl-py ==0.9.0
  • autograd ==1.3
  • cachetools ==4.1.1
  • certifi ==2020.6.20
  • chardet ==3.0.4
  • cycler ==0.10.0
  • decorator ==4.4.2
  • future ==0.18.2
  • google-auth ==1.20.1
  • google-auth-oauthlib ==0.4.1
  • grpcio ==1.31.0
  • idna ==2.10
  • imageio ==2.9.0
  • importlib-metadata ==1.7.0
  • kiwisolver ==1.2.0
  • matplotlib ==3.3.0
  • networkx ==2.4
  • numpy ==1.19.1
  • oauthlib ==3.1.0
  • protobuf ==3.12.4
  • pyasn1 ==0.4.8
  • pyasn1-modules ==0.2.8
  • pyparsing ==2.4.7
  • python-dateutil ==2.8.1
  • requests ==2.24.0
  • requests-oauthlib ==1.3.0
  • rsa ==4.6
  • scikit-image ==0.17.2
  • scipy ==1.5.2
  • six ==1.15.0
  • tensorboard ==2.3.0
  • tensorboard-plugin-wit ==1.7.0
  • tifffile ==2020.7.24
  • torch ==1.6.0
  • torchvision ==0.7.0
  • tqdm ==4.48.2
  • urllib3 ==1.25.10
  • zipp ==3.1.0
lidar/requirements.txt pypi
  • matplotlib *
  • numpy *
  • tensorflow ==2.5.0
  • tqdm *
  • waymo-open-dataset-tf-2-5-0 *
vae/requirements.txt pypi
  • Automat ==0.8.0
  • Babel ==2.6.0
  • Click ==7.0
  • Cython ==0.29.14
  • Deprecated ==1.2.7
  • Jinja2 ==2.10.1
  • Keras-Preprocessing ==1.1.2
  • Markdown ==3.3.4
  • MarkupSafe ==1.1.0
  • Pillow ==7.0.0
  • Pivy ==0.6.5
  • PyGObject ==3.36.0
  • PyGithub ==1.43.7
  • PyHamcrest ==1.9.0
  • PyJWT ==1.7.1
  • PyNaCl ==1.3.0
  • PyOpenGL ==3.1.0
  • PyQRCode ==1.2.1
  • PyQt5 ==5.14.1
  • PyTrie ==0.2
  • PyYAML ==5.3.1
  • Pygments ==2.3.1
  • QtAwesome ==0.4.4
  • QtPy ==1.9.0
  • SecretStorage ==2.3.1
  • Send2Trash ==1.5.0
  • Sphinx ==1.8.5
  • Twisted ==18.9.0
  • Werkzeug ==2.0.1
  • absl-py ==0.13.0
  • alabaster ==0.7.8
  • appdirs ==1.4.3
  • argcomplete ==1.8.1
  • astroid ==2.3.3
  • astunparse ==1.6.3
  • atomicwrites ==1.1.5
  • attrs ==19.3.0
  • autobahn ==17.10.1
  • backcall ==0.1.0
  • bcrypt ==3.1.7
  • beautifulsoup4 ==4.8.2
  • bleach ==3.1.1
  • blinker ==1.4
  • breezy ==3.0.2
  • cachetools ==4.2.2
  • carla ==0.9.12
  • catkin-pkg ==0.4.23
  • catkin-pkg-modules ==0.4.23
  • catkin-tools ==0.7.1
  • cbor ==1.0.0
  • certifi ==2019.11.28
  • chardet ==3.0.4
  • clang ==5.0
  • cloudpickle ==1.3.0
  • colcon-argcomplete ==0.3.3
  • colcon-bash ==0.4.2
  • colcon-cd ==0.1.1
  • colcon-cmake ==0.2.26
  • colcon-common-extensions ==0.2.1
  • colcon-core ==0.6.1
  • colcon-defaults ==0.2.5
  • colcon-devtools ==0.2.2
  • colcon-library-path ==0.2.1
  • colcon-metadata ==0.2.5
  • colcon-notification ==0.2.13
  • colcon-output ==0.2.12
  • colcon-package-information ==0.3.3
  • colcon-package-selection ==0.2.10
  • colcon-parallel-executor ==0.2.4
  • colcon-pkg-config ==0.1.0
  • colcon-powershell ==0.3.6
  • colcon-python-setup-py ==0.2.7
  • colcon-recursive-crawl ==0.2.1
  • colcon-ros ==0.3.21
  • colcon-test-result ==0.3.8
  • colcon-zsh ==0.4.0
  • colorama ==0.4.3
  • command-not-found ==0.3
  • compressai ==1.1.6
  • configobj ==5.0.6
  • constantly ==15.1.0
  • cov-core ==1.15.0
  • coverage ==4.5.2
  • cryptography ==2.8
  • cycler ==0.10.0
  • dbus-python ==1.2.16
  • decorator ==4.4.2
  • defusedxml ==0.6.0
  • distlib ==0.3.0
  • distro ==1.4.0
  • distro-info ===0.23ubuntu1
  • docutils ==0.16
  • dulwich ==0.19.15
  • empy ==3.3.2
  • entrypoints ==0.3
  • fastimport ==0.9.8
  • filelock ==3.0.12
  • flake8 ==3.7.9
  • flatbuffers ==1.12
  • gast ==0.4.0
  • gauss ==0.0.9
  • google-auth ==1.34.0
  • google-auth-oauthlib ==0.4.5
  • google-pasta ==0.2.0
  • gpg ===1.13.1
  • grpcio ==1.39.0
  • h5py ==3.1.0
  • html5lib ==1.0.1
  • httplib2 ==0.14.0
  • hyperlink ==19.0.0
  • idna ==2.8
  • ifcfg ==0.18
  • imagesize ==1.2.0
  • importlib-metadata ==1.5.0
  • incremental ==16.10.1
  • iotop ==0.6
  • ipykernel ==5.2.0
  • ipython ==7.13.0
  • ipython-genutils ==0.2.0
  • ipywidgets ==6.0.0
  • isort ==4.3.4
  • jedi ==0.15.2
  • joblib ==0.14.0
  • jsonschema ==3.2.0
  • jupyter-client ==6.1.2
  • jupyter-core ==4.6.3
  • kazam ==1.4.5
  • keras ==2.6.0
  • keras-nightly ==2.5.0.dev2021032900
  • keyring ==18.0.1
  • kiwisolver ==1.0.1
  • language-selector ==0.1
  • lark-parser ==0.8.1
  • launchpadlib ==1.10.13
  • lazr.restfulclient ==0.14.2
  • lazr.uri ==1.0.3
  • lazy-object-proxy ==1.4.3
  • logilab-common ==1.4.3
  • lpips ==0.1.3
  • lxml ==4.5.0
  • lz4 ==3.0.2
  • matplotlib ==3.1.2
  • mccabe ==0.6.1
  • mistune ==0.8.4
  • mock ==3.0.5
  • more-itertools ==4.2.0
  • mpi4py ==3.0.3
  • nbconvert ==5.6.1
  • nbformat ==5.0.4
  • netifaces ==0.10.4
  • nose ==1.3.7
  • nose2 ==0.9.1
  • notebook ==6.0.3
  • notify2 ==0.3
  • numexpr ==2.7.1
  • numpy ==1.19.5
  • numpydoc ==0.7.0
  • oauthlib ==3.1.0
  • olefile ==0.46
  • opt-einsum ==3.3.0
  • osrf-pycommon ==1.0.0
  • packaging ==20.3
  • pandas ==1.3.1
  • pandocfilters ==1.4.2
  • paramiko ==2.6.0
  • parso ==0.5.2
  • patsy ==0.5.1
  • pbr ==5.4.5
  • pexpect ==4.6.0
  • pickleshare ==0.7.5
  • plotly ==5.1.0
  • pluggy ==0.13.0
  • prometheus-client ==0.7.1
  • prompt-toolkit ==2.0.10
  • protobuf ==3.17.3
  • psutil ==5.5.1
  • psycopg2 ==2.8.4
  • py ==1.8.1
  • py-ubjson ==0.14.0
  • pyOpenSSL ==19.0.0
  • pyasn1 ==0.4.2
  • pyasn1-modules ==0.2.1
  • pycairo ==1.16.2
  • pycodestyle ==2.5.0
  • pycryptodomex ==3.6.1
  • pydocstyle ==2.1.1
  • pydot ==1.4.1
  • pyflakes ==2.1.1
  • pylint ==2.4.4
  • pymacaroons ==0.13.0
  • pyparsing ==2.4.6
  • pypng ==0.0.20
  • pyproj ==2.5.0
  • pyrsistent ==0.15.5
  • pyserial ==3.4
  • pysvn ==1.9.9
  • pytest ==4.6.9
  • pytest-cov ==2.8.1
  • python-apt ==2.0.0
  • python-dateutil ==2.7.3
  • python-gitlab ==2.0.1
  • python-gnupg ==0.4.5
  • python-snappy ==0.5.3
  • pytorch-msssim ==0.2.1
  • pytz ==2019.3
  • pyxdg ==0.26
  • pyzmq ==18.1.1
  • qtconsole ==4.6.0
  • requests ==2.22.0
  • requests-oauthlib ==1.3.0
  • requests-unixsocket ==0.2.0
  • roman ==2.0.0
  • rope ==0.10.5
  • rosdep ==0.21.0
  • rosdep-modules ==0.21.0
  • rosdistro ==0.8.3
  • rosdistro-modules ==0.8.3
  • rospkg ==1.3.0
  • rospkg-modules ==1.3.0
  • rsa ==4.7.2
  • scikit-learn ==0.22.2.post1
  • scipy ==1.3.3
  • screen-resolution-extra ==0.0.0
  • seaborn ==0.11.1
  • service-identity ==18.1.0
  • simplejson ==3.16.0
  • sip ==4.19.21
  • six ==1.15.0
  • snowballstemmer ==2.0.0
  • soupsieve ==1.9.5
  • spyder ==3.3.6
  • spyder-kernels ==0.5.2
  • ssh-import-id ==5.10
  • statsmodels ==0.12.2
  • stevedore ==1.32.0
  • systemd-python ==234
  • tables ==3.6.1
  • tenacity ==8.0.1
  • tensorboard ==2.6.0
  • tensorboard-data-server ==0.6.1
  • tensorboard-plugin-wit ==1.8.0
  • tensorflow ==2.5.0
  • tensorflow-estimator ==2.6.0
  • tensorflow-gpu ==2.6.0
  • tensorflow-hub ==0.12.0
  • termcolor ==1.1.0
  • terminado ==0.8.2
  • testpath ==0.4.4
  • torch ==1.9.0
  • torchvision ==0.10.0
  • tornado ==5.1.1
  • tqdm ==4.61.2
  • traitlets ==4.3.3
  • txaio ==2.10.0
  • typing-extensions ==3.7.4.3
  • u-msgpack-python ==2.1
  • ubuntu-advantage-tools ==27.2
  • ufw ==0.36
  • unattended-upgrades ==0.1
  • urllib3 ==1.25.8
  • vboxapi ==1.0
  • vcstools ==0.1.42
  • virtualenv ==20.0.17
  • virtualenv-clone ==0.3.0
  • virtualenvwrapper ==4.8.4
  • wadllib ==1.3.3
  • wcwidth ==0.1.8
  • webencodings ==0.5.1
  • widgetsnbextension ==2.0.0
  • wrapt ==1.12.1
  • wsaccel ==0.6.2
  • wstool ==0.1.18
  • wxPython ==4.0.7
  • xkit ==0.0.0
  • zipp ==1.0.0
  • zope.interface ==4.7.1