evo

Python package for the evaluation of odometry and SLAM

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
✓
Committers with academic emails
1 of 18 committers (5.6%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (11.5%) to scientific vocabulary

Keywords

benchmark euroc evaluation kitti mapping metrics odometry robotics ros ros2 slam trajectory trajectory-analysis trajectory-evaluation tum

Last synced: 6 months ago · JSON representation

Repository

Python package for the evaluation of odometry and SLAM

Basic Info

Host: GitHub
Owner: MichaelGrupp
License: gpl-3.0
Language: Python
Default Branch: master
Homepage: https://michaelgrupp.github.io/evo/
Size: 7.06 MB

Statistics

Stars: 3,902
Watchers: 47
Forks: 777
Open Issues: 13
Releases: 0

Topics

benchmark euroc evaluation kitti mapping metrics odometry robotics ros ros2 slam trajectory trajectory-analysis trajectory-evaluation tum

Created over 8 years ago · Last pushed 7 months ago

Metadata Files

Readme License

evo

Python package for the evaluation of odometry and SLAM

| Linux / macOS / Windows / ROS / ROS2 | | :---: | | |

This package provides executables and a small library for handling, evaluating and comparing the trajectory output of odometry and SLAM algorithms.

Supported trajectory formats:

'TUM' trajectory files
'KITTI' pose files
'EuRoC MAV' (.csv groundtruth and TUM trajectory file)
ROS and ROS2 bagfile with geometry_msgs/PoseStamped, geometry_msgs/TransformStamped, geometry_msgs/PoseWithCovarianceStamped, geometry_msgs/PointStamped or nav_msgs/Odometry topics or TF messages

See here for more infos about the formats.

Why?

evo has several advantages over other public benchmarking tools:

common tools for different formats
algorithmic options for association, alignment, scale adjustment for monocular SLAM etc.
flexible options for output, plotting or export (e.g. LaTeX plots or Excel tables)
a powerful, configurable CLI that can cover many use cases
modular core and tools libraries for custom extensions
faster than other established Python-based tools (see here)

What it's not: a 1-to-1 re-implementation of a particular evaluation protocol tailored to a specific dataset.

Installation / Upgrade

Installation is easy-peasy if you're familiar with this: https://xkcd.com/1987/#

The latest version of evo supports Python 3.10+. You might also want to use a virtual environment.

From PyPi

If you just want to use the executables of the latest release version, the easiest way is to run: bash pip install evo This will download the package and its dependencies from PyPI and install or upgrade them. If you want, you can subscribe to new releases via https://libraries.io/pypi/evo.

To upgrade to a newer version: pip install --upgrade evo

From Source

Run this in the repository's base folder: bash pip install --editable .

Tab completion

Tab completion is supported via the argcomplete package. Run activate-global-python-argcomplete after the installation to use it.

Dependencies

Python packages

evo has some required dependencies that are automatically resolved during installation with pip. See the pyproject.toml file for all details.

PyQt5 (optional)

PyQt5 will give you the enhanced GUI for plot figures from the "Qt5Agg" matplotlib backend (otherwise: "TkAgg"). If PyQt5 is already installed when installing this package, it will be used as a default (see evo_config show). To change the plot backend afterwards, run evo_config set plot_backend Qt5Agg.

If you run into issues with installing tkinter, trying PyQt5 is a good idea.

ROS (optional)

Some ROS-related features require a ROS installation, see here. We are testing this package with ROS Kilted.

Reading ROS bag files works also without a ROS installation thanks to the great rosbags package that is installed together with evo. This allows you also to read ROS 1 & 2 bags even if you don't have one of those ROS distros installed. (except for reading /tf topics, because there we need the buffer implementation from ROS)

contextily (optional)

contextily is required for adding map tiles to plots of geo-referenced data.

Command Line Interface

After installation with pip, the following executables can be called globally from your command-line:

Metrics:

evo_ape - absolute pose error
evo_rpe - relative pose error

Tools:

evo_traj - tool for analyzing, plotting or exporting one or more trajectories
evo_res - tool for comparing one or multiple result files from evo_ape or evo_rpe
evo_fig - (experimental) tool for re-opening serialized plots (saved with --serialize_plot)
evo_config - tool for global settings and config file manipulation

Call the commands with --help to see the options, e.g. evo_ape --help. Tab-completion of command line parameters is available on UNIX-like systems.

More documentation Check out the Wiki on GitHub.

Example Workflow

There are some example trajectories in the source folder in test/data.

1.) Plot multiple trajectories

Here, we plot two KITTI pose files and the ground truth using evo_traj: cd test/data evo_traj kitti KITTI_00_ORB.txt KITTI_00_SPTAM.txt --ref=KITTI_00_gt.txt -p --plot_mode=xz

2.) Run a metric on trajectories

For example, here we calculate the absolute pose error for two trajectories from ORB-SLAM and S-PTAM using evo_ape (KITTI_00_gt.txt is the reference (ground truth)) and plot and save the individual results to .zip files for evo_res:

First trajectory (ORB Stereo):

mkdir results evo_ape kitti KITTI_00_gt.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip

Second trajectory (S-PTAM):

evo_ape kitti KITTI_00_gt.txt KITTI_00_SPTAM.txt -va --plot --plot_mode xz --save_results results/SPTAM.zip

3.) Process multiple results from a metric

evo_res can be used to compare multiple result files from the metrics, i.e.: * print infos and statistics (default) * plot the results * save the statistics in a table

Here, we use the results from above to generate a plot and a table: evo_res results/*.zip -p --save_table results/table.csv

IPython / Jupyter Resources

For an interactive source code documentation, open the Jupyter notebook metrics_tutorial.ipynb in the notebooks folder of the repository. More infos on Jupyter notebooks: see here

If you have IPython installed, you can launch an IPython shell with a custom evo profile with the command evo_ipython.

Contributing Utilities

A few "inoffical" scripts for special use-cases are collected in the contrib/ directory of the repository. They are inofficial in the sense that they don't ship with the package distribution and thus aren't regularly tested in continuous integration.

Trouble

"😱, this piece of 💩 software doesn't do what I want!!1!1!!"

First aid: * append -h/ --help to your command * check the Wiki * check the previous issues * open a new issue

Contributing

Patches are welcome, preferably as pull requests.

License

GPL-3.0 or later

If you use this package for your research, a footnote with the link to this repository is appreciated: github.com/MichaelGrupp/evo.

...or, for citation with BibTeX: @misc{grupp2017evo, title={evo: Python package for the evaluation of odometry and SLAM.}, author={Grupp, Michael}, howpublished={\url{https://github.com/MichaelGrupp/evo}}, year={2017} }

Owner

Name: Michael Grupp
Login: MichaelGrupp
Kind: user
Location: München
Company: TU München

Repositories: 9
Profile: https://github.com/MichaelGrupp

Roboticist, software engineer, maker. Senior Developer @magazino in Munich.

GitHub Events

Total

Create event: 26
Issues event: 42
Watch event: 418
Delete event: 16
Issue comment event: 46
Push event: 55
Pull request review comment event: 2
Pull request review event: 3
Gollum event: 1
Pull request event: 42
Fork event: 37

Last Year

Create event: 26
Issues event: 42
Watch event: 418
Delete event: 16
Issue comment event: 46
Push event: 55
Pull request review comment event: 2
Pull request review event: 3
Gollum event: 1
Pull request event: 42
Fork event: 37

Committers

Last synced: 9 months ago

All Time

Total Commits: 460
Total Committers: 18
Avg Commits per committer: 25.556
Development Distribution Score (DDS): 0.048

Past Year

Commits: 41
Committers: 2
Avg Commits per committer: 20.5
Development Distribution Score (DDS): 0.024

Top Committers

Name	Email	Commits
Michael Grupp	g**p@m**u	438
stefangachter	s****r	3
Valerio Magnago	m**o@m**u	3
Toni Rosinol	t**l@h**m	2
Javier Cremona	c**a@c**r	1
Mariia Gladkova	g**a@m**u	1
Daniel Sánchez de Pedro Rada	7****R	1
Duilio Protti	d****i	1
GAECHTER TOYA Stefan	s**r@h**m	1
KevinWei	w**i@g**m	1
Michel Hidalgo	m**l@e**m	1
Mohd Omama	m**a@g**m	1
Parker Lusk	p**k@g**m	1
Patrick Pfreundschuh	p**r@e**h	1
Usama Ali	u**i@f**e	1
azure-pipelines[bot]	a****]	1
deepakyeleshetty	3****y	1
miseryofdevice	b**r@g**m	1

Committer Domains (Top 20 + Academic)

magazino.eu: 3 fau.de: 1 ethz.ch: 1 ekumenlabs.com: 1 hexagon.com: 1 cifasis-conicet.gov.ar: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 187
Total pull requests: 128
Average time to close issues: about 2 months
Average time to close pull requests: 5 days
Total issue authors: 155
Total pull request authors: 13
Average comments per issue: 2.62
Average comments per pull request: 0.2
Merged pull requests: 106
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 25
Pull requests: 38
Average time to close issues: 12 days
Average time to close pull requests: 3 days
Issue authors: 19
Pull request authors: 5
Average comments per issue: 1.76
Average comments per pull request: 0.13
Merged pull requests: 29
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

MichaelGrupp (5)
TwiceMao (4)
Davidyao99 (4)
fdila (3)
Delbina (3)
110sha (3)
YoungHaulk (3)
pbeeson (2)
BamsaAhmed (2)
majdros (2)
astef3 (2)
sagardhatrak (2)
an99990 (2)
JACKLiuDay (2)
Shubham-2302 (2)

Pull Request Authors

MichaelGrupp (129)
stefangachter (9)
EmmanuelMess (4)
ValerioMa (3)
xubin1991 (2)
scrouthtv (2)
akshaygulabrao (2)
hidmic (2)
TanPinDa (2)
dantenoguera (1)
patripfr (1)
mitchallain (1)

Top Labels

Issue Labels

question (50) data (45) feature request (12) bug (10) installation (6) dependencies (5) enhancement (3) wontfix (3) runtime error (1) help wanted (1) more info needed (1)

Pull Request Labels

Packages

Total packages: 3
Total downloads:
- pypi 52,800 last-month
Total docker downloads: 45

Total dependent packages: 0
(may contain duplicates)
Total dependent repositories: 18
(may contain duplicates)
Total versions: 284
Total maintainers: 1

pypi.org: evo

Python package for the evaluation of odometry and SLAM

Homepage: https://github.com/MichaelGrupp/evo
Documentation: https://evo.readthedocs.io/
License: GNU General Public License v3 (GPLv3)
Latest release: 1.31.1
published 11 months ago

Versions: 110
Dependent Packages: 0
Dependent Repositories: 18
Downloads: 52,800 Last month
Docker Downloads: 45

Rankings

Stargazers count: 1.4%

Downloads: 1.6%

Forks count: 1.7%

Docker downloads count: 3.0%

Dependent repos count: 3.4%

Average: 3.5%

Dependent packages count: 10.1%

Maintainers (1)

MichaelGrupp

Last synced: 6 months ago

proxy.golang.org: github.com/MichaelGrupp/evo

Documentation: https://pkg.go.dev/github.com/MichaelGrupp/evo#section-documentation
License: gpl-3.0
Latest release: v1.31.1
published 11 months ago

Versions: 87
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 7.0%

Average: 8.2%

Dependent repos count: 9.3%

Last synced: 6 months ago

proxy.golang.org: github.com/michaelgrupp/evo

Documentation: https://pkg.go.dev/github.com/michaelgrupp/evo#section-documentation
License: gpl-3.0
Latest release: v1.31.1
published 11 months ago

Versions: 87
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 7.0%

Average: 8.2%

Dependent repos count: 9.3%

Last synced: 6 months ago

Dependencies

setup.py pypi

argcomplete *
colorama >=0.3
matplotlib *
natsort *
numpy >=1.18.5
pandas *
pillow *
pygments *
pyyaml *
rosbags >=0.9.10
scipy >=1.2
seaborn >=0.9

evo

Science Score: 36.0%

Keywords

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

evo

Why?

Installation / Upgrade

From PyPi

From Source

Tab completion

Dependencies

Command Line Interface

Example Workflow

1.) Plot multiple trajectories

2.) Run a metric on trajectories

3.) Process multiple results from a metric

IPython / Jupyter Resources

Contributing Utilities

Trouble

Contributing

License

Owner

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: evo

Rankings

Maintainers (1)

proxy.golang.org: github.com/MichaelGrupp/evo

Rankings

proxy.golang.org: github.com/michaelgrupp/evo

Rankings

Dependencies