stars-carla-experiments
This repository analyzes driving data recorded with the Carla Simulator using the STARS framework.
Science Score: 77.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 2 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
✓Committers with academic emails
1 of 1 committers (100.0%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.6%) to scientific vocabulary
Keywords
Repository
This repository analyzes driving data recorded with the Carla Simulator using the STARS framework.
Basic Info
Statistics
- Stars: 5
- Watchers: 5
- Forks: 5
- Open Issues: 5
- Releases: 7
Topics
Metadata Files
README.md
STARS experiments based on data from CARLA
This repository analyzes driving data recorded with the Carla Simulator using the STARS framework. The Carla data was recorded using the stars-export-carla Repository.
Setup
The analysis requires the recorded data. To receive the data, there are two options:
1. Set DOWNLOAD_EXPERIMENTS_DATA in experimentsConfiguration.kt to true. This will automatically download and
unzip the necessary data.
2. Manually download the data.
1. Go to the Zenodo artifact where the experiments data is stored
2. Download the stars-reproduction-source.zip
3. Place the Zip-File into the root folder of this project.
Remark: The downloaded data has a size of approximately 1.3GB. The downloaded zip-file will be extracted during the analysis. Make sure, that you have at least 3GB of free space.
Running the Analysis
This project is a Gradle project with a shipped gradle wrapper. To execute the analysis simply execute:
- Linux/Mac:
./gradlew run - Windows:
./gradlew.bat run
Additional Arguments
The project provides the possibility of adjusting the analysis and its environment. The following arguments are provided:
--help: Returns a list of all available arguments.--input(String): Points to the directory of the input files that should be analyzed.--allEgo(Boolean): Sets, whether every vehicle should be treated as ego and therefore increasing the analyzed data by the amount of available vehicles.--minSegmentTicks(Int): Sets the minimum amount of ticks that have to be present for a segment to be analyzed.--sorted(Boolean): Sets, whether the input files should be sorted be theseedin the files name.--dynamicFilter(String): A regex filter that is applied to all dynamic files effectively shrinking the analysis set.--staticFilter(String): A regex filter that is applied to all static files effectively shrinking the analysis set.--ignore(String, separated by ','): A string list filter that is applied to all TSC projections. Make sure to remove all unnecessary white spaces, as they propagate to the final ignore list.
Each argument has a default value to recreate the experiment data from the linked paper in which the results of this repository are used.
Arguments can be passed to the analysis with the following command structure:
- Linux/Mac:
./gradlew run --args="--allEgo --minSegmentTicks=20 --sorted --dynamicFilter='seed18' --staticFilter='Town01' --ignore='all,static'" - Windows:
./gradlew.bat run --args="--allEgo --minSegmentTicks=20 --sorted --dynamicFilter='seed18' --staticFilter='Town01' --ignore='all,static'"
Problem Solving
If you are running into OutOfMemory exceptions, you can add the following statement to the application{} block in
your build.gradle.kts file to adjust the available heap size for the application:
applicationDefaultJvmArgs = listOf("-Xmx12g", "-Xms2g")
The example above will set the max heap size to 12GB (-Xmx12g) and the initial heap size to 2GB (-Xms2g).
Analysis Results
After the analysis is finished you can find the results in the analysis-result-logs subfolder which will be
created during the analysis.
For each execution of the analysis pipeline a subfolder with the start date and time ist created. In it, each metric
of the analysis has its own subfolder. The analysis separates the results of each metric into different categories
with different detail levels.
- *-severe.txt lists all failed metric results
- *-warning.txt lists all warnings that occurred during analysis
- *-info.txt contains the summarized result of the metric
- *-fine.txt contains a more detailed result of the metric
- *-finer.txt contains all possible results of the metric
- *-finest.txt contains all possible results of the metric including meta information
IntelliJ Analysis Helper
Intellij can (by default) only handle file up to 2.6MB. As the result file are in many cases larger than this threshold, you can increase the size of files that should be analyzed by IntelliJ. You have to restart IntelliJ for the changes to take effect.
```
custom IntelliJ IDEA properties (expand/override 'bin\idea.properties')
idea.max.intellisense.filesize=999999 ```
(Optional) Git Hooks
If you want to use our proposed Git Hooks you can execute the following command:
shell
git config --local core.hooksPath .githooks
Owner
- Name: AQUA - Automated Quality Assurance
- Login: tudo-aqua
- Kind: organization
- Repositories: 17
- Profile: https://github.com/tudo-aqua
Citation (CITATION.cff)
# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!
cff-version: 1.2.0
title: >-
Tree-Based Scenario Classification - A Formal Framework for Measuring Domain Coverage When Testing Autonomous Systems
message: >-
If you use this software, please cite it using the
metadata from this file.
type: software
authors:
- given-names: Till
family-names: Schallau
email: till.schallau@tu-dortmund.de
affiliation: TU Dortmund University
orcid: 'https://orcid.org/0000-0002-1769-3486'
- given-names: Stefan
family-names: Naujokat
email: stefan.naujokat@tu-dortmund.de
affiliation: TU Dortmund University
orcid: 'https://orcid.org/0000-0002-6265-6641'
- given-names: Fiona
family-names: Kullmann
email: fiona.kullmann@tu-dortmund.de
affiliation: TU Dortmund University
orcid: 'https://orcid.org/0000-0001-5858-0659'
- given-names: Falk
family-names: Howar
email: falk.howar@tu-dortmund.de
affiliation: 'TU Dortmund University, Fraunhofer ISST'
orcid: 'https://orcid.org/0000-0002-9524-4459'
repository-code: 'https://github.com/tudo-aqua/stars'
abstract: >-
Scenario-based testing is envisioned as a key approach for the
safety assurance of automated driving systems. In scenario-based testing,
relevant (driving) scenarios are the basis of tests. Many recent works
focus on specification, variation, generation, and execution of individual
scenarios. In this work, we address the open challenges of classifying
sets of recorded test drives into such scenarios and measuring scenario
coverage in these test drives. Technically, we specify features in logic
formulas over complex data streams and construct tree-based classifiers
for scenarios from these feature specifications. For such specifications,
we introduce CMFTBL, a new logic that extends existing linear-time
temporal logics with aspects that are essential for concise specifications
that work on field-recorded data. We demonstrate the expressiveness
and effectiveness of our approach by defining a family of related scenario
classifiers for different aspects of urban driving.
keywords:
- automated driving systems
- temporal logic
- scenario classification
- autonomous-driving
- carla
license: Apache-2.0
preferred-citation:
authors:
- given-names: Till
family-names: Schallau
email: till.schallau@tu-dortmund.de
affiliation: TU Dortmund University
orcid: 'https://orcid.org/0000-0002-1769-3486'
- given-names: Stefan
family-names: Naujokat
email: stefan.naujokat@tu-dortmund.de
affiliation: TU Dortmund University
orcid: 'https://orcid.org/0000-0002-6265-6641'
- given-names: Fiona
family-names: Kullmann
email: fiona.kullmann@tu-dortmund.de
affiliation: TU Dortmund University
orcid: 'https://orcid.org/0000-0001-5858-0659'
- given-names: Falk
family-names: Howar
email: falk.howar@tu-dortmund.de
affiliation: 'TU Dortmund University, Fraunhofer ISST'
orcid: 'https://orcid.org/0000-0002-9524-4459'
title: >-
Tree-Based Scenario Classification - A Formal Framework for Measuring Domain Coverage When Testing Autonomous Systems
type: conference-paper
year: 2024
doi: '10.1007/978-3-031-60698-4_15'
collection-title: NASA Formal Methods Symposium
publisher:
name: Springer Nature Switzerland
pages: 259-278
isbn: '978-3-031-60697-7'
editors:
- family-names: Nathaniel
given-names: Benz
- family-names: Divya
given-names: Gopinath
- family-names: Nija
given-names: Shi
GitHub Events
Total
- Create event: 13
- Release event: 1
- Issues event: 2
- Watch event: 1
- Delete event: 7
- Issue comment event: 2
- Push event: 34
- Pull request review event: 4
- Pull request event: 20
- Fork event: 4
Last Year
- Create event: 13
- Release event: 1
- Issues event: 2
- Watch event: 1
- Delete event: 7
- Issue comment event: 2
- Push event: 34
- Pull request review event: 4
- Pull request event: 20
- Fork event: 4
Committers
Last synced: almost 2 years ago
Top Committers
| Name | Commits | |
|---|---|---|
| Till Schallau | t****u@t****e | 16 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 4 months ago
All Time
- Total issues: 6
- Total pull requests: 20
- Average time to close issues: about 1 month
- Average time to close pull requests: 11 days
- Total issue authors: 2
- Total pull request authors: 3
- Average comments per issue: 0.0
- Average comments per pull request: 0.2
- Merged pull requests: 12
- Bot issues: 0
- Bot pull requests: 14
Past Year
- Issues: 3
- Pull requests: 13
- Average time to close issues: N/A
- Average time to close pull requests: 21 days
- Issue authors: 2
- Pull request authors: 2
- Average comments per issue: 0.0
- Average comments per pull request: 0.15
- Merged pull requests: 6
- Bot issues: 0
- Bot pull requests: 11
Top Authors
Issue Authors
- tillschallau (5)
- dominikmaeckel (1)
Pull Request Authors
- dependabot[bot] (16)
- tillschallau (8)
- dominikmaeckel (4)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- tools.aqua:stars-core $starsVersion implementation
- tools.aqua:stars-data-av $starsVersion implementation
- tools.aqua:stars-import-carla $starsVersion implementation
- tools.aqua:stars-logic-kcmftbl $starsVersion implementation