dnn-tip

A collection of dnn test input prioritizers often used as benchmarks in recent literature.

https://github.com/testingautomated-usi/dnn-tip

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 4 DOI reference(s) in README
✓
Academic publication links
Links to: zenodo.org
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (13.4%) to scientific vocabulary

Last synced: 10 months ago · JSON representation ·

Repository

A collection of dnn test input prioritizers often used as benchmarks in recent literature.

Basic Info

Host: GitHub
Owner: testingautomated-usi
License: mit
Language: Python
Default Branch: main
Homepage: https://testingautomated-usi.github.io/dnn-tip/
Size: 23.4 KB

Statistics

Stars: 18
Watchers: 0
Forks: 1
Open Issues: 1
Releases: 2

Created about 4 years ago · Last pushed over 3 years ago

Metadata Files

Readme License Citation

DNN-TIP: Common Test Input Prioritizers Library

docstr-coverage

Implemented Approaches

Surprise Adequacies
- Distance-based Surprise Adequacy (DSA)
- Likelihood-based Surprise Adequacy (LSA)
- MultiModal-Likelihood-based Surprise Adequacy (MLSA)
- Mahalanobis-based Surprise Adequacy (MDSA)
- abstract MultiModal Surprise Adequacy
Surprise Coverage
- Neuron-Activation Coverage (NAC)
- K-Multisection Neuron Coverage (KMNC)
- Neuron Boundary Coverage (NBC)
- Strong Neuron Activation Coverage (SNAC)
- Top-k Neuron Coverage (TKNC)
Utilities
- APFD calculation
- Coverage-Added and Coverage-Total Prioritization Methods (CAM and CTM)

If you are looking for the uncertainty metrics we also tested (including DeepGini), head over to the sister repository uncertainty-wizard.

If you want to reproduce our exact experiments, there's a reproduction package and docker stuff available at testingautomated-usi/simple-tip.

Installation

It's as easy as pip install dnn-tip.

Documentation

Find the documentation at https://testingautomated-usi.github.io/dnn-tip/.

Citation

Here's the reference to the paper as part of which this library was release:

``` @inproceedings{10.1145/3533767.3534375, author = {Weiss, Michael and Tonella, Paolo}, title = {Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)}, year = {2022}, isbn = {9781450393799}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3533767.3534375}, doi = {10.1145/3533767.3534375}, booktitle = {Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis}, pages = {139–150}, numpages = {12}, keywords = {neural networks, Test prioritization, uncertainty quantification}, location = {Virtual, South Korea}, series = {ISSTA 2022} }

Owner

Name: testingautomated-usi
Login: testingautomated-usi
Kind: organization

Repositories: 11
Profile: https://github.com/testingautomated-usi

Citation (CITATION.cff)

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: >-
  Simple Techniques Work Surprisingly Well for Neural
  Network Test Prioritization and Active Learning
  (Replication Paper)
message: >-
  When using this software, please cite paper from
  which this software is an artifact (
      Simple Techniques Work Surprisingly Well for Neural
      Network Test Prioritization and Active Learning
      (Replication Paper), to appear at ISSTA 2022.
  )
type: software
authors:
  - given-names: Michael
    family-names: Weiss
    email: michael.weiss@usi.ch
    affiliation: Università della Svizzera italiana
    orcid: 'https://orcid.org/0000-0002-8944-389X'
  - given-names: Paolo
    family-names: Tonella
    email: paolo.tonella@usi.ch
    affiliation: Università della Svizzera italiana
    orcid: 'https://orcid.org/0000-0003-3088-0339'
preferred-citation:
  type: article
  authors:
  - given-names: Michael
    family-names: Weiss
    email: michael.weiss@usi.ch
    affiliation: Università della Svizzera italiana
    orcid: 'https://orcid.org/0000-0002-8944-389X'
  - given-names: Paolo
    family-names: Tonella
    email: paolo.tonella@usi.ch
    affiliation: Università della Svizzera italiana
    orcid: 'https://orcid.org/0000-0003-3088-0339'
  journal: "Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis"
  title: "Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replication Paper)"
  year: 2022

GitHub Events

Total

Watch event: 3

Last Year

Watch event: 3

Issues and Pull Requests

Last synced: 11 months ago

All Time

Total issues: 1
Total pull requests: 4
Average time to close issues: N/A
Average time to close pull requests: 3 days
Total issue authors: 1
Total pull request authors: 1
Average comments per issue: 0.0
Average comments per pull request: 0.25
Merged pull requests: 4
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

liuyunhui123 (1)

Pull Request Authors

MiWeiss (4)

Top Labels

Issue Labels

Pull Request Labels

Packages

Total packages: 1
Total downloads:
- pypi 92 last-month

Total dependent packages: 0
Total dependent repositories: 1
Total versions: 2
Total maintainers: 1

pypi.org: dnn-tip

A collection of DNN test input prioritizers,in particular neuron coverage and surprise adequacy.

Homepage: https://github.com/testingautomated-usi/dnn-tip
Documentation: https://dnn-tip.readthedocs.io/
License: MIT
Latest release: 0.1.1
published over 3 years ago

Versions: 2
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 92 Last month

Rankings

Dependent packages count: 10.1%

Stargazers count: 16.5%

Average: 21.0%

Dependent repos count: 21.6%

Forks count: 22.6%

Downloads: 34.2%

Maintainers (1)

mweiss

Last synced: 10 months ago

Dependencies

setup.py pypi

psutil *

docs/Gemfile rubygems

jekyll-feed ~> 0.12 development
http_parser.rb ~> 0.6.0
jekyll ~> 4.2.2
jekyll-remote-theme >= 0
minima ~> 2.5
rake >= 0
tzinfo ~> 1.2
tzinfo-data >= 0
wdm ~> 0.1.1

docs/Gemfile.lock rubygems

addressable 2.8.0
colorator 1.1.0
concurrent-ruby 1.1.10
em-websocket 0.5.3
eventmachine 1.2.7
ffi 1.15.5
forwardable-extended 2.6.0
http_parser.rb 0.8.0
i18n 1.10.0
jekyll 4.2.2
jekyll-feed 0.16.0
jekyll-remote-theme 0.4.3
jekyll-sass-converter 2.2.0
jekyll-seo-tag 2.8.0
jekyll-watch 2.2.1
kramdown 2.4.0
kramdown-parser-gfm 1.1.0
liquid 4.0.3
listen 3.7.1
mercenary 0.4.0
minima 2.5.1
pathutil 0.16.2
public_suffix 4.0.7
rake 13.0.6
rb-fsevent 0.11.1
rb-inotify 0.10.1
rexml 3.2.5
rouge 3.28.0
rubyzip 2.3.2
safe_yaml 1.0.5
sassc 2.4.0
terminal-table 2.0.0
unicode-display_width 1.8.0

.github/workflows/test.yml actions

actions/checkout v2 composite
actions/setup-python v1 composite

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science