Foolbox Native

Foolbox Native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX - Published in JOSS (2020)

https://github.com/bethgelab/foolbox

Science Score: 95.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 9 DOI reference(s) in README and JOSS metadata
✓
Academic publication links
Links to: arxiv.org, joss.theoj.org
✓
Committers with academic emails
4 of 35 committers (11.4%) from academic institutions
○
Institutional organization owner
✓
JOSS paper metadata
Published in Journal of Open Source Software

Keywords

adversarial-attacks adversarial-examples jax keras machine-learning python pytorch tensorflow

Keywords from Contributors

mesh

Scientific Fields

Mathematics Computer Science - 84% confidence

Last synced: 4 months ago · JSON representation

Repository

A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX

Basic Info

Host: GitHub
Owner: bethgelab
License: mit
Language: Python
Default Branch: master
Homepage: https://foolbox.jonasrauber.de
Size: 10.7 MB

Statistics

Stars: 2,895
Watchers: 46
Forks: 433
Open Issues: 29
Releases: 59

Topics

adversarial-attacks adversarial-examples jax keras machine-learning python pytorch tensorflow

Created over 8 years ago · Last pushed over 1 year ago

Metadata Files

Readme Funding License

README.rst

.. raw:: html

   

.. image:: https://badge.fury.io/py/foolbox.svg
   :target: https://badge.fury.io/py/foolbox

.. image:: https://readthedocs.org/projects/foolbox/badge/?version=latest
    :target: https://foolbox.readthedocs.io/en/latest/

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
   :target: https://github.com/ambv/black

.. image:: https://joss.theoj.org/papers/10.21105/joss.02607/status.svg
   :target: https://doi.org/10.21105/joss.02607

===============================================================================================================================
Foolbox: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX
===============================================================================================================================

`Foolbox `_ is a **Python library** that lets you easily run adversarial attacks against machine learning models like deep neural networks. It is built on top of EagerPy and works natively with models in `PyTorch `_, `TensorFlow `_, and `JAX `_.

🔥 Design 
----------

**Foolbox 3** has been rewritten from scratch
using `EagerPy `_ instead of
NumPy to achieve native performance on models
developed in PyTorch, TensorFlow and JAX, all with one code base without code duplication.

- **Native Performance**: Foolbox 3 is built on top of EagerPy and runs natively in PyTorch, TensorFlow, and JAX and comes with real batch support.
- **State-of-the-art attacks**: Foolbox provides a large collection of state-of-the-art gradient-based and decision-based adversarial attacks.
- **Type Checking**: Catch bugs before running your code thanks to extensive type annotations in Foolbox.

📖 Documentation
-----------------

- **Guide**: The best place to get started with Foolbox is the official `guide `_.
- **Tutorial**: If you are looking for a tutorial, check out this `Jupyter notebook `_ |colab|.
- **Documentation**: The API documentation can be found on `ReadTheDocs `_.

.. |colab| image:: https://colab.research.google.com/assets/colab-badge.svg
   :target: https://colab.research.google.com/github/jonasrauber/foolbox-native-tutorial/blob/master/foolbox-native-tutorial.ipynb

🚀 Quickstart
--------------

.. code-block:: bash

   pip install foolbox

Foolbox is tested with Python 3.8 and newer - however, it will most likely also work with version 3.6 - 3.8. To use it with `PyTorch `_, `TensorFlow `_, or `JAX `_, the respective framework needs to be installed separately. These frameworks are not declared as dependencies because not everyone wants to use and thus install all of them and because some of these packages have different builds for different architectures and CUDA versions. Besides that, all essential dependencies are automatically installed.

You can see the versions we currently use for testing in the `Compatibility section <#-compatibility>`_ below, but newer versions are in general expected to work.

🎉 Example
-----------

.. code-block:: python

   import foolbox as fb

   model = ...
   fmodel = fb.PyTorchModel(model, bounds=(0, 1))

   attack = fb.attacks.LinfPGD()
   epsilons = [0.0, 0.001, 0.01, 0.03, 0.1, 0.3, 0.5, 1.0]
   _, advs, success = attack(fmodel, images, labels, epsilons=epsilons)


More examples can be found in the `examples <./examples/>`_ folder, e.g.
a full `ResNet-18 example <./examples/single_attack_pytorch_resnet18.py>`_.

📄 Citation
------------

If you use Foolbox for your work, please cite our `JOSS paper on Foolbox Native (i.e., Foolbox 3.0) `_ and our `ICML workshop paper on Foolbox `_ using the following BibTeX entries:

.. code-block::

   @article{rauber2017foolboxnative,
     doi = {10.21105/joss.02607},
     url = {https://doi.org/10.21105/joss.02607},
     year = {2020},
     publisher = {The Open Journal},
     volume = {5},
     number = {53},
     pages = {2607},
     author = {Jonas Rauber and Roland Zimmermann and Matthias Bethge and Wieland Brendel},
     title = {Foolbox Native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX},
     journal = {Journal of Open Source Software}
   }

.. code-block::

   @inproceedings{rauber2017foolbox,
     title={Foolbox: A Python toolbox to benchmark the robustness of machine learning models},
     author={Rauber, Jonas and Brendel, Wieland and Bethge, Matthias},
     booktitle={Reliable Machine Learning in the Wild Workshop, 34th International Conference on Machine Learning},
     year={2017},
     url={http://arxiv.org/abs/1707.04131},
   }


👍 Contributions
-----------------

We welcome contributions of all kind, please have a look at our
`development guidelines `_.
In particular, you are invited to contribute
`new adversarial attacks `_.
If you would like to help, you can also have a look at the issues that are
marked with `contributions welcome
`_.

💡 Questions?
--------------

If you have a question or need help, feel free to open an issue on GitHub.
Once GitHub Discussions becomes publicly available, we will switch to that.

💨 Performance
--------------

Foolbox 3.0 is much faster than Foolbox 1 and 2. A basic `performance comparison`_ can be found in the `performance` folder.

🐍 Compatibility
-----------------

We currently test with the following versions:

* PyTorch 1.10.1
* TensorFlow 2.6.3
* JAX 0.2.517
* NumPy 1.18.1

.. _performance comparison: performance/README.md

Owner

Name: Bethge Lab
Login: bethgelab
Kind: organization
Location: Tübingen

Website: http://bethgelab.org
Repositories: 23
Profile: https://github.com/bethgelab

Perceiving Neural Networks

JOSS Publication

Foolbox Native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX

Published

September 27, 2020

DOI

10.21105/joss.02607

Volume 5, Issue 53, Page 2607

Authors

Jonas Rauber

Tübingen AI Center, University of Tübingen, Germany, International Max Planck Research School for Intelligent Systems, Tübingen, Germany

Roland Zimmermann
Tübingen AI Center, University of Tübingen, Germany, International Max Planck Research School for Intelligent Systems, Tübingen, Germany

Matthias Bethge
Tübingen AI Center, University of Tübingen, Germany, Bernstein Center for Computational Neuroscience Tübingen, Germany

Wieland Brendel
Tübingen AI Center, University of Tübingen, Germany, Bernstein Center for Computational Neuroscience Tübingen, Germany

Editor

Yuan Tang

GitHub Events

Total

Issues event: 2
Watch event: 146
Issue comment event: 1
Pull request event: 2
Fork event: 15

Last Year

Issues event: 2
Watch event: 146
Issue comment event: 1
Pull request event: 2
Fork event: 15

Committers

Last synced: 5 months ago

All Time

Total Commits: 1,437
Total Committers: 35
Avg Commits per committer: 41.057
Development Distribution Score (DDS): 0.364

Past Year

Commits: 0
Committers: 0
Avg Commits per committer: 0.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
Jonas Rauber	g**t@j**e	914
FlashTek	r**n@g**m	199
Jonas Rauber	j****r	110
Wieland Brendel	w****l	57
Behar Veliqi	b**r@v**e	41
Evgenia Rusak	r**k@f**e	28
Jianbo Chen	j**n@b**u	15
renebidart	r**t@g**m	11
dependabot[bot]	4****]	8
Jerome Rony	j**1@e**t	7
maurapintor	m**r@g**m	7
Ben Feinstein	f**n@g**m	5
Eric Meissner	e**s@a**m	4
LarsHoldijk	l**k@g**m	4
lukas.schott	l**t@g**m	3
Carl Johann Simon-Gabriel	c**n@t**e	2
Apostolos Modas	a**s@e**h	2
Roland Zimmermann	F****k	2
yaoshiang	y**g@g**m	2
Aidan Kelley	a**y@g**m	1
Alexander Matyasko	a****t	1
Wieland Brendel	w**d@W**x	1
ChrisP	1****n	1
Dario Serez	6****D	1
Dmytro Mishkin	d**i@g**m	1
Jan Zikes	z**0@g**m	1
Luca Demetrio	l**o@o**t	1
Weiguang (Gavin) Ding	g****g	1
Will Greedy	w**5@s**k	1
Xuefei Ning	f**e@g**m	1
and 5 more...

Committer Domains (Top 20 + Academic)

soton.ac.uk: 1 outlook.it: 1 wielands-mbp.fritz.box: 1 epfl.ch: 1 tuebingen.mpg.de: 1 amazon.com: 1 etsmtl.net: 1 berkeley.edu: 1 fzi.de: 1 veliqi.de: 1 jonasrauber.de: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 80
Total pull requests: 59
Average time to close issues: 12 months
Average time to close pull requests: 3 months
Total issue authors: 69
Total pull request authors: 10
Average comments per issue: 2.81
Average comments per pull request: 1.37
Merged pull requests: 40
Bot issues: 0
Bot pull requests: 12

Past Year

Issues: 2
Pull requests: 2
Average time to close issues: N/A
Average time to close pull requests: 1 day
Issue authors: 1
Pull request authors: 2
Average comments per issue: 0.0
Average comments per pull request: 0.0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

Changgun-Choi (3)
John-Chin (3)
lmsasu (3)
jS5t3r (2)
volmodaoist (2)
pgg3 (2)
pratyushmaini (2)
zimmerrol (2)
giangnguyen2412 (1)
jangop (1)
skat00sh (1)
KarthikGanesan88 (1)
scantaio (1)
Buhua-Liu (1)
ck-amrahd (1)

Pull Request Authors

zimmerrol (46)
dependabot[bot] (13)
Srong1080p (2)
lcycode (2)
SerezD (2)
p-chaurand (2)
Mole1424 (2)
zhuangzi926 (1)
wielandbrendel (1)
RobinTher (1)

Top Labels

Issue Labels

question (18) feature request (6) bug (4) enhancement (4) waiting for reply (4) contributions welcome (4) help wanted (1)

Pull Request Labels

dependencies (13) enhancement (1)

Packages

Total packages: 3
Total downloads:
- pypi 4,305 last-month
Total docker downloads: 405

Total dependent packages: 4
(may contain duplicates)
Total dependent repositories: 132
(may contain duplicates)
Total versions: 126
Total maintainers: 2

pypi.org: foolbox

Foolbox is an adversarial attacks library that works natively with PyTorch, TensorFlow and JAX

Homepage: https://github.com/bethgelab/foolbox
Documentation: https://foolbox.readthedocs.io/
License: MIT License
Latest release: 3.3.4
published almost 2 years ago

Versions: 71
Dependent Packages: 3
Dependent Repositories: 132
Downloads: 4,305 Last month
Docker Downloads: 405

Rankings

Dependent repos count: 1.3%

Stargazers count: 1.4%

Docker downloads count: 1.8%

Forks count: 2.6%

Average: 2.7%

Dependent packages count: 3.2%

Downloads: 5.6%

Maintainers (2)

jonasrauber zimmerrol

Last synced: 4 months ago

proxy.golang.org: github.com/bethgelab/foolbox

Documentation: https://pkg.go.dev/github.com/bethgelab/foolbox#section-documentation
License: mit
Latest release: v3.3.4+incompatible
published almost 2 years ago

Versions: 50
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.4%

Average: 5.6%

Dependent repos count: 5.8%

Last synced: 4 months ago

Foolbox is a Python library that let's you easily run adversarial attacks against machine learning models like deep neural networks. It is built on top of EagerPy and works natively with models in PyTorch, TensorFlow, JAX, and NumPy.

Homepage: https://foolbox.jonasrauber.de/
License: MIT
Latest release: 3.3.1
published almost 5 years ago

Versions: 5
Dependent Packages: 1
Dependent Repositories: 0

Rankings

Forks count: 7.4%

Stargazers count: 7.5%

Average: 19.4%

Dependent packages count: 28.8%

Dependent repos count: 34.0%

Last synced: 4 months ago

Dependencies

docs/requirements.txt pypi

sphinx ==4.5.0
sphinx-autobuild ==2021.3.14
sphinx-typlog-theme ==0.8.0
sphinx_rtd_theme ==1.0.0

requirements.txt pypi

black ==22.3.0
codecov ==2.1.12
coverage ==6.3.2
coveralls ==3.3.1
flake8 ==4.0.1
mypy ==0.942
pre-commit ==2.17.0
pytest ==7.1.1
pytest-cov ==3.0.0

setup.py pypi

GitPython >=3.0.7
eagerpy >=0.30.0
numpy *
requests >=2.24.0
scipy *
setuptools *
typing-extensions >=3.7.4.1

tests/requirements.txt pypi

jax ==0.2.17 test
matplotlib ==3.5.1 test
numba ==0.55.1 test
pillow ==9.0.1 test
responses ==0.10.9 test
tensorboardX ==2.0 test
tensorflow ==2.6.4 test
torch ==1.10.1 test
torchvision ==0.11.2 test

.github/workflows/docs.yml actions

actions/cache v1 composite
actions/checkout v2 composite
actions/setup-python v1 composite

.github/workflows/guide.yml actions

actions/checkout v2 composite

.github/workflows/pypi.yml actions

actions/checkout v1 composite
actions/setup-python v1 composite

.github/workflows/tests.yml actions

actions/cache v1 composite
actions/checkout v2 composite
actions/setup-python v1 composite

pyproject.toml pypi

Foolbox Native

Science Score: 95.0%

Keywords

Keywords from Contributors

Scientific Fields

Repository

Basic Info

Statistics

Topics

Metadata Files

README.rst

Owner

JOSS Publication

Foolbox Native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX

Authors

Editor

Tags

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: foolbox

Rankings

Maintainers (2)

proxy.golang.org: github.com/bethgelab/foolbox

Rankings

conda-forge.org: foolbox

Rankings

Dependencies