Black-it

Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models - Published in JOSS (2022)

https://github.com/bancaditalia/black-it

Science Score: 93.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
✓
DOI references
Found 8 DOI reference(s) in README and JOSS metadata
✓
Academic publication links
Links to: joss.theoj.org
○
Committers with academic emails
○
Institutional organization owner
✓
JOSS paper metadata
Published in Journal of Open Source Software

Keywords

agent-based agent-based-modeling agent-based-simulation calibration-toolbox optimization

Scientific Fields

Sociology Social Sciences - 93% confidence

Economics Social Sciences - 85% confidence

Artificial Intelligence and Machine Learning Computer Science - 69% confidence

Last synced: 4 months ago · JSON representation

Repository

Black-box abm calibration kit by the Bank of Italy

Basic Info

Host: GitHub
Owner: bancaditalia
License: agpl-3.0
Language: Python
Default Branch: main
Homepage: https://bancaditalia.github.io/black-it/
Size: 26.1 MB

Statistics

Stars: 53
Watchers: 7
Forks: 2
Open Issues: 10
Releases: 7

Topics

agent-based agent-based-modeling agent-based-simulation calibration-toolbox optimization

Created over 3 years ago · Last pushed 7 months ago

Metadata Files

Readme Changelog Contributing License Authors

Black-box abm calibration kit

Black-it is an easy-to-use toolbox designed to help you calibrate the parameters in your agent-based models and simulations (ABMs), using state-of-the-art techniques to sample the parameter search space, with no need to reinvent the wheel.

Models from economics, epidemiology, biology, logistics, and more can be dealt with. The software can be used as-is - if your main interest is the ABM model itself. However, in case your research thing is to, e.g., devise new sampling strategies for ginormous search spaces and highly non-linear model, then you can deploy and test your new ideas on a solid, reusable, modular foundation, in a matter of days, with no need to reimplement all the plumbings from scratch.

Installation

This project requires Python v3.9 or later.

To install the latest version of the package from PyPI: pip install black-it

Or, directly from GitHub:

pip install git+https://github.com/bancaditalia/black-it.git#egg=black-it

If you'd like to contribute to the package, please read the CONTRIBUTING.md guide.

Quick Example

The GitHub repo of Black-it contains a series ready-to-run calibration examples.

To experiment with them, simply clone the repo and enter the examples folder

git clone https://github.com/bancaditalia/black-it.git cd black-it/examples

You'll find several scripts and notebooks. The following is the script named main.py, note that copying and pasting the lines below will not work in general as the script needs to be inside the "examples" folder in order to run correctly.

```python import models.simple_models as md

from blackit.calibrator import Calibrator from blackit.lossfunctions.msm import MethodOfMomentsLoss from blackit.samplers.bestbatch import BestBatchSampler from blackit.samplers.halton import HaltonSampler from blackit.samplers.randomforest import RandomForestSampler

trueparams = [0.20, 0.20, 0.75] bounds = [ [0.10, 0.10, 0.10], # LOWER bounds [1.00, 1.00, 1.00], # UPPER bounds ] boundsstep = [0.01, 0.01, 0.01] # Step size in range between bounds

batchsize = 8 haltonsampler = HaltonSampler(batchsize=batchsize) randomforestsampler = RandomForestSampler(batchsize=batchsize) bestbatchsampler = BestBatchSampler(batchsize=batchsize)

define a model to be calibrated

model = md.MarkovC_KP

generate a synthetic dataset to test the calibrator

N = 2000 seed = 1 realdata = model(trueparams, N, seed)

define a loss

loss = MethodOfMomentsLoss()

define the calibration seed

calibration_seed = 1

initialize a Calibrator object

cal = Calibrator( samplers=[haltonsampler, randomforestsampler, bestbatchsampler], realdata=realdata, model=model, parametersbounds=bounds, parametersprecision=boundsstep, ensemblesize=3, lossfunction=loss, randomstate=calibrationseed, )

calibrate the model

params, losses = cal.calibrate(n_batches=15)

print(f"True parameters: {true_params}") print(f"Best parameters found: {params[0]}") ```

When the calibration terminates (~half a minute), towards the end of the output you should see the following messages: True parameters: [0.2, 0.2, 0.75] Best parameters found: [0.19 0.21 0.68]

Docs

Black-it calibration is initiated via the Calibrator which, when called, performs three main steps.

First, a Sampler is summoned to suggest a set of promising parameter configurations to explore.

Second, the model to be calibrated is simulated for all the selected parameters.

Third, a specific loss function, measuring the goodness of fitness of the simulation data with respect to the real data, is evaluated.

These steps are performed in a loop, and this allows the samplers to progress towards better parameter values by exploiting the knowledge of previously computed loss functions.

A more detailed explanation of how Black-it works is available here, while the full documentation -complete with examples and tutorials- is available here.

Citing Black-it

A description of the package is available here.

Please consider citing it if you found this package useful for your research

bib @article{black_it, title = {Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models}, journal = {Journal of Open Source Software}, publisher = {The Open Journal}, year = {2022}, volume = {7}, number = {79}, pages = {4622}, doi = {10.21105/joss.04622}, url = {https://doi.org/10.21105/joss.04622}, author = {Marco Benedetti and Gennaro Catapano and Francesco {De Sclavis} and Marco Favorito and Aldo Glielmo and Davide Magnanimi and Antonio Muci} }

Disclaimer

This package is an outcome of a research project. All errors are those of the authors. All views expressed are personal views, not those of Bank of Italy.

* Credits to Sara Corbo for the logo.

Owner

Name: Banca d'Italia
Login: bancaditalia
Kind: organization
Location: Rome

Repositories: 1
Profile: https://github.com/bancaditalia

JOSS Publication

Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models

Published

November 03, 2022

DOI

10.21105/joss.04622

Volume 7, Issue 79, Page 4622

Authors

Marco Benedetti
Banca d'Italia, Italy

Gennaro Catapano
Banca d'Italia, Italy

Francesco De Sclavis
Banca d'Italia, Italy

Marco Favorito

Banca d'Italia, Italy

Aldo Glielmo

Banca d'Italia, Italy

Davide Magnanimi

Banca d'Italia, Italy

Antonio Muci
Banca d'Italia, Italy

Editor

Vincent Knight

GitHub Events

Total

Create event: 17
Release event: 1
Issues event: 6
Watch event: 8
Delete event: 18
Issue comment event: 23
Push event: 98
Pull request event: 20

Last Year

Create event: 17
Release event: 1
Issues event: 6
Watch event: 8
Delete event: 18
Issue comment event: 23
Push event: 98
Pull request event: 20

Committers

Last synced: 5 months ago

All Time

Total Commits: 324
Total Committers: 5
Avg Commits per committer: 64.8
Development Distribution Score (DDS): 0.611

Past Year

Commits: 76
Committers: 2
Avg Commits per committer: 38.0
Development Distribution Score (DDS): 0.171

Top Committers

Name	Email	Commits
Marco Favorito	m**o@g**m	126
Aldo	e**o@g**m	79
muxator	a**i@b**t	69
Marco Favorito	m**o@b**t	49
mabene-BI	m**i@b**t	1

Committer Domains (Top 20 + Academic)

bancaditalia.it: 3

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 11
Total pull requests: 114
Average time to close issues: 9 months
Average time to close pull requests: 21 days
Total issue authors: 5
Total pull request authors: 9
Average comments per issue: 0.64
Average comments per pull request: 1.46
Merged pull requests: 91
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 4
Pull requests: 37
Average time to close issues: N/A
Average time to close pull requests: 8 days
Issue authors: 3
Pull request authors: 3
Average comments per issue: 0.75
Average comments per pull request: 0.65
Merged pull requests: 24
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

muxator (4)
marcofavoritobi (3)
SGHoekstra (2)
linwenqi2024 (1)
wangy8989 (1)

Pull Request Authors

marcofavoritobi (36)
AldoGl (35)
muxator (34)
aldoglielmo (2)
marcofavorito (2)
mmartinoli87 (2)
JLDC (1)
xuanxu (1)
danielskatz (1)

Top Labels

Issue Labels

development (1)

Pull Request Labels

enhancement (1)

Packages

Total packages: 1
Total downloads:
- pypi 56 last-month

Total dependent packages: 0
Total dependent repositories: 1
Total versions: 8
Total maintainers: 1

pypi.org: black-it

black-it: Black-box abm calibration kit

Homepage: https://github.com/bancaditalia/black-it
Documentation: https://bancaditalia.github.io/black-it
License: AGPL-3.0-or-later
Latest release: 0.3.2
published 9 months ago

Versions: 8
Dependent Packages: 0
Dependent Repositories: 1
Downloads: 56 Last month

Rankings

Dependent packages count: 7.3%

Stargazers count: 11.2%

Forks count: 19.2%

Dependent repos count: 22.1%

Average: 23.4%

Downloads: 57.2%

Maintainers (1)

bancaditalia

Last synced: 5 months ago

Dependencies

.github/workflows/docs.yml actions

actions/checkout master composite
actions/setup-python master composite

.github/workflows/lint.yml actions

actions/checkout master composite
actions/setup-python master composite

.github/workflows/test.yml actions

actions/checkout master composite
actions/setup-python master composite
codecov/codecov-action v3 composite

poetry.lock pypi

201 dependencies

pyproject.toml pypi

Markdown ^3.3.7 develop
Mesa ^0.9.0 develop
bandit ^1.7.4 develop
black ^22.3.0 develop
codecov ^2.1.12 develop
darglint ^1.8.1 develop
flake8 ^4.0.1 develop
flake8-bugbear ^22.4.25 develop
flake8-docstrings ^1.6.0 develop
flake8-eradicate ^1.2.1 develop
flake8-isort ^4.1.1 develop
hypothesis ^6.46.5 develop
ipython ^8.3.0 develop
isort ^5.10.1 develop
jupyter ^1.0.0 develop
markdown-include ^0.6.0 develop
mistletoe ^0.8.2 develop
mkdocs ^1.3.0 develop
mkdocs-autorefs ^0.4.1 develop
mkdocs-material ^8.2.15 develop
mkdocstrings ^0.18.1 develop
mknotebooks ^0.7.1 develop
mypy ^0.950 develop
nbmake ^1.3.0 develop
ndlib ^5.1.1 develop
pydocstyle ^6.1.1 develop
pylint ^2.13.9 develop
pymdown-extensions ^9.4 develop
pytest ^7.1.2 develop
pytest-cov ^3.0.0 develop
pytest-randomly ^3.12.0 develop
safety ^1.10.3 develop
tox ^3.25.0 develop
twine ^4.0.0 develop
vulture ^2.3 develop
GPy ^1.10.0
ipywidgets ^7.7.0
matplotlib ^3.5.2
numpy ^1.22.3
pandas ^1.4.2
python >=3.8,<3.11
scikit-learn ^1.1.0
seaborn ^0.11.2
statsmodels ^0.13.2

Black-it

Science Score: 93.0%

Keywords

Scientific Fields

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

Black-box abm calibration kit

Installation

Quick Example

define a model to be calibrated

generate a synthetic dataset to test the calibrator

define a loss

define the calibration seed

initialize a Calibrator object

calibrate the model

Docs

Citing Black-it

Disclaimer

Owner

JOSS Publication

Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models

Authors

Editor

Tags

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: black-it

Rankings

Maintainers (1)

Dependencies