smac

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization

Science Score: 64.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org
✓
Committers with academic emails
16 of 42 committers (38.1%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (19.6%) to scientific vocabulary

Keywords

algorithm-configuration automated-machine-learning automl bayesian-optimisation bayesian-optimization configuration gaussian-process hyperparameter-optimization hyperparameter-search hyperparameter-tuning random-forest

Keywords from Contributors

algorithm-selection tabular-data benchmarking generic openml interactive meta-learning reinforcement-learning datascience optim

Last synced: 4 months ago · JSON representation ·

Repository

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization

Basic Info

Host: GitHub
Owner: automl
License: other
Language: Python
Default Branch: main
Homepage: https://automl.github.io/SMAC3/latest/
Size: 155 MB

Statistics

Stars: 1,188
Watchers: 40
Forks: 238
Open Issues: 116
Releases: 46

Topics

Created over 9 years ago · Last pushed 4 months ago

Metadata Files

Readme Changelog Contributing Funding License Citation

README.md

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization

SMAC offers a robust and flexible framework for Bayesian Optimization to support users in determining well-performing hyperparameter configurations for their (Machine Learning) algorithms, datasets and applications at hand. The main core consists of Bayesian Optimization in combination with an aggressive racing mechanism to efficiently decide which of two configurations performs better.

SMAC3 is written in Python3 and continuously tested with Python 3.8, 3.9, and 3.10 (and works with newer python versions). Its Random Forest is written in C++. In further texts, SMAC is representatively mentioned for SMAC3.

Documentation

Roadmap

Important: Changes in v2.0

With the next big major release of SMAC, we drastically boosted the user experience by improving the APIs and how the pipelining is done (see changelog). All facades/intensifiers support multi-objective, multi-fidelity, and multi-threading natively now! That includes having an ask-and-tell interface and continuing a run wherever you left off. pSMAC is removed because when specifying the number of workers, SMAC automatically uses multi-threading for evaluating trials. When cleaning the code base, however, we removed the command-line interface (calling a target function from a script is still supported), and runtime optimization. Also, python 3.7 is not supported anymore. If you depend on those functionalities, please keep using v1.4.

We are excited to introduce the new major release and look forward to developing new features on the new code base. We hope you enjoy this new user experience as much as we do. 🚀

Installation

This instruction is for the installation on a Linux system, for Windows and Mac and further information see the documentation.

Create a new environment with python 3.10 and make sure swig is installed either on your system or inside the environment. We demonstrate the installation via anaconda in the following:

Create and activate environment: conda create -n SMAC python=3.10 conda activate SMAC

Install swig: conda install gxx_linux-64 gcc_linux-64 swig

Install SMAC via PyPI: pip install smac

If you want to contribute to SMAC, use the following steps instead: git clone https://github.com/automl/SMAC3.git && cd SMAC3 make install-dev

Minimal Example

```py from ConfigSpace import Configuration, ConfigurationSpace

import numpy as np from smac import HyperparameterOptimizationFacade, Scenario from sklearn import datasets from sklearn.svm import SVC from sklearn.modelselection import crossval_score

iris = datasets.load_iris()

def train(config: Configuration, seed: int = 0) -> float: classifier = SVC(C=config["C"], randomstate=seed) scores = crossval_score(classifier, iris.data, iris.target, cv=5) return 1 - np.mean(scores)

configspace = ConfigurationSpace({"C": (0.100, 1000.0)})

Scenario object specifying the optimization environment

scenario = Scenario(configspace, deterministic=True, n_trials=200)

Use SMAC to find the best configuration/hyperparameters

smac = HyperparameterOptimizationFacade(scenario, train) incumbent = smac.optimize() ```

More examples can be found in the documentation.

Visualization via DeepCAVE

With DeepCAVE (Repo, Paper) you can visualize your SMAC runs. It is a visualization and analysis tool for AutoML (especially for the sub-problem hyperparameter optimization) runs.

License

This program is free software: you can redistribute it and/or modify it under the terms of the 3-clause BSD license (please see the LICENSE file).

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

You should have received a copy of the 3-clause BSD license along with this program (see LICENSE file). If not, see here.

Contacting us

If you have trouble using SMAC, a concrete question or found a bug, please create an issue. This is the easiest way to communicate about these things with us.

For all other inquiries, please write an email to smac[at]ai[dot]uni[dash]hannover[dot]de.

Miscellaneous

SMAC3 is developed by the AutoML Groups of the Universities of Hannover and Freiburg. It is a featured optimizer on AutoML Space.

If you have found a bug, please report to issues. Moreover, we are appreciating any kind of help. Find our guidelines for contributing to this package here.

If you use SMAC in one of your research projects, please cite our JMLR paper: @article{JMLR:v23:21-0888, author = {Marius Lindauer and Katharina Eggensperger and Matthias Feurer and André Biedenkapp and Difan Deng and Carolin Benjamins and Tim Ruhkopf and René Sass and Frank Hutter}, title = {SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization}, journal = {Journal of Machine Learning Research}, year = {2022}, volume = {23}, number = {54}, pages = {1--9}, url = {http://jmlr.org/papers/v23/21-0888.html} }

Owner

Name: AutoML-Freiburg-Hannover
Login: automl
Kind: organization
Location: Freiburg and Hannover, Germany

Website: www.automl.org
Repositories: 186
Profile: https://github.com/automl

Citation (CITATION.cff)

---
cff-version: 1.2.0

message: "If you used SMAC in one of your research projects, please cite us:"

title: "SMAC3"
date-released: "2016-08-17"

url: "https://automl.github.io/SMAC3/master/index.html"
repository-code: "https://github.com/automl/SMAC3"

version: "2.3.1"

type: "software"
keywords:
  - "blackbox optimization"
  - "optimization"
  - "bayesian optimization"
  - "algorithm configuration"
  - "machine learning"
  - "algorithms"

license: "BSD-3-Clause"

authors:
  - family-names: "Lindauer"
    given-names: "Marius"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Benjamins"
    given-names: "Carolin"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Biedenkapp"
    given-names: "André"
    orcid: "https://orcid.org/0000-0002-8703-8559"
    affiliation: "University of Freiburg, Germany"

  - family-names: "Deng"
    given-names: "Difan"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Eggensperger"
    given-names: "Katharina" 
    orcid: "https://orcid.org/0000-0002-0309-401X"
    affiliation: "University of Freiburg, Germany"

  - family-names: "Falkner"
    given-names: "Stefan"
    orcid: "https://orcid.org/0000-0002-6303-9418"
    affiliation: "Bosch Center for Artificial Intelligence, Rennigen, Germany"

  - family-names: "Feurer"
    given-names: "Matthias"
    orcid: "https://orcid.org/0000-0001-9611-8588"
    affiliation: "University of Freiburg, Germany"

  - family-names: "Graf"
    given-names: "Helena"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Ruhkopf"
    given-names: "Tim"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Sass"
    given-names: "René"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Segel"
    given-names: "Sarah"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Tornede"
    given-names: "Alexander"
    affiliation: "Leibniz University Hannover, Germany"

  - family-names: "Hutter"
    given-names: "Frank"
    affiliation: "University of Freiburg, Germany"

preferred-citation:
  type: "article"
  title: "SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization"
  journal: "Journal of Machine Learning Research"
  year: "2022"
  volume: "23"
  number: "54"
  start: "1"
  end: "9"
  url: "https://www.jmlr.org/papers/volume23/21-0888/21-0888.pdf"

  authors:
    - family-names: "Lindauer"
      given-names: "Marius"
      affiliation: "Leibniz University Hannover"

    - family-names: "Eggensperger"
      given-names: "Katharina" 
      orcid: "https://orcid.org/0000-0002-0309-401X"
      affiliation: "University of Freiburg"

    - family-names: "Feurer"
      given-names: "Matthias"
      orcid: "https://orcid.org/0000-0001-9611-8588"
      affiliation: "University of Freiburg"

    - family-names: "Biedenkapp"
      given-names: "André "
      orcid: "https://orcid.org/0000-0002-8703-8559"
      affiliation: "University of Freiburg"

    - family-names: "Deng"
      given-names: "Difan"
      affiliation: "Leibniz University Hannover"

    - family-names: "Benjamins"
      given-names: "Carolin"
      affiliation: "Leibniz University Hannover"

    - family-names: "Ruhkopf"
      given-names: "Tim"
      affiliation: "Leibniz University Hannover"

    - family-names: "Sass"
      given-names: "René"
      affiliation: "Leibniz University Hannover"

    - family-names: "Hutter"
      given-names: "Frank"
      affiliation: "University of Freiburg"
...

GitHub Events

Total

Create event: 44
Release event: 2
Issues event: 89
Watch event: 112
Delete event: 22
Member event: 4
Issue comment event: 160
Push event: 178
Pull request review comment event: 74
Pull request review event: 95
Pull request event: 65
Fork event: 23

Last Year

Create event: 44
Release event: 2
Issues event: 89
Watch event: 112
Delete event: 22
Member event: 4
Issue comment event: 160
Push event: 178
Pull request review comment event: 74
Pull request review event: 95
Pull request event: 65
Fork event: 23

Committers

Last synced: 6 months ago

All Time

Total Commits: 1,657
Total Committers: 42
Avg Commits per committer: 39.452
Development Distribution Score (DDS): 0.796

Past Year

Commits: 4
Committers: 2
Avg Commits per committer: 2.0
Development Distribution Score (DDS): 0.25

Top Committers

Name	Email	Commits
Matthias Feurer	f**m@i**e	338
Marius Lindauer	m**s@g**m	325
Katharina Eggensperger	e**k@i**e	305
Joshua Marben	j**n@n**e	196
biedenka	b**a@i**e	109
Marius Lindauer	l**r@i**e	97
Ashwin Raaghav	a**v@g**m	64
eddiebergman	e**s@g**m	51
Aaron Klein	k**a@c**e	23
René Sass	m**l@r**e	22
Marius Lindauer	l**r@c**e	20
Aaron Kimmig	k**a@i**e	20
Francisco Rivera Valverde	4****a	12
Carolin Benjamins	b**n@t**e	11
Jost Tobias Springenberg	s**j@i**e	11
René Sass	s**s@t**e	7
dengdifan	d**g@g**m	4
helegraf	h**f@a**e	4
PhMueller	m**l@g**t	4
dependabot[bot]	4****]	3
Sarah Segel	3****l	3
Jorn Tuyls	j**s@J**l	3
Anatolii Domashnev	a**v@g**m	2
lindauer	l**r@l**u	2
Jorn Tuyls	j**s@h**t	2
Stefan Heid	s**d@w**e	2
eggenspk	e**k@m**t	2
Frank Hutter	h**r@c**e	1
lindauer	l**r@a**m	1
Aaron Kimmig	1****g	1
and 12 more...

Committer Domains (Top 20 + Academic)

informatik.uni-freiburg.de: 6 cs.uni-freiburg.de: 3 ai.uni-hannover.de: 2 tnt.uni-hannover.de: 2 lohr-ffb.de: 1 dengster.tnt.uni-hannover.de: 1 freenet.de: 1 mllap06.informatik.privat: 1 host-10-125-1-182.eduroam-extern.privat: 1 gmx.net: 1 renesass.de: 1 neptun.uni-freiburg.de: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time

Total issues: 195
Total pull requests: 156
Average time to close issues: about 1 year
Average time to close pull requests: 22 days
Total issue authors: 83
Total pull request authors: 28
Average comments per issue: 2.48
Average comments per pull request: 0.71
Merged pull requests: 124
Bot issues: 0
Bot pull requests: 13

Past Year

Issues: 45
Pull requests: 62
Average time to close issues: about 1 month
Average time to close pull requests: 13 days
Issue authors: 18
Pull request authors: 12
Average comments per issue: 1.76
Average comments per pull request: 0.34
Merged pull requests: 47
Bot issues: 0
Bot pull requests: 5

View more stats

Top Authors

Issue Authors

benjamc (32)
helegraf (17)
bbudescu (11)
LukasFehring (11)
alexandertornede (6)
eddiebergman (6)
mfeurer (6)
KEggensperger (5)
dengdifan (5)
FlorianPommerening (4)
TheEimer (4)
sarah-segel (4)
simonprovost (4)
thijssnelleman (4)
daphne12345 (3)

Pull Request Authors

benjamc (34)
helegraf (19)
dependabot[bot] (16)
LukasFehring (14)
daphne12345 (14)
eddiebergman (13)
timruhkopf (13)
sarah-segel (12)
dengdifan (9)
mlindauer (3)
mfeurer (3)
lhennig0103 (3)
bpkroth (3)
jeroenrook (3)
hadarshavit (3)

Top Labels

Issue Labels

bug (37) enhancement (35) documentation (33) feature (20) discussion (17) question (6) test (5) parallel (5) stale (3) dependency (2)

Pull Request Labels

dependency (13) enhancement (6) bug (4) documentation (4) dependencies (4) github_actions (4) feature (3) stale (3)

Packages

Total packages: 3
Total downloads:
- pypi 18,963 last-month
Total docker downloads: 2,393

Total dependent packages: 17
(may contain duplicates)
Total dependent repositories: 130
(may contain duplicates)
Total versions: 98
Total maintainers: 11

pypi.org: smac

SMAC3, a Python implementation of 'Sequential Model-based Algorithm Configuration'.

Homepage: https://www.automl.org/
Documentation: https://automl.github.io/SMAC3/main
License: BSD 3-Clause License
Latest release: 2.3.1
published 9 months ago

Versions: 51
Dependent Packages: 16
Dependent Repositories: 129
Downloads: 18,963 Last month
Docker Downloads: 2,393

Rankings

Dependent packages count: 1.1%

Dependent repos count: 1.3%

Average: 1.5%

Docker downloads count: 1.7%

Downloads: 1.8%

Maintainers (11)

eddiebergman mfeurer ahetzer biedenka keggensperger mlindauer benjamc ddeng helegraf sarahkrebs automl_hannover

Last synced: 4 months ago

proxy.golang.org: github.com/automl/SMAC3

Documentation: https://pkg.go.dev/github.com/automl/SMAC3#section-documentation
License: other
Latest release: v2.3.0+incompatible
published 11 months ago

Versions: 37
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 7.0%

Average: 8.2%

Dependent repos count: 9.3%

Last synced: 4 months ago

conda-forge.org: smac

SMAC is a tool for algorithm configuration to optimize the parameters of arbitrary algorithms across a set of instances. This also includes hyperparameter optimization of ML algorithms. The main core consists of Bayesian Optimization in combination with an aggressive racing mechanism to efficiently decide which of two configurations performs better.

Homepage: https://www.automl.org/automated-algorithm-design/algorithm-configuration/smac/
License: BSD-3-Clause
Latest release: 1.4.0
published over 3 years ago

Versions: 10
Dependent Packages: 1
Dependent Repositories: 1

Rankings

Forks count: 12.9%

Stargazers count: 14.0%

Average: 20.1%

Dependent repos count: 24.4%

Dependent packages count: 29.0%

Last synced: 4 months ago

Dependencies

.github/workflows/citation.yml actions

actions/checkout v3 composite
citation-file-format/cffconvert-github-action 2.0.0 composite

.github/workflows/dist.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/docs.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/examples.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/pre-commit.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/pytest.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite
conda-incubator/setup-miniconda v2 composite

.github/workflows/stale_reminder.yml actions

dawidd6/action-send-mail v3 composite
lee-dohm/select-matching-issues v1 composite

setup.py pypi

ConfigSpace >=0.5.0
dask *
distributed *
emcee >=3.0.0
joblib *
numpy >=1.7.1
psutil *
pynisher <1.0.0
pyrfr >=0.8.3
regex *
scikit-learn >=0.22.0
scipy >=1.7.0