PyXAB - Python X-Armed Bandit

PyXAB is a Python open-source library for X-armed bandit algorithms, a prestigious set of optimizers for online black-box optimization and hyperparameter optimization.

PyXAB contains the implementations of 10+ optimization algorithms, including the classic ones such as Zooming, StoSOO, and HCT, and the most recent works such as GPO, StroquOOL and VHCT. PyXAB also provides the most commonly-used synthetic objectives to evaluate the performance of different algorithms and the implementations for different hierarchical partitions

PyXAB is featured for:

• User-friendly APIs, clear documentation, and detailed examples
• Comprehensive library of optimization algorithms, partitions and synthetic objectives
• High standard code quality and high testing coverage
• Low dependency for flexible combination with other packages such as PyTorch, Scikit-Learn

Reminder: The algorithms are maximization algorithms!

Quick Links

• Quick Example
• Documentations
• Installation
• Features
  • X-armed bandit algorithms
  • Hierarchical partition
  • Synthetic objectives
• Contributing
• Citations

Quick Example

PyXAB follows a natural and straightforward API design completely aligned with the online blackbox optimization paradigm. The following is a simple 6-line usage example.

First, we define the parameter domain and the algorithm to run. At every round t, call algo.pull(t) to get a point and call algo.receive_reward(t, reward) to give the algorithm the objective evaluation (reward)

```python3 from PyXAB.algos.HOO import T_HOO

domain = [[0, 1]] # Parameter is 1-D and between 0 and 1 algo = THOO(rounds=1000, domain=domain) for t in range(1000): point = algo.pull(t) reward = 1 #TODO: User-defined objective returns the reward algo.receivereward(t, reward) ```

More detailed examples can be found here

Documentations

• The most up-to-date documentations

• The roadmap for our project

• Our manuscript for the library

Installation

To install via pip, run the following lines of code bash pip install PyXAB # normal install pip install --upgrade PyXAB # or update if needed

To install via git, run the following lines of code bash git clone https://github.com/WilliamLwj/PyXAB.git cd PyXAB pip install .

Features:

X-armed bandit algorithms

• Algorithm starred are meta-algorithms (wrappers)

| Algorithm | Research Paper | Year | |-------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------| | Zooming | Multi-Armed Bandits in Metric Spaces | 2008 | | T-HOO | X-Armed Bandit | 2011 | | DOO | Optimistic Optimization of a Deterministic Function without the Knowledge of its Smoothness | 2011 | | SOO | Optimistic Optimization of a Deterministic Function without the Knowledge of its Smoothness | 2011 | | StoSOO | Stochastic Simultaneous Optimistic Optimization | 2013 | | HCT | Online Stochastic Optimization Under Correlated Bandit Feedback | 2014 | | POO* | Black-box optimization of noisy functions with unknown smoothness | 2015 | | GPO* | General Parallel Optimization Without A Metric | 2019 | | PCT | General Parallel Optimization Without A Metric | 2019 | | SequOOL | A Simple Parameter-free And Adaptive Approach to Optimization Under A Minimal Local Smoothness Assumption | 2019 | | StroquOOL | A Simple Parameter-free And Adaptive Approach to Optimization Under A Minimal Local Smoothness Assumption | 2019 | | VROOM | Derivative-Free & Order-Robust Optimisation | 2020 | | VHCT | Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization | 2023 | | VPCT | N.A. (GPO + VHCT) | N.A. |

Hierarchical partition

| Partition | Description | |-----------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------| | BinaryPartition | Equal-size binary partition of the parameter space, the split dimension is chosen uniform randomly | | RandomBinaryPartition | Random-size binary partition of the parameter space, the split dimension is chosen uniform randomly | | DimensionBinaryPartition | Equal-size partition of the space with a binary split on each dimension, the number of children of one node is 2^d | | KaryPartition | Equal-size K-ary partition of the parameter space, the split dimension is chosen uniform randomly | | RandomKaryPartition | Random-size K-ary partition of the parameter space, the split dimension is chosen uniform randomly |

Synthetic objectives

• Some of these objectives can be found on Wikipedia

| Objectives | Image | | --- |--- | | Garland | | | DoubleSine | | | DifficultFunc | | | Ackley | | | Himmelblau | | | Rastrigin | |

Contributing

We appreciate all forms of help and contributions, including but not limited to

• Star and watch our project
• Open an issue for any bugs you find or features you want to add to our library
• Fork our project and submit a pull request with your valuable codes

Please read the contributing instructions before submitting a pull request.

Citations

If you use our package in your research or projects, we kindly ask you to cite our work

text @article{Li2023PyXAB, doi = {10.21105/joss.06507}, url = {https://joss.theoj.org/papers/10.21105/joss.06507}, author = {Li, Wenjie and Li, Haoze and Song, Qifan and Honorio, Jean}, title = {PyXAB -- A Python Library for $\mathcal{X}$-Armed Bandit and Online Blackbox Optimization Algorithms}, journal={Journal of Open Source Software}, year = {2024}, issn={2475-9066}, }

We would also appreciate it if you could cite our related works.

text @article{li2023optimumstatistical, title={Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization}, author={Wenjie Li and Chi-Hua Wang and Guang Cheng and Qifan Song}, journal={Transactions on Machine Learning Research}, issn={2835-8856}, year={2023}, url={https://openreview.net/forum?id=ClIcmwdlxn}, note={} }

```text

@article{Li2022Federated, title={Federated $\chi$-armed Bandit}, volume={38}, url={https://ojs.aaai.org/index.php/AAAI/article/view/29267}, DOI={10.1609/aaai.v38i12.29267}, number={12}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Li, Wenjie and Song, Qifan and Honorio, Jean and Lin, Guang}, year={2024}, month={Mar.}, pages={13628-13636}

} ```

```text

@InProceedings{Li2024Personalized, title = {Personalized Federated $\chi$-armed Bandit}, author = {Li, Wenjie and Song, Qifan and Honorio, Jean}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {37--45}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/li24a/li24a.pdf}, url = {https://proceedings.mlr.press/v238/li24a.html}, } ```