optuna - v4.5.0

This is the release note of v4.5.0.

Highlights

GPSampler for constrained multi-objective optimization

GPSampler is now able to handle multiple objective and constraints simultaneously using the newly introduced constrained LogEHVI acquisition function.

The figures below show the difference between GPSampler (LogEHVI, unconstrained) vs GPSampler (constrained LogEHVI, new feature). The 3-dimensional version of the C2DTLZ2 benchmark problem we used is a problem where some areas of the Pareto front of the original DTLZ2 problem are made infeasible by constraints. Therefore, even if constraints are not taken into account, it is possible to obtain the Pareto front. Experimental results show that both LogEHVI and constrained LogEHVI can approximate the Pareto front, but the latter has significantly fewer infeasible solutions, demonstrating its efficiency.

| Optuna v4.4 (LogEHVI) | Optuna v4.5 (Constrained LogEHVI) | |:--:|:--:| |||

Significant speedup of TPESampler

TPESampler is significantly (about 5x as listed in the table below) faster! It enables a larger number of trials in each study. The speedup was achieved through a series of enhancements in constant factors.

The following table shows the speed comparison of TPESampler between v4.4.0 and v4.5.0. The experiments were conducted using multivariate=True on a search space with 3 continuous parameters and 3 numerical discrete parameters. Each row shows the runtime for each number of objectives and each column shows each number of trials to be evaluated. Each runtime is shown along with the standard error over 3 random seeds. The numbers in parentheses represent the speedup factor in comparison to v4.4.0. For example, (5.1x) means the runtime of v4.5.0 is 5.1 times faster than that of v4.4.0.

|n_objectives/n_trials|500|1000|1500|2000| |:--:|:--:|:--:|:--:|:--:| |1|1.4 $\pm$ 0.03 (5.1x)|3.9 $\pm$ 0.07 (5.3x)|7.3 $\pm$ 0.09 (5.4x)|11.9 $\pm$ 0.10 (5.4x)| |2|1.8 $\pm$ 0.01 (4.7x)|4.7 $\pm$ 0.02 (4.8x)|8.7 $\pm$ 0.03 (4.8x)|13.9 $\pm$ 0.04 (4.9x)| |3|2.0 $\pm$ 0.01 (4.2x)|5.4 $\pm$ 0.03 (4.4x)|10.0 $\pm$ 0.03 (4.6x)|15.9 $\pm$ 0.03 (4.7x)| |4|4.2 $\pm$ 0.11 (3.2x)|12.1 $\pm$ 0.14 (3.9x)|20.9 $\pm$ 0.23 (4.2x)|31.3 $\pm$ 0.05 (4.4x)| |5|12.1 $\pm$ 0.59 (4.7x)|30.8 $\pm$ 0.16 (5.8x)|50.7 $\pm$ 0.46 (6.5x)|72.8 $\pm$ 1.13 (7.1x)|

Significant speedup of plot_hypervolume_history

plot_hypervolume_history is essential to assess the performance of multi-objective optimization, but it was unbearably slow when a large number of trials are evaluated on a many-objective (The number of objectives > 3) problem. v4.5.0 addressed this issue by incrementally updating the hypervolume instead of calculating each hypervolume from scratch.

The following figure shows the elapsed times of hypervolume history plot in Optuna v4.4.0 and v4.5.0 using a four-objective problem. The x-axis represents the number of trials and the y-axis represents the elapsed times for each setup. The blue and red lines are the results of v4.4.0 and v4.5.0, respectively.

CmaEsSampler now supports 1D search space

Up until Optuna v4.4, CmaEsSampler could not handle one-dimensional space and fell back to random search. Optuna v4.5 now allows the CMA-ES algorithm to be used for one-dimensional space.

The optunahub library is available on conda-forge

Now, you can install the optunahub library via conda-forge as follows.

sh conda install conda-forge::optunahub

New Features

• Add ConstrainedLogEHVI (#6198)
• Add support for constrained multi-objective optimization in GPSampler (#6224)
• Support 1D Search Spaces in CmaEsSampler (#6228)

Enhancements

• Move optuna._lightgbm_tuner module (https://github.com/optuna/optuna-integration/pull/233, thanks @milkcoffeen!)
• Fix numerical issue warning on qehvi_candidates_func (https://github.com/optuna/optuna-integration/pull/242, thanks @LukeGT!)
• Calculate hypervolume in HSSP using sum of contributions (#6130)
• Use hypervolume difference as upperbound of contribs in HSSP (#6131)
• Refactor tell_with_warning to avoid unnecessary get_trial call (#6133)
• Print fully qualified name of experimental function by default (#6162, thanks @ktns!)
• Include scipy-stubs in the type-check dependencies (#6174, thanks @jorenham!)
• Warn when GPSampler falls back to RandomSampler (#6179, thanks @sisird864!)
• Handle slowdown of GPSampler due to L-BFGS in SciPy v1.15 (#6191)
• Use the Newton method instead of bisect in ndtri_exp (#6194)
• Speed up erf for TPESampler (#6200)
• Avoid duplications in _log_gauss_mass evaluations (#6202)
• Remove unnecessary NumPy usage (#6215)
• Use subset comparator to judge if trials are included in search space (#6218)
• Speed up log pdf in _BatchedTruncNormDistributions by vectorization (#6220)
• Speed up WFG by skipping is_pareto_front and using simple Python loops (#6223)
• Vectorize ndtri_exp (#6229)
• Speed up plot_hypervolume_history (#6232)
• Speed up HSSP 4D+ by using a decremental approach (#6234)
• Use lru_cache to skip HSSP (#6240, thanks @fusawa-yugo!)
• Add hypervolume computation for a zero size array (#6245)

Bug Fixes

• Fix: Resolve PG17 incompatibility for ENUMS in CASE statements (#6099, thanks @vcovo!)
• Fix ill-combination of journal and gRPC (#6175)
• Fix a bug in constrained GPSampler (#6181)
• Fix TPESampler with multivariate and constant_liar (#6189)

Installation

• Remove version constraint for torch with Python 3.13 (#6233)

Documentation

• Add missing spaces in error message about inconsistent intermediate values (https://github.com/optuna/optuna-integration/pull/239, thanks @Greesb!)
• Improve parallelization document (#6123)
• Add FAQ for case sensitivity problem with MySQL (#6127, thanks @fusawa-yugo!)
• Add an FAQ entry about specifying optimization parameters (#6157)
• Update link to survey (#6169)
• Add introduction of OptunaHub in docs (#6171, thanks @fusawa-yugo!)
• Add GPSampler as a sampler that supports constraints (#6176, thanks @1kastner!)
• Remove optuna-fast-fanova references from documentation (#6178)
• Fix stale docs in GP-related modules (#6184)
• Embed link to OptunaHub in documentation (#6192, thanks @fusawa-yugo!)
• Update README.md (#6222, thanks @muhammadibrahim313!)
• Add a note about unrelated changes in PR (#6226)
• Document the default evaluator in Optuna Dashboard (#6238)

Examples

• Add Spark example using ask-and-tell interface (https://github.com/optuna/optuna-examples/pull/328, thanks @dhyeyinf!)

Tests

• Fix test_log_completed_trial_skip_storage_access (#6208)

Code Fixes

• Clean up GP-related docs (#6125)
• Refactor return style #6136 (#6151, thanks @unKnownNG!)
• Refactor KernelParamsTensor towards cleaner GP-related modules (#6152)
• Rename KernelParamsTensor to GPRegressor (#6153)
• Refactor returns in v3.0.0.d.py (#6154, thanks @dross20!)
• Refactor acquisition function minimally (#6166)
• Implement Type-Checking for optuna/_imports.py (#6167, thanks @AdrianStrymer!)
• Fix type checking in optuna.artifacts._download.py (#6177, thanks @dross20!)
• Integrate is_categorical to search space (#6182)
• Fix type checking in optuna.artifacts._list_artifact_meta.py (#6187, thanks @dross20!)
• Introduce the independent sampling warning template (#6188)
• Use warning template for independent sampling in GPSampler (#6195)
• Refactor SearchSpace in GP (#6197)
• Refactor _truncnorm (#6201)
• Use TYPE_CHECKING in optuna/_gp/acqf.py to avoid circular imports (#6204, thanks @CarvedCoder!)
• Use TYPE_CHECKING in optuna/_gp/optim_mixed.py to avoid circular imports (#6205, thanks @Subodh-12!)
• Flip the sign of constraints in GPSampler (#6213)
• Implement NSGA-III using BaseGASampler (#6219)
• Replace torch.newaxis with None for old PyTorch (#6237)

Continuous Integration

• Fix CI (https://github.com/optuna/optuna-integration/pull/238)
• Add sklearn version constraint (https://github.com/optuna/optuna-integration/pull/243)
• Fix README for blackdoc==0.3.10 (#6150)
• Fix CI (#6161)
• Add pytest-xdist to speed up the CI (#6170)
• Delete test_get_timeline_plot_with_killed_running_trials (#6210)
• Fix fragile test test_experimental (#6211)
• Mark fragile test as xfail (#6217)
• Add unit tests for constrained multi-objective GPSampler (#6235)
• Fix CI (#6246)

Other

• Update the example list in the README (https://github.com/optuna/optuna-integration/pull/234, thanks @ParagEkbote!)
• Bump up version number to 4.5.0.dev (https://github.com/optuna/optuna-integration/pull/237)
• Bump up the version number to 4.5.0 (https://github.com/optuna/optuna-integration/pull/244)
• Bump up version number to v4.5.0.dev (#6149)
• Update News section in README (#6159)
• Bump up version to v4.5.0 (#6251)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@1kastner, @AdrianStrymer, @CarvedCoder, @Greesb, @HideakiImamura, @LukeGT, @ParagEkbote, @Subodh-12, @c-bata, @contramundum53, @dhyeyinf, @dross20, @fusawa-yugo, @gen740, @hvy, @jorenham, @kAIto47802, @ktns, @milkcoffeen, @muhammadibrahim313, @nabenabe0928, @not522, @nzw0301, @sawa3030, @sisird864, @toshihikoyanase, @unKnownNG, @vcovo, @y0z

- Python
Published by y0z 10 months ago

optuna - v4.4.0

This is the release note of v4.4.0.

Highlights

In addition to new features, bug fixes, and improvements in documentation and testing, version 4.4 introduces a new tool called the Optuna MCP Server.

Optuna MCP Server

The Optuna MCP server can be accessed by any MCP client via uv — for instance, with Claude Desktop, simply add the following configuration to your MCP server settings file. Of course, other LLM clients like VSCode or Cline can also be used similarly. You can also access it via Docker. If you want to persist the results, you can use the — storage option. For details, please refer to the repository.

{ "mcpServers": { … (Other MCP Servers' settings) "Optuna": { "command": "uvx", "args": [ "optuna-mcp" ] } } }

Gaussian Process-Based Multi-objective Optimization

Optuna’s GPSampler, introduced in version 3.6, offers superior speed and performance compared to existing Bayesian optimization frameworks, particularly when handling objective functions with discrete variables. In Optuna v4.4, we have extended this GPSampler to support multi-objective optimization problems. The applications of multi-objective optimization are broad, and the new multi-objective capabilities introduced in this GPSampler are expected to find applications in fields such as material design, experimental design problems, and high-cost hyperparameter optimization.

GPSampler can be easily integrated into your program and performs well against the existing BoTorchSampler. We encourage you to try it out with your multi-objective optimization problems.

python sampler = optuna.samplers.GPSampler() study = optuna.create_study(directions=["minimize", "minimize"], sampler=sampler)

New Features in OptunaHub

During the development period of Optuna v4.4, several new features were also introduced to OptunaHub, the feature-sharing platform for Optuna:

• A sampler utilizing Google Vizier is now newly available.
• The CMA-ES-based sampler with restart strategy, which was previously part of the Optuna core, has been migrated to OptunaHub, making it simpler and more user-friendly.
• A benchmark problem solving aircraft design as a black-box optimization task has been added, further enhancing the convenience of algorithm development using OptunaHub.
• A visualization feature has also been added, allowing users to see how the acquisition function of the default TPESampler evolves as trials progress.
• A novel mutation operation has been added to the MOEA/D evolutionary computation algorithm for multi-objective optimization.

| Vizier sampler performance | | --- | | |

| TPE acquisition visualizer | | --- | | |

Breaking Changes

• Update consider_prior Behavior and Remove Support for False (#6007)
• Remove restart_strategy and inc_popsize to simplify CmaEsSampler (#6025)
• Make all arguments of TPESampler keyword-only (#6041)

New Features

• Add a module to preprocess solutions for hypervolume improvement calculation (#6039)
• Add AcquisitionFuncParams for LogEHVI (#6052)
• Support Multi-Objective Optimization GPSampler (#6069)
• Add n_recent_trials to plot_timeline (#6110, thanks @msdsm!)

Enhancements

• Adapt TYPE_CHECKING of samplers/_gp/sampler.py (#6059)
• Avoid deepcopy in _tell_with_warning (#6079)
• Add _compute_3d for hypervolume computation (#6112, thanks @shmurai!)
• Improve performance of plot_hypervolume_history (#6115, thanks @shmurai!)
• add deprecated/removed version specification to calls of convert_positional_args (#6117, thanks @shmurai!)
• Optimize Study.best_trial performance by avoiding unnecessary deep copy (#6119, thanks @msdsm!)
• Refactor and speed up HV3D (#6124)
• Add assume_pareto for hv calculation in _calculate_weights_below_for_multi_objective (#6129)

Bug Fixes

• Update vsbx (#6033, thanks @hrntsm!)
• Fix request.values in OptunaStorageProxyService (#6044, thanks @hitsgub!)
• Fix a bug in distributed optimization using NSGA-II/III (#6066, thanks @leevers!)
• Fix: fetch all trials in BruteForceSampler for HyperbandPruner (#6107)

Documentation

• Add Pycma Example (https://github.com/optuna/optuna-integration/pull/226, thanks @ParagEkbote!)
• Add SHAP Example (https://github.com/optuna/optuna-integration/pull/227, thanks @ParagEkbote!)
• Document Behavior of optuna.pruners.MedianPruner and optuna.pruners.PatientPruner (#6055, thanks @ParagEkbote!)
• Change the link of tutorial docs of optunahub (#6063, thanks @fusawa-yugo!)
• Update the documentation string of GPSampler (#6081)
• Add a warning about the combination of gRPC Proxy and Journal Storage (#6097)
• Cosmetic fix to the terminator documents (#6100)
• Note in docstring that heartbeat mechanism is experimental (#6111, thanks @lan496!)
• Update docstrings of _get_best_trial to follow coding conventions (#6122)

Examples

• Add Example for Comet (https://github.com/optuna/optuna-examples/pull/305, thanks @ParagEkbote!)
• Adding an OpenML example (https://github.com/optuna/optuna-examples/pull/310, thanks @SubhadityaMukherjee!)
• Add workflow dispatch (https://github.com/optuna/optuna-examples/pull/311)
• Update PyTorch Checkpoint Example using tempfile (https://github.com/optuna/optuna-examples/pull/313, thanks @ParagEkbote!)
• [hotfix] Fix Dask-ML example by adding the version constraint on numpy (https://github.com/optuna/optuna-examples/pull/315)
• Setup Pre-Commit (https://github.com/optuna/optuna-examples/pull/316, thanks @ParagEkbote!)
• Remove Python 3.9 from haiku CI (https://github.com/optuna/optuna-examples/pull/318)
• Add a transformers example (https://github.com/optuna/optuna-examples/pull/322, thanks @ParagEkbote!)
• Add transformer item to RAEDME.md (https://github.com/optuna/optuna-examples/pull/323)
• Remove version constraints of tensorflow and numpy (https://github.com/optuna/optuna-examples/pull/324)
• Update pre-commit hooks (https://github.com/optuna/optuna-examples/pull/326, thanks @lan496!)
• Add preferential optimization picture (https://github.com/optuna/optuna-examples/pull/327, thanks @milkcoffeen!)

Tests

• Add float precision tests for storages (#6040)
• Refactor test_base_gasampler.py (#6104)
• chore: run tests for importance only with in-memory (#6109)
• Improve test cases for n_recent_trials of plot_timeline (follow-up #6110) (#6116)
• Performance optimization for test_study.py by removing redundancy (#6120)

Code Fixes

• Optional mypy check (#6028)
• Update Type-Checking for optuna/_experimental.py (#6045, thanks @ParagEkbote!)
• Update Type-Checking for optuna/importance/_base.py (#6046, thanks @ParagEkbote!)
• Update Type-Checking for optuna/_convert_positional_args.py (#6050, thanks @ParagEkbote!)
• Update Type-Checking for optuna/_deprecated.py (#6051, thanks @ParagEkbote!)
• Update Type-Checking for optuna/_gp/gp.py (#6053, thanks @ParagEkbote!)
• Add validate eta in sbx (#6056, thanks @hrntsm!)
• Remove CmaEsAttrKeys and _attr_keys for Simplification (#6068)
• Replace np.isnan with math.isnan (#6080)
• Refactor warning handling of _tell_with_warning (#6082)
• Implement Type-Checking for optuna/distributions.py (#6086, thanks @AdrianStrymer!)
• Update TYPECHECKING for `optuna/gp/gp.py` (#6090, thanks @Samarthi!)
• Support mypy 1.16.0 (#6102)
• Emit ExperimentalWarning if heartbeat is enabled (#6106, thanks @lan496!)
• Simplify tuple return in optuna/visualization/_terminator_improvement.py (#6139, thanks @Prashantdhaka23!)
• Refactor return standardization in optim_mixed.py (#6140, thanks @Ajay-Satish-01!)
• Simplify tuple return in test_trial.py (#6141, thanks @saishreyakumar!)
• Refactor return statement style in optuna/storages/_rdb/models.py for consistency among the codebase (#6143, thanks @Shubham05122002!)

Continuous Integration

• Add type ignore in wandb (https://github.com/optuna/optuna-integration/pull/228)
• Fix to prevent daily checks-optional CI on the fork repositories (#6103)
• Fix CI (#6137)
• [hotfix] Add version constraint on blackdoc (#6145)

Other

• Bump up version number to v4.4.0.dev (https://github.com/optuna/optuna-integration/pull/220)
• Add pre commit config (https://github.com/optuna/optuna-integration/pull/231, thanks @milkcoffeen!)
• Bump up version number to 4.4.0 (https://github.com/optuna/optuna-integration/pull/236)
• Bump up version to 4.4.0dev (#6038)
• Update the news section in README.md (#6049)
• Fix README for v5 roadmap (#6094)
• Update pre-commit hooks (#6108, thanks @lan496!)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@AdrianStrymer, @Ajay-Satish-01, @Alnusjaponica, @Copilot, @HideakiImamura, @ParagEkbote, @Prashantdhaka23, @Samarthi, @Shubham05122002, @SubhadityaMukherjee, @c-bata, @contramundum53, @copilot-pull-request-reviewer[bot], @fusawa-yugo, @gen740, @himkt, @hitsgub, @hrntsm, @kAIto47802, @lan496, @leevers, @milkcoffeen, @msdsm, @nabenabe0928, @not522, @nzw0301, @saishreyakumar, @sawa3030, @shmurai, @toshihikoyanase, @y0z

- Python
Published by HideakiImamura about 1 year ago

optuna - v4.3.0

This is the release note of v4.3.0.

Highlights

This has various bug fixes and improvements to the documentation and more.

Breaking Changes

• [fix] lgbm 4.6.0 compatibility (https://github.com/optuna/optuna-integration/pull/207, thanks @ffineis!)

Enhancements

• Accept custom objective in LightGBMTuner (https://github.com/optuna/optuna-integration/pull/203, thanks @sawa3030!)
• Improve time complexity of IntersectionSearchSpace (#5982, thanks @GittyHarsha!)
• Add _prev_waiting_trial_number in InMemoryStorage to improve the efficiency of _pop_waiting_trial_id (#5993, thanks @sawa3030!)
• Add arguments of versions to convert_positional_args (#6009, thanks @fusawa-yugo!)
• Add wait_server_ready method in GrpcStorageProxy (#6010, thanks @hitsgub!)
• Remove warning messages for Matplotlib-based plot_contour and plot_rank (#6011)
• Fix type checking in optuna._callbacks.py (#6030)
• Enhance SBXCrossover (#6008, thanks @hrntsm!)

Bug Fixes

• Convert storage into InMemoryStorage before copying to the local (https://github.com/optuna/optuna-integration/pull/213)
• Fix contour plot of matplotlib (#5892, thanks @fusawa-yugo!)
• Fix threading lock logic (#5922)
• Use _LazyImport for grpcio package (#5954)
• Prevent Lock Blocking by Adding Timeout to JournalStorage (#5971, thanks @sawa3030!)
• Fix a minor bug in GPSampler for objective that returns inf (#5995)
• Fix a bug that a gRPC server doesn't work with JournalStorage (#6004, thanks @fusawa-yugo!)
• Fix _pop_waiting_trial_id for finished trial (#6012)
• Resolve the issue where BruteForceSampler fails to suggest all combinations (#5893)

Documentation

• Follow recent changes in optuna/optuna's document sphinx config (https://github.com/optuna/optuna-integration/pull/197)
• Fix links to external modules (https://github.com/optuna/optuna-integration/pull/198)
• Update CONTRIBUTING.md (https://github.com/optuna/optuna-integration/pull/200, thanks @sawa3030!)
• Update comment in .readthedocs.yml (#5976)
• Add comments on the reproducibility of HyperBandPruner (#6018)

Examples

• [hotfix] Add the version constraint on dask (https://github.com/optuna/optuna-examples/pull/296)
• [hotfix] Add the version constraint on dask for dask-ml (https://github.com/optuna/optuna-examples/pull/297)
• Extends execution span of hiplot and sklearn (https://github.com/optuna/optuna-examples/pull/298, thanks @fusawa-yugo!)
• Apply black to fix CI (https://github.com/optuna/optuna-examples/pull/300)
• Bump up to 3.12 for CI (https://github.com/optuna/optuna-examples/pull/301)
• [hotfix] Add the version constraint on lightgbm (https://github.com/optuna/optuna-examples/pull/302)
• Fix Skorch Example (https://github.com/optuna/optuna-examples/pull/303, thanks @ParagEkbote!)
• Add version constraint for tensorflow-related CI (https://github.com/optuna/optuna-examples/pull/304)
• Temporarily skip Python 3.9 in fastai example (https://github.com/optuna/optuna-examples/pull/308)
• Run the skorch example in the CI (https://github.com/optuna/optuna-examples/pull/309)
• Fix fastai Example (https://github.com/optuna/optuna-examples/pull/312)

Tests

• Use JournalStorage in test_cli.py (#5990, thanks @sawa3030!)

Code Fixes

• Add BaseGASampler (#5864)
• Fix comments in pyproject.toml (#5972)
• Remove FirstTrialOnlyRandomSampler (#5973, thanks @mehakmander11!)
• Remove _check_and_set_param_distribution (#5975, thanks @siddydutta!)
• Remove testing/distributions.py (#5977, thanks @mehakmander11!)
• Remove _StudyInfo's param_distribution in _cached_storage.py (#5978, thanks @tarunprabhu11!)
• Introduce UpdateFinishedTrialError to raise an error when attempting to modify a finished trial (#6001, thanks @sawa3030!)
• Deprecate consider_prior in TPESampler (#6005, thanks @sawa3030!)
• Improve Code Readability by Following PEP8 Standards (#6006, thanks @sawa3030!)
• Made error message for create_study's direction easier to understand optuna.study (#6021, thanks @sinano1107!)

Continuous Integration

• Hotfix ci (https://github.com/optuna/optuna-integration/pull/199)
• Add flake8 in CI (https://github.com/optuna/optuna-integration/pull/201, thanks @sawa3030!)
• Remove test cases that uses UnsupportedDistribution (https://github.com/optuna/optuna-integration/pull/208)
• Fix a mypy error when using numpy>=2.2.4 (https://github.com/optuna/optuna-integration/pull/212)
• Fix a bug of lightgbm tuner for Python 3.8 users (https://github.com/optuna/optuna-integration/pull/214)
• Add a version constraint on xgboost (https://github.com/optuna/optuna-integration/pull/217)
• Run (https://github.com/optuna/optuna-integration/pull/218)
• Ensure gRPC server readiness before proceeding to prevent test failures (#5938, thanks @sawa3030!)
• Apply black to fix CI (#5952)
• Add workflow_dispatch trigger to all the CI (#6019)
• Fix CI (#6026)

Other

• Bump up version number to 4.3.0.dev (https://github.com/optuna/optuna-integration/pull/192)
• Bump the version up to v4.2.1 (https://github.com/optuna/optuna-integration/pull/195)
• Set repository url (https://github.com/optuna/optuna-integration/pull/196, thanks @ktns!)
• Bump up version number to v4.3.0 (https://github.com/optuna/optuna-integration/pull/221)
• Bump the version up to v4.3.0.dev (#5927)
• Add the article to the news section (#5928)
• Update news section for 4.2.0 release (#5934)
• Update News (#5936)
• Update README with the new blog entry (#5980)
• Add GPSampler blog to the announcement (#6014)
• Add grpc blog to README (#6020)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @GittyHarsha, @HideakiImamura, @ParagEkbote, @c-bata, @contramundum53, @ffineis, @fusawa-yugo, @gen740, @hitsgub, @hrntsm, @kAIto47802, @ktns, @mehakmander11, @nabenabe0928, @not522, @nzw0301, @porink0424, @sawa3030, @siddydutta, @sinano1107, @tarunprabhu11, @toshihikoyanase, @y0z

- Python
Published by y0z about 1 year ago

optuna - v4.2.1

This is the release note of v4.2.1. This release includes a bug fix addressing an issue where Optuna was unable to import if an older version of the grpcio package was installed.

Bug

• [backport] Use _LazyImport for grpcio package (#5965)

Other

• Bump up version number to v4.2.1 (#5964)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions. @c-bata @HideakiImamura @nabenabe0928

- Python
Published by c-bata over 1 year ago

optuna - v3.6.2

This is the release note of v3.6.2.

Bug Fixes

• [Backport] Fix the default sampler of load_study function.

Other

• Bump up version number to v3.6.2

- Python
Published by c-bata over 1 year ago

optuna - v3.5.1

This is the release note of v3.5.1.

Bug Fixes

• [Backport] Fix the default sampler of load_study function.

Other

• Bump up version number to v3.5.1.

- Python
Published by c-bata over 1 year ago

optuna - v3.4.1

This is the release note of v3.4.1.

Bug Fixes

• [Backport] Fix the default sampler of load_study function.

Other

• Bump up version number to v3.4.1.

- Python
Published by c-bata over 1 year ago

optuna - v4.2.0

This is the release note of v4.2.0. In conjunction with the Optuna release, OptunaHub 0.2.0 is released. Please refer to the release note of OptunaHub 0.2.0 for more details.

Highlights of this release include:

• 🚀gRPC Storage Proxy for Scalable Hyperparameter Optimization
• 🤖 SMAC3: Support for New State-of-the-art Optimization Algorithm by AutoML.org (@automl)
• 📁 OptunaHub Now Supports Benchmark Functions
• 🧑‍💻 Gaussian Process-Based Bayesian Optimization with Inequality Constraints
• 🧑‍💻 c-TPE: Support Constrained TPESampler

Highlights

gRPC Storage Proxy for Scalable Hyperparameter Optimization

The gRPC storage proxy is a feature designed to support large-scale distributed optimization. As shown in the diagram below, gRPC storage proxy sits between the optimization workers and the database server, proxying the calls of Optuna’s storage APIs.

In large-scale distributed optimization settings where hundreds to thousands of workers are operating, placing a gRPC storage proxy for every few tens can significantly reduce the load on the RDB server which would otherwise be a single point of failure. The gRPC storage proxy enables sharing the cache about Optuna studies and trials, which can further mitigate load. Please refer to the official documentation for further details on how to utilize gRPC storage proxy.

SMAC3: Random Forest-Based Bayesian Optimization Developed by AutoML.org

SMAC3 is a hyperparameter optimization framework developed by AutoML.org, one of the most influential AutoML research groups. The Optuna-compatible SMAC3 sampler is now available thanks to the contribution to OptunaHub by Difan Deng (@dengdifan), one of the core members of AutoML.org. We can now use the method widely used in AutoML research and real-world applications from Optuna.

```python

pip install optunahub smac

import optuna import optunahub from optuna.distributions import FloatDistribution

def objective(trial: optuna.Trial) -> float: x = trial.suggestfloat("x", -10, 10) y = trial.suggestfloat("y", -10, 10) return x2 + y2

smacmod = optunahub.loadmodule("samplers/smacsampler") ntrials = 100 sampler = smacmod.SMACSampler( {"x": FloatDistribution(-10, 10), "y": FloatDistribution(-10, 10)}, ntrials=ntrials, ) study = optuna.createstudy(sampler=sampler) study.optimize(objective, ntrials=ntrials) ```

Please refer to https://hub.optuna.org/samplers/smac_sampler/ for more details.

OptunaHub Now Supports Benchmark Functions

Benchmarking the performance of optimization algorithms is an essential process indispensable to the research and development of algorithms. The newly added OptunaHub Benchmarks in the latest version v0.2.0 of optunahub is a new feature for Optuna users to conduct benchmarks conveniently.

```python

pip install optunahub>=4.2.0 scipy torch

import optuna import optunahub

bbobmod = optunahub.loadmodule("benchmarks/bbob") smacmod = optunahub.loadmodule("samplers/smacsampler") sphere2d = bbobmod.Problem(function_id=1, dimension=2)

ntrials = 100 studies = [] for studyname, sampler in [ ("random", optuna.samplers.RandomSampler(seed=1)), ("tpe", optuna.samplers.TPESampler(seed=1)), ("cmaes", optuna.samplers.CmaEsSampler(seed=1)), ("smac", smacmod.SMACSampler(sphere2d.searchspace, ntrials, seed=1)), ]: study = optuna.createstudy(directions=sphere2d.directions, sampler=sampler, studyname=studyname) study.optimize(sphere2d, ntrials=ntrials) studies.append(study)

optuna.visualization.plotoptimizationhistory(studies).show() ```

In the above sample code, we compare and display the performance of the four kinds of samplers using a two-dimensional Sphere function, which is part of a group of benchmark functions widely used in the black-box optimization research community known as Blackbox Optimization Benchmarking (BBOB).

Gaussian Process-Based Bayesian Optimization with Inequality Constraints

We worked on its extension and adapted GPSampler to constrained optimization in Optuna v4.2.0 since Gaussian process-based Bayesian optimization is a very popular method in various research fields such as aircraft engineering and materials science. We show the basic usage below.

```python

pip install optuna>=4.2.0 scipy torch

import numpy as np import optuna

def objective(trial: optuna.Trial) -> float: x = trial.suggestfloat("x", 0.0, 2 * np.pi) y = trial.suggestfloat("y", 0.0, 2 * np.pi) c = float(np.sin(x) * np.sin(y) + 0.95) trial.setuserattr("c", c) return float(np.sin(x) + y)

def constraints(trial: optuna.trial.FrozenTrial) -> tuple[float]: return (trial.user_attrs["c"],)

sampler = optuna.samplers.GPSampler(constraintsfunc=constraints) study = optuna.createstudy(sampler=sampler) study.optimize(objective, n_trials=50) ```

Please try out GPSampler for constrained optimization especially when only a small number of trials are available!

c-TPE: Support Constrained TPESampler

Although Optuna has supported constrained optimization for TPESampler, which is the default Optuna sampler, since v3.0.0, its algorithm design and performance comparison have not been verified academically. OptunaHub now supports c-TPE, which is another constrained optimization method for TPESampler. Importantly, the algorithm design and its performance comparison are publicly reviewed to be accepted to IJCAI, a top-tier AI international conference. Please refer to https://hub.optuna.org/samplers/ctpe/ for details.

New Features

• Enable GPSampler to support constraint functions (#5715)
• Update output format options in CLI to include the value choice (#5822, thanks @iamarunbrahma!)
• Add gRPC storage proxy server and client (#5852)

Enhancements

• Introduce client-side cache in GrpcStorageProxy (#5872)

Bug Fixes

• Fix CI (https://github.com/optuna/optuna-integration/pull/185)
• Fix ticks in Matplotlib contour plot (#5778, thanks @sulan!)
• Adding check in cli.py to handle an empty database (#5828, thanks @willdavidson05!)
• Avoid the input validation fail in Wilcoxon signed ranked test for Scipy 1.15 (#5912)
• Fix the default sampler of load_study function (#5924)

Documentation

• Update OptunaHub example in README (#5763)
• Update distributions.rst to list deprecated distribution classes (#5764)
• Remove deprecation comment for step in IntLogUniformDistribution (#5767)
• Update requirements for OptunaHub in README (#5768)
• Use inline code rather than italic for step (#5769)
• Add notes to ask_and_tell tutorial - batch optimization recommendations (#5817, thanks @SimonPop!)
• Fix the explanation of returned values of get_trial_params (#5820)
• Introduce sphinx-notfound-page for better 404 page (#5898)
• Follow-up #5872: Update the docstring of run_grpc_proxy_server (#5914)
• Modify doc-string of gRPC-related modules (#5916)

Examples

• Adapt docker recipes to Python 3.11 (https://github.com/optuna/optuna-examples/pull/292)
• Add version constraint for wandb (https://github.com/optuna/optuna-examples/pull/293)

Tests

• Add unit tests for retry_history method in RetryFailedTrialCallback (#5865, thanks @iamarunbrahma!)
• Add tests for value format in CLI (#5866)
• Import grpc lazy to fix the CI (#5878)

Code Fixes

• Fix annotations for distributions.py (#5755, thanks @KannanShilen!)
• Simplify type annotations for tests/visualization_tests/test_pareto_front.py (#5756, thanks @boringbyte!)
• Fix type annotations for test_hypervolume_history.py (#5760, thanks @boringbyte!)
• Simplify type annotations for tests/test_cli.py (#5765, thanks @boringbyte!)
• Simplify type annotations for tests/test_distributions.py (#5773, thanks @boringbyte!)
• Simplify type annotations for tests/samplers_tests/test_qmc.py (#5775, thanks @boringbyte!)
• Simplify type annotations for tests/sampler_tests/tpe_tests/test_sampler.py (#5779, thanks @boringbyte!)
• Simplify type annotations for tests/samplers_tests/tpe_tests/test_multi_objective_sampler.py (#5781, thanks @boringbyte!)
• Simplify type annotations for tests/samplers_tests/tpe_tests/test_parzen_estimator.py (#5782, thanks @boringbyte!)
• Simplify type annotations for tests/storage_tests/journal_tests/test_journal.py (#5783, thanks @boringbyte!)
• Simplify type annotations for tests/storage_tests/rdb_tests/create_db.py (#5784, thanks @boringbyte!)
• Simplify type annotations for tests/storage_tests/rdb_tests/ (#5785, thanks @boringbyte!)
• Simplify type annotations for tests/test_deprecated.py (#5786, thanks @boringbyte!)
• Simplify type annotations for tests/test_convert_positional_args.py (#5787, thanks @boringbyte!)
• Simplify type annotations for tests/importance_tests/test_init.py (#5790, thanks @boringbyte!)
• Simplify type annotations for tests/storages_tests/test_storages.py (#5791, thanks @boringbyte!)
• Simplify type annotations for tests/storages_tests/test_heartbeat.py (#5792, thanks @boringbyte!)
• Simplify type annotations optuna/cli.py (#5793, thanks @willdavidson05!)
• Simplify type annotations for tests/trial_tests/test_frozen.py (#5794, thanks @boringbyte!)
• Simplify type annotations for tests/trial_tests/test_trial.py (#5795, thanks @boringbyte!)
• Simplify type annotations for tests/trial_tests/test_trials.py (#5796, thanks @boringbyte!)
• Refactor some funcs in NSGA-III (#5798)
• Simplify type annotations optuna/_transform.py (#5799, thanks @JLX0!)
• Make the assertion messages in test_trial.py readable (#5800)
• Simplify type annotations for tests/pruners_tests/test_hyperband.py (#5801, thanks @boringbyte!)
• Simplify type annotations for tests/pruners_tests/test_median.py (#5802, thanks @boringbyte!)
• Simplify type annotations for tests/pruners_tests/test_patient.py (#5803, thanks @boringbyte!)
• Use study.ask() in tests instead of create_new_trial (#5807, thanks @unKnownNG!)
• Simplify type annotations for tests/pruners_tests/test_percentile.py (#5808, thanks @boringbyte!)
• Simplify type annotations for tests/pruners_tests/test_successive_halving.py (#5809, thanks @boringbyte!)
• Simplify type annotations for tests/study_tests/test_optimize.py (#5810, thanks @boringbyte!)
• Simplify type annotations for tests/hypervolume_tests/test_hssp.py (#5812, thanks @boringbyte!)
• Refactor the MOTPE split (#5813)
• Simplify type annotations for optuna/_callbacks.py (#5818, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/_random.py (#5819, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/_gp/sampler.py (#5823, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/nsgaii/_crossovers/_sbx.py (#5824, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/nsgaii/_crossovers/_spx.py (#5825, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/nsgaii/_crossovers/_undx.py (#5826, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/nsgaii/_crossovers/_vsbx.py (#5827, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/nsgaii/_crossover.py (#5831, thanks @boringbyte!)
• Simplify type annotations for optuna/samplers/testing/threading.py (#5832, thanks @boringbyte!)
• Simplify type annotations for optuna/pruners/_patient.py (#5833, thanks @boringbyte!)
• Use study.ask() in tests/pruners_tests/test_percentile.py (#5834, thanks @fusawa-yugo!)
• Simplify type annotations for optuna/search_space/group_decomposed.py (#5836, thanks @boringbyte!)
• Simplify type annotations for optuna/search_space/intersection.py (#5837, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/journal/_base.py (#5838, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/journal/_redis.py (#5840, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/journal/_storage.py (#5841, thanks @boringbyte!)
• Simplify type annotations for optuna/study/_dataframe.py (#5842, thanks @boringbyte!)
• Fix logger.warn() deprecation warning in GP module (#5843, thanks @iamarunbrahma!)
• Simplify type annotations for optuna/study/_optimize.py (#5844, thanks @boringbyte!)
• Simplify type annotations for optuna/study/_tell.py (#5845, thanks @boringbyte!)
• Fix mypy errors due to numpy 2.2.0 (#5848)
• Simplify type annotations for optuna/visualization/matplotlib/_contour.py (#5851, thanks @boringbyte!)
• Refactor the fix for MyPy errors due to NumPy v2.2.0 (#5853)
• Simplify type annotations for optuna/visualization/matplotlib/_parallel_coordinate.py (#5854, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/matplotlib/_param_importances.py (#5855, thanks @boringbyte!)
• Use study.ask() in tests/pruners_tests/test_successive_halving.py (#5856, thanks @willdavidson05!)
• Simplify type annotations for optuna/visualization/matplotlib/_pareto_front.py (#5857, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/matplotlib/_rank.py (#5858, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/matplotlib/_slice.py (#5859, thanks @boringbyte!)
• Refactor plot contour (#5867)
• Refactor MOTPE weighting (#5871)
• Simplify type annotations for optuna/visualization/_utils.py (#5876, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/_contour.py (#5877, thanks @boringbyte!)
• Simplify type annotations for optuna/_gp/gp.py (#5879, thanks @boringbyte!)
• Simplify type annotations for optuna/study/_tell.py (#5880, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/_heartbeat.py (#5882, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/_rdb/storage.py (#5883, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/_rdb/alembic/versions/v3.0.0.a.py (#5884, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/_rdb/alembic/versions/v3.0.0.c.py (#5885, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/_rdb/alembic/versions/v3.0.0.d.py (#5886, thanks @boringbyte!)
• Simplify type annotations for optuna/_deprecated.py (#5887, thanks @boringbyte!)
• Simplify type annotations for optuna/_experimental.py (#5888, thanks @boringbyte!)
• Simplify type annotations for optuna/_imports.py (#5889, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/_slice.py (#5894, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/_parallel_coordinate.py (#5895, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/_rank.py (#5896, thanks @boringbyte!)
• Simplify type annotations for optuna/visualization/_param_importances.py (#5897, thanks @boringbyte!)
• Simplify type annotations for optuna/_callbacks.py (#5899, thanks @boringbyte!)
• Simplify type annotations for optuna/storages/journal/_file.py (#5900, thanks @boringbyte!)
• Simplify type annotations for tests/storages_tests/test_with_server.py (#5901, thanks @boringbyte!)
• Simplify type annotations for tests/test_multi_objective.py (#5902, thanks @boringbyte!)
• Simplify type annotations for tests/artifacts_tests/test_gcs.py (#5903, thanks @boringbyte!)
• Simplify type annotations for tests/samplers_tests/test_grid.py (#5904, thanks @boringbyte!)
• Simplify type annotations for optuna/study/_dataframe.py (#5905, thanks @boringbyte!)
• Simplify type annotations for tests/visualization_tests/test_optimization_history.py (#5906, thanks @boringbyte!)
• Simplify type annotations for tests/visualization_tests/test_intermediate_plot.py (#5907, thanks @boringbyte!)
• Simplify type annotations for multiple test files in tests/visualization_tests/ (#5908, thanks @boringbyte!)
• Avoid the port number conflict in the copy study tests (#5913)
• Simplify annotations in tests/study_tests/test_study.py (#5923, thanks @sawa3030!)

Continuous Integration

• Fix a GitHub action workflow for publishing to PyPI (https://github.com/optuna/optuna-integration/pull/181)
• Fix CI by adding a version constraint (https://github.com/optuna/optuna-integration/pull/186)
• Rename test_cache_is_invalidated and remove assert study._thread_local.cached_all_trials is None (#5733)
• Use Python 3.12 for docs build CI (#5742)
• Fix a GitHub action workflow for publishing to PyPI (#5759)
• Install older kaleido to fix CI errors (#5771)
• Add Python 3.12 to the Docker image CI (#5789)
• Move mypy related entries in setup.cfg to pyproject.toml (#5861)

Other

• Bump up version number to v4.2.0.dev (https://github.com/optuna/optuna-integration/pull/178)
• Bump up version number to 4.2.0 (https://github.com/optuna/optuna-integration/pull/191)
• Add CITATION.cff (#5746)
• Update news to add the autosampler article (#5748)
• Bump the version up to v4.2.0.dev (#5752)
• Update the news section for Optuna 4.1 release (#5758)
• Update pre-commit configuration file (#5847)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @JLX0, @KannanShilen, @SimonPop, @boringbyte, @c-bata, @fusawa-yugo, @gen740, @himkt, @iamarunbrahma, @kAIto47802, @ktns, @mist714, @nabenabe0928, @not522, @nzw0301, @porink0424, @sawa3030, @sulan, @unKnownNG, @willdavidson05, @y0z

- Python
Published by c-bata over 1 year ago

optuna - v4.1.0

This is the release note of v4.1.0. Highlights of this release include: - 🤖 AutoSampler: Automatic Selection of Optimization Algorithms - 🚀 More scalable RDB Storage Backend - 🧑‍💻 Five New Algorithms in OptunaHub (MO-CMA-ES, MOEA/D, etc.) - 🐍 Support Python 3.13

The updated list of tested and supported Python releases is as follows: - Optuna 4.1: supported by Python 3.8 - 3.13 - Optuna Integration 4.1: supported by Python 3.8 - 3.12 - Optuna Dashboard 0.17.0: supported by Python 3.8 - 3.13

Highlights

AutoSampler: Automatic Selection of Optimization Algorithms

AutoSampler automatically selects a sampler from those implemented in Optuna, depending on the situation. Using AutoSampler, as in the code example below, users can achieve optimization performance equal to or better than Optuna's default without being aware of which optimization algorithm to use.

$ pip install optunahub cmaes torch scipy

```python import optuna import optunahub

autosamplermodule = optunahub.loadmodule("samplers/autosampler") study = optuna.createstudy(sampler=autosampler_module.AutoSampler()) ```

See the Medium blog post for details.

Enhanced RDB Storage Backend

This release incorporates comprehensive performance tuning on Optuna’s RDBStorage, leading to significant performance improvements. The table below shows the comparison results of execution times between versions 4.0 and 4.1.

| # trials | v4.0.0 | v4.1.0 | Diff | | --- | --- | --- | --- | | 1000 | 72.461 sec (±1.026) | 59.706 sec (±1.216) | -17.60% | | 10000 | 1153.690 sec (±91.311) | 664.830 sec (±9.951) | -42.37% | | 50000 | 12118.413 sec (±254.870) | 4435.961 sec (±190.582) | -63.39% |

For fair comparison, all experiments were repeated 10 times, and the mean execution time was compared. Additional detailed benchmark settings include the following: - Objective Function: Each trial consists of 10 parameters and 10 user attributes - Storage: MySQL 8.0 (with PyMySQL) - Sampler: RandomSampler - Execution Environment: Kubernetes Pod with 5 cpus and 8Gi RAM

Please note, due to extensive execution time, the figure for v4.0.0 with 50,000 trials represents the average of 7 runs instead of 10.

Benchmark Script

```python import optuna import time import os import numpy as np optuna.logging.set_verbosity(optuna.logging.ERROR) storage_url = "mysql+pymysql://user:password@:/" n_repeat = 10 def objective(trial: optuna.Trial) -> float: s = 0 for i in range(10): trial.set_user_attr(f"attr{i}", "dummy user attribute") s += trial.suggest_float(f"x{i}", -10, 10) ** 2 return s def bench(n_trials): elapsed = [] for i in range(n_repeat): start = time.time() study = optuna.create_study( storage=storage_url, sampler=optuna.samplers.RandomSampler() ) study.optimize(objective, n_trials=n_trials, n_jobs=10) elapsed.append(time.time() - start) optuna.delete_study(study_name=study.study_name, storage=storage_url) print(f"{np.mean(elapsed)=} {np.std(elapsed)=}") for n_trials in [1000, 10000, 50000]: bench(n_trials) ```

Five New Algorithms in OptunaHub (MO-CMA-ES, MOEA/D, etc.)

The following five new algorithms were added to OptunaHub!

• Multi-objective CMA-ES (MO-CMA-ES)by @y0z
• MOEA/D sampler by @hrntsm
• MAB Epsilon-Greedy Sampler by @ryota717
• NSGAII sampler with Initial Trials by @hrntsm
• CMA-ES with User Prior by @nabenabe0928

MO-CMA-ES is an extension of CMA-ES for multi-objective optimization. Its search mechanism is based on multiple (1+1)-CMA-ES and inherits good invariance properties from CMA-ES, such as invariance against rotation of the search space.

MOEA/D solves a multi-objective optimization problem by decomposing it into multiple single-objective problems. It allows for the maintenance of a good diversity of solutions during optimization. Please take a look at the article from Hiroaki NATSUME(@hrntsm) for more details.

Enhancements

• Update sklearn.py by addind catch to OptunaSearchCV (https://github.com/optuna/optuna-integration/pull/163, thanks @muhlbach!)
• Reduce SELECT statements by passing study_id to check_and_add in TrialParamModel (#5702)
• Introduce UPSERT in set_trial_user_attr (#5703)
• Reduce SELECT statements of _CachedStorage.get_all_trials by fixing filtering conditions (#5704)
• Reduce SELECT statements by removing unnecessary distribution compatibility check in set_trial_param() (#5709)
• Introduce UPSERT in set_trial_system_attr (#5741)

Bug Fixes

• Accept Mapping as param_distributions in OptunaSearchCV (https://github.com/optuna/optuna-integration/pull/172, thanks @yu9824!)
• Allow use of OptunaSearchCV with cross_val_predict (https://github.com/optuna/optuna-integration/pull/174, thanks @yu9824!)
• Fix GPSampler's suggestion failure within torch.no_grad() context manager (#5671, thanks @kAIto47802!)
• Fix a concurrency issue in GPSampler (#5737)
• Fix a concurrency issue in QMCSampler (#5740)

Installation

• Drop the support for Python 3.7 and update package metadata for Python 3.13 support (#5727)
• Remove dependency specifier for installing SciPy (#5736)

Documentation

• Add badges of PyPI and Conda Forge (https://github.com/optuna/optuna-integration/pull/167)
• Update news (#5655)
• Update news section in README.md (#5657)
• Add InMemoryStorage to document (#5672, thanks @kAIto47802!)
• Add the news about blog of JournalStorage to README.md (#5674)
• Fix broken link in the artifacts tutorial (#5677, thanks @kAIto47802!)
• Add pandas installation guide to RDB tutorial (#5685, thanks @kAIto47802!)
• Add installation guide to multi-objective tutorial (#5686, thanks @kAIto47802!)
• Update document and notice interoperability between NSGA-II and Pruners (#5688)
• Fix typo in EMMREvaluator (#5694)
• Fix RegretBoundEvaluator document (#5696)
• Update news section in README.md (#5705)
• Add link to related blog post in emmr.py (#5707)
• Update FAQ entry for model preservation using Optuna Artifact (#5716, thanks @chitvs!)
• Escape \D for Python 3.12 with sphinx build (#5735)
• Avoid using functions in sphinx_gallery_conf to remove document build error with Python 3.12 (#5738)
• Remove all generated directories in docs/source recursively (#5739)
• Add information about AutoSampler to the docs (#5745)

Examples

• Update CI config for fastai (https://github.com/optuna/optuna-examples/pull/279)
• Introduce optuna.artifacts to the PyTorch checkpoint example (https://github.com/optuna/optuna-examples/pull/280, thanks @kAIto47802!)
• Fix inline code in README files in kubernetes directory (https://github.com/optuna/optuna-examples/pull/282)
• Test ray example with Python 3.12 (https://github.com/optuna/optuna-examples/pull/284)
• Add a link to molecule LLM notebook (https://github.com/optuna/optuna-examples/pull/285)
• Fix syntax error in YAML file of rapids CI (https://github.com/optuna/optuna-examples/pull/286)
• Another fix for syntax error in YAML file of rapids CI (https://github.com/optuna/optuna-examples/pull/287)
• Add checking for Python 3.12 (https://github.com/optuna/optuna-examples/pull/288, thanks @kAIto47802!)
• Add Python 3.12 to tfkeras CI and remove warning message (https://github.com/optuna/optuna-examples/pull/289)
• Add Python 3.12 to lightgbm CI (https://github.com/optuna/optuna-examples/pull/290)
• Remove Python 3.7 from the workflow (https://github.com/optuna/optuna-examples/pull/291)

Code Fixes

• Add a comment for an unexpected bug in CategoricalDistribution (#5683)
• Add more information about the hack in WFG (#5687)
• Simplify type annotations to _imports.py (#5692, thanks @Prabhat-Thapa45!)
• Use __future__.annotations in optuna/_experimental.py (#5714, thanks @Jonathan43!)
• Use __future__.annotations in tests/importance_tests/fanova_tests/test_tree.py (#5731, thanks @guisp03!)
• Resolve TODO comments related to dropping Python 3.7 support (#5734)

Continuous Integration

• Fix CI for fastaiv2 (https://github.com/optuna/optuna-integration/pull/164)
• Fix for mypy (https://github.com/optuna/optuna-integration/pull/166)
• Fix mlflow integration for CI (https://github.com/optuna/optuna-integration/pull/168)
• Update CI to support Python 3.12 (https://github.com/optuna/optuna-integration/pull/170, thanks @kAIto47802!)
• Update artifact version to v4 (https://github.com/optuna/optuna-integration/pull/176)
• Add a tri-objective problem to speed benchmarking (#5635)
• Bump actions/download-artifact from 2 to 4.1.7 in /.github/workflows (#5660)
• Update MySQL version in CI from 5.7 to 8 (#5673)
• Remove performance benchmarks (#5675)
• Update PostgreSQL version in CI to latest (#5676)
• Update the version of Python used in the checks from 3.8 to 3.11 (#5684)
• Run tests with Python 3.13 (#5691)
• Fix for mypy (#5706)
• [hotfix] Add a version constraint on fakeredis (#5726)
• Update Python versions used in CI workflow (#5728)
• Update sphinx build's upload-artifact version (#5744)

Other

• Bump up version number to 4.1.0.dev (https://github.com/optuna/optuna-integration/pull/159)
• Update supported Python versions (https://github.com/optuna/optuna-integration/pull/175)
• Bump up to v4.1.0 (https://github.com/optuna/optuna-integration/pull/179)
• Add pre-commit config (#5408)
• Add link to MOEA/D blog post in News (#5719)
• Include all test files in sdist by updating MANIFEST.in (#5720)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @Jonathan43, @Prabhat-Thapa45, @c-bata, @chitvs, @contramundum53, @eukaryo, @gen740, @guisp03, @kAIto47802, @muhlbach, @nabenabe0928, @not522, @nzw0301, @porink0424, @toshihikoyanase, @y0z, @yu9824

- Python
Published by c-bata over 1 year ago

optuna - v4.0.0

Here is the release note of v4.0.0. Please also check out the release blog post.

If you want to update the Optuna version of your existing projects to v4.0, please see the migration guide.

We have also published blog posts about the development items. Please check them out! - OptunaHub, a Feature-Sharing Platform for Optuna, Now Available in Official Release! - File Management during LLM (Large Language Model) Trainings by Optuna v4.0.0 Artifact Store - Significant Speed Up of Multi-Objective TPESampler in Optuna v4.0.0

Highlights

Official Release of Feature-Sharing Platform OptunaHub

We officially released OptunaHub, a feature-sharing platform for Optuna. A large number of optimization and visualization algorithms are available in OptunaHub. Contributors can easily register their methods and deliver them to Optuna users around the world.

Please also read the OptunaHub release blog post.

Enhanced Experiment Management Feature: Official Support of Artifact Store

Artifact Store is a file management feature for files generated during optimization, dubbed artifacts. In Optuna v4.0, we stabilized the existing file upload API and further enhanced the usability of Artifact Store by adding some APIs such as the artifact download API. We also added features to show JSONL and CSV files on Optuna Dashboard in addition to the existing support for images, audio, and video. With this official support, the API backward compatibility will be guaranteed.

For more details, please check the blog post.

JournalStorage: Official Support of Distributed Optimization via Network File System

JournalStorage is a new Optuna storage experimentally introduced in Optuna v3.1 (see the blog post for details). Optuna has JournalFileBackend, a storage backend for various file systems. It can be used on NFS, allowing Optuna to scale to multiple nodes.

In Optuna v4.0, the API for JournalStorage has been reorganized, and JournalStorage is officially supported. This official support guarantees its backward compatibility from v4.0. For details on the API changes, please refer to the Optuna v4.0 Migration Guide.

```python import optuna from optuna.storages import JournalStorage from optuna.storages.journal import JournalFileBackend

def objective(trial: optuna.Trial) -> float: ...

storage = JournalStorage(JournalFileBackend("./optunajournalstorage.log")) study = optuna.create_study(storage=storage) study.optimize(objective) ```

Significant Speedup of Multi-Objective TPESampler

Before v4.0, the multi-objective TPESampler sometimes limits the number of trials during optimization due to the sampler bottleneck after a few hundred trials. Optuna v4.0 drastically improves the sampling speed, e.g., 300 times faster for three objectives with 200 trials, and enables users to handle much more trials. Please check the blog post for details.

Introduction of a New Terminator Algorithm

Optuna Terminator was originally introduced for hyperparameter optimization of machine learning algorithms using cross-validation. To accept broader use cases, Optuna v4.0 introduced the Expected Minimum Model Regret (EMMR) algorithm. Please refer to the EMMREvaluator document for details.

Enhancements of Constrained Optimization

We have gradually expanded the support for constrained optimization. In v4.0, study.best_trial and study.best_trials start to support constraint optimization. They are guaranteed to satisfy the constraints, which was not the case previously.

Breaking Changes

Optuna removes deprecated features in major releases. To prevent users' code from suddenly breaking, we take a long interval between when a feature is deprecated and when it is removed. By default, features are removed when the major version has increased by two since the feature was deprecated. For this reason, the main target features for removal in v4.0 were deprecated at v2.x. Please refer to the migration guide for the removed features list.

• Delete deprecated three integrations, skopt, catalyst, and fastaiv1 (https://github.com/optuna/optuna-integration/pull/114)
• Remove deprecated CmaEsSampler from integration (https://github.com/optuna/optuna-integration/pull/116)
• Remove verbosity of LightGBMTuner (https://github.com/optuna/optuna-integration/pull/136)
• Move positional args of LightGBMTuner (https://github.com/optuna/optuna-integration/pull/138)
• Remove multi_objective (#5390)
• Delete deprecated _ask and _tell (#5398)
• Delete deprecated --direction(s) arguments in the ask command (#5405)
• Delete deprecated three integrations, skopt, catalyst, and fastaiv1 (#5407)
• Remove the default normalization of importance in f-ANOVA (#5411)
• Remove samplers.intersection (#5414)
• Drop implicit create-study in ask command (#5415)
• Remove deprecated study optimize CLI command (#5416)
• Remove deprecated CmaEsSampler from integration (#5417)
• Support constrained optimization in best_trial (#5426)
• Drop --study in cli.py (#5430)
• Deprecate constraints_func in plot_pareto_front function (#5455)
• Rename some class names related to JournalStorage (#5539)
• Remove optuna.samplers.MOTPESampler (#5640)

New Features

• Add Comet ML integration (https://github.com/optuna/optuna-integration/pull/63, thanks @caleb-kaiser!)
• Add Knowledge Gradient candidates functions (https://github.com/optuna/optuna-integration/pull/125, thanks @alxhslm!)
• Add is_exhausted() function in the GridSampler class (#5306, thanks @aaravm!)
• Remove experimental from plot (#5413)
• Implement download_artifact (#5448)
• Add a function to list linked artifact information (#5467)
• Stabilize artifact APIs (#5567)
• Stabilize JournalStorage (#5568)
• Add EMMREvaluator and MedianErrorEvaluator (#5602)

Enhancements

• Pass two arguments to the forward of ConstrainedMCObjective to support botorch=0.10.0 (https://github.com/optuna/optuna-integration/pull/106)
• Speed up non-dominated sort (#5302)
• Make 2d hypervolume computation twice faster (#5303)
• Reduce the time complexity of HSSP 2d from O(NK^2 log K) to O((N - K)K) (#5346)
• Introduce lazy hypervolume calculations in HSSP for speedup (#5355)
• Make plot_contour faster (#5369)
• Speed up to_internal_repr in CategoricalDistribution (#5400)
• Allow users to modify categorical distance more easily (#5404)
• Speed up WFG by NumPy vectorization (#5424)
• Check whether the study is multi-objective in sample_independent of GPSampler (#5428)
• Suppress warnings from numpy in hypervolume computation (#5432)
• Make an option to assume Pareto optimality in WFG (#5433)
• Adapt multi objective to NumPy v2.0.0 (#5493)
• Enhance the error message for integration installation (#5498)
• Reduce journal size of JournalStorage (#5526)
• Simplify a SQL query for getting the trial_id of best_trial (#5537)
• Add import check for artifact store objects (#5565)
• Refactor _is_categorical() in optuna/optuna/visualization (#5587, thanks @kAIto47802!)
• Speed up WFG by using a fact that hypervolume calculation does not need (second or later) duplicated Pareto solutions (#5591)
• Enhance the error message of multi_objective deletion (#5641)

Bug Fixes

• Log None objective trials correctly (https://github.com/optuna/optuna-integration/pull/119, thanks @neel04!)
• Pass two arguments to the forward of ConstrainedMCObjective in qnei_candidates_func (https://github.com/optuna/optuna-integration/pull/124, thanks @alxhslm!)
• Allow single split cv in OptunaSearchCV (https://github.com/optuna/optuna-integration/pull/128, thanks @sgerloff!)
• Update BoTorch samplers to support new constraints interface (https://github.com/optuna/optuna-integration/pull/132, thanks @alxhslm!)
• Fix WilcoxonPruner bug when best_trial has no intermediate value (#5354)
• Debug an error caused by convergence in GPSampler (#5359)
• Fix average_is_best implementation in WilcoxonPruner (#5366)
• Create an unique renaming filename for each release operation in lock systems of JournalStorage (#5389)
• Fix _normalize_value for incomplete trials (#5422)
• Fix heartbeat for race condition (#5431)
• Guarantee weights_below to be finite in MOTPE (#5435)
• Fix _log_complete_trial for constrained optimization (#5462)
• Convert step to int in report (#5488)
• Use a fixed seed value when seed=None in GridSampler (#5490)
• Acquire session lock only for write in heartbeat (#5496)
• Fix a bug in _create_new_trial and refactor it (#5497)
• Add rdb create new trial test (#5525)
• Fix a bug in sample_normalized_param for GPSampler (#5543)
• Fix the error caused in plot_contour() with an impossible pair of variables (#5630, thanks @kAIto47802!)
• Fix inappropriate behavior in plot_rank() when None values exist in trial (#5634, thanks @kAIto47802!)

Installation

• Add an option to install integration dependencies via pip (https://github.com/optuna/optuna-integration/pull/130)
• Add version constraint to numpy (https://github.com/optuna/optuna-integration/pull/131)

Documentation

• Enhance README.md (https://github.com/optuna/optuna-integration/pull/126)
• Add document page and fix docstring and comments (https://github.com/optuna/optuna-integration/pull/134)
• Update the docstring of PyCmaSampler (https://github.com/optuna/optuna-integration/pull/145)
• Add Dashboard and OptunaHub links to the document header (https://github.com/optuna/optuna-integration/pull/155)
• Remove duplicated license definition (https://github.com/optuna/optuna-integration/pull/156)
• Use sphinx rst syntax (#5345)
• Fix typo (#5351)
• Update docs about show_progress_bar (#5393)
• Fix artifact tutorial (#5451)
• Revise docs to specify ArtifactStore methods as non-public (#5474)
• Improve the docstring of JournalFileStorage (#5475)
• Improve make clean in docs (#5487)
• Improve the example of chemical structures in optuna artifact tutorial (#5491)
• Add FAQ for artifact store remove API (#5501)
• Add a documentation for ArtifactMeta (#5511)
• Change docs version from latest to stable (#5518)
• Add a list of supported storage for artifact store (#5532)
• Rename filenames for JournalFileStorage in documents and a tutorial (#5535)
• Add explanation for lock_obj to JournalFileStorage docstring (#5540)
• Fix storages document sections (#5553)
• Add Returns to the docstring of load_study (#5554, thanks @kAIto47802!)
• Fix paths related to storages.journal (#5560)
• Rename filename for JournalFileBackend in examples (#5562)
• Add thumbnail (#5575)
• Add link to optunahub-registry (#5586)
• Fix minor artifacts document issues (#5592)
• Add the OptunaHub link to the document header (#5595)
• Improve a docstring of CmaEsSampler (#5603)
• Fix the expired links caused by the directory name change in optuna-examples (#5623, thanks @kAIto47802!)
• Remove the use of evaluator.evaluate function in the example of PedAnovaImportanceEvaluator (#5632)
• Add the news section to README.md (#5636)
• Add SNS URLs to README.md (#5637)
• Add TPE blog post link to doc (#5638)
• Add artifact store post link (#5639)
• Add installation guide for visualization tutorial (#5644, thanks @kAIto47802!)
• Remove a broken link (#5648)
• Update the News section of README.md (#5649)

Examples

• Support tensorflow 2.16.1 and separate CI run for tensorflow estimator (https://github.com/optuna/optuna-examples/pull/248)
• Replace deprecated jax function (https://github.com/optuna/optuna-examples/pull/250)
• Drop Python 3.7 support for wandb (https://github.com/optuna/optuna-examples/pull/252)
• Support python 3.12 in the CIs (https://github.com/optuna/optuna-examples/pull/253)
• Remove dask version constraint (https://github.com/optuna/optuna-examples/pull/254)
• Update GitHub actions versions to actions/checkout@v4 and actions/setup-python@v5 (https://github.com/optuna/optuna-examples/pull/255)
• Delete an example using deprecated fastaiv1 (https://github.com/optuna/optuna-examples/pull/256)
• Rename fastaiv2 to fastai (https://github.com/optuna/optuna-examples/pull/257)
• Fix CI (https://github.com/optuna/optuna-examples/pull/258)
• Fix path in pruners workflow (https://github.com/optuna/optuna-examples/pull/259)
• Install tensorflow-cpu in CIs (https://github.com/optuna/optuna-examples/pull/260)
• Add python 3.12 and run terminator search cv (https://github.com/optuna/optuna-examples/pull/264)
• Enhance README.md (https://github.com/optuna/optuna-examples/pull/265)
• Organize the directory structure (https://github.com/optuna/optuna-examples/pull/266)
• Fix CI with version constraints of packages (https://github.com/optuna/optuna-examples/pull/267)
• Install numpy<2.0.0 for catboost example (https://github.com/optuna/optuna-examples/pull/268)
• Add Python 3.12 to aim test python versions (https://github.com/optuna/optuna-examples/pull/270)
• Add Python 3.12 to fastai test python versions (https://github.com/optuna/optuna-examples/pull/271)
• Use auc score to unify the intermediate and objective values (https://github.com/optuna/optuna-examples/pull/272)
• Add python 3.12 for mlflow CI python versions (https://github.com/optuna/optuna-examples/pull/274)
• Add python 3.12 to CI python versions for keras and tensorboard examples (https://github.com/optuna/optuna-examples/pull/275)
• Add an example of artifact store (https://github.com/optuna/optuna-examples/pull/276)
• Separate basic and faq directories (https://github.com/optuna/optuna-examples/pull/277, thanks @kAIto47802!)
• Remove chainer CI (https://github.com/optuna/optuna-examples/pull/278)

Tests

• Suppress ExperimentalWarnings (https://github.com/optuna/optuna-integration/pull/108)
• Add a unit test for convergence of acquisition function in GPSampler (#5365)
• Remove unused block (#5368)
• Implement backward compatibility tests for JournalStorage (#5486)
• Add some tests for visualization functions (#5599)

Code Fixes

• Align run name strategy between MLflowCallback and track_in_mlflow method (https://github.com/optuna/optuna-integration/pull/111, thanks @TTRh!)
• Use TYPE_CHECKING for ObjectiveFuncType (https://github.com/optuna/optuna-integration/pull/113)
• Ignore ExperimentalWarning for track_in_wandb (https://github.com/optuna/optuna-integration/pull/122)
• Remove unused module (https://github.com/optuna/optuna-integration/pull/127)
• Apply formatter and follow the Optuna conventions (https://github.com/optuna/optuna-integration/pull/133)
• Refactor an internal process of BoTorchSampler (https://github.com/optuna/optuna-integration/pull/147)
• Add missing experimental decorators to the comet integration (https://github.com/optuna/optuna-integration/pull/148)
• Use __future__.annotations (https://github.com/optuna/optuna-integration/pull/150)
• Fix E721 errors (https://github.com/optuna/optuna-integration/pull/151)
• Fix future annotations in percentile.py (#5322, thanks @aaravm!)
• Delete examples directory (#5329)
• Simplify annotations in optuna/pruners/_hyperband.py (#5338, thanks @keita-sa!)
• Simplify annotations in optuna/pruners/_successive_halving.py and optuna/pruners/_threshold.py (#5343, thanks @keita-sa!)
• Reduce test warnings (#5344)
• Simplify type annotations for storages/_base.py (#5352, thanks @Obliquedbishop!)
• Use TYPE_CHECKING for Study in samplers (#5391)
• Refactor _normalize_objective_values in NSGA-III to supress RuntimeWarning (#5399)
• Make ObjectiveFuncType available externally (#5401)
• Replace numpy with np (#5412)
• Bundle experimental feature warning (#5434)
• Simplify annotations in _brute_force.py (#5441)
• Add __future__.annotations to base sampler (#5442)
• Introduce __future__.annotations to TPE-related modules (#5443)
• Adapt to __future__.annotations in optuna/storages/_rdb/models.py (#5452, thanks @aisha-partha!)
• Debug an unintended assertion error in GPSampler (#5484)
• Make WFG a function (#5504)
• Enhance the comments in create_new_trial (#5510)
• Replace single trailing underscore with double trailing underscore in .rst files (#5514, thanks @47aamir!)
• Fix hyperlinks to use double trailing underscores (#5515, thanks @virendrapatil24!)
• Replace _ with __ in the link in Sphinx (#5517)
• Refactor plot_parallel_coordinate() (#5527, thanks @karthikkurella!)
• Remove an obsolete TODO comment (#5528)
• Unite the argument order of artifact APIs (#5533)
• Make assume_unique_lexsorted in is_pareto_front required (#5534)
• Simplify annotations in terminator module (#5536)
• Simplify the type annotations in samplers/_qmc.py (#5538, thanks @kAIto47802!)
• Rename solution_set to loss_vals in hypervolume computation (#5541)
• Expand the type of callbacks in optimize to Iterable (#5542, thanks @kz4killua!)
• Organize the directory structure of JournalStorage (#5544)
• Add followup of journal storage module structure organization PR (#5546)
• Move RetryFailedTrialCallback to optuna.storages._callbacks (#5551)
• Change removed_version of deprecated JournalStorage classes (#5552)
• Reorganize a journal storage module structure (#5555)
• Remove unused private deprecated class BaseJournalLogSnapshot (#5561)
• Replace relative import to absolute path (#5569, thanks @RektPunk!)
• Simplify annotation in importance (#5578, thanks @RektPunk!)
• Simplify annotation in trial (#5579, thanks @RektPunk!)
• Fix mypy warnings in matplotlib/_contour.py with Python3.11 (#5580)
• Remove redundant loop in _get_rank_subplot_info (#5581, thanks @RektPunk!)
• Simplify annotations in _partial_fixed.py (#5583)
• Simplify annotation in _cmaes.py (#5584, thanks @RektPunk!)
• Simplify annotations related to GridSampler (#5588)
• Reorganize names related to search_space in terminator/improvement/evaluator.py (#5594)
• Refactor _bisect in _truncnorm.py (#5598)
• Simplify annotation storages (#5608, thanks @RektPunk!)
• Add a note about np.unique (#5615)
• Fix ruff errors except for E731 (#5627)

Continuous Integration

• Split ci (https://github.com/optuna/optuna-integration/pull/98)
• Install pipdeptree v2.16.2 to hotfix parse error (https://github.com/optuna/optuna-integration/pull/109)
• Remove dask version constraint (https://github.com/optuna/optuna-integration/pull/112)
• Rename CI name (https://github.com/optuna/optuna-integration/pull/120)
• Add deprecated arg for the comet CI job (https://github.com/optuna/optuna-integration/pull/135)
• Hotfix the CI in LightGBM test (https://github.com/optuna/optuna-integration/pull/142)
• Hotfix the mypy error of test_pytorch_lightning.py (https://github.com/optuna/optuna-integration/pull/144)
• Rename CI jobs (#5353, thanks @Obliquedbishop!)
• Update GitHub actions versions of actions/setup-python and actions/checkout (#5367)
• Add type: ignore for CI hotfix (#5419)
• Use CPU-only PyTorch wheels on GitHub Actions (#5465)
• Add a version constraint to NumPy (#5492)
• Add the note of the Hotfix in the workflow file (#5494)
• Remove unnecessary version constraint from workflow (#5495)
• Remove skipped tests for BoTorch with Python 3.12 (#5519)
• Fix the version conflicts in the workflow for the minimum version constraints (#5523)
• Replace sleep function with spin waiting to stabilize the frequent Mac test failure (#5549)

Other

• Bump up version number to 4.0.0dev (https://github.com/optuna/optuna-integration/pull/102)
• Update PyPI classifiers for Python 3.11 support (https://github.com/optuna/optuna-integration/pull/129)
• Bump up version number to 4.0.0b0 (https://github.com/optuna/optuna-integration/pull/139)
• Bump up version number to 4.0.0 (https://github.com/optuna/optuna-integration/pull/158)
• Bump up to version number v4.0.0.dev (#5319)
• Bump up to version number 4.0.0b0 (#5572)
• Split LICENSE file (#5597)
• Add OptunaHub section in README.md (#5601)
• Remove duplicated definition of license in pyproject.toml (#5645)
• Bump up to version number 4.0.0 (#5653)
• Bump up to version number 4.1.0dev (#5654)

Thanks to All the Contributors and Sponsors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@47aamir, @Alnusjaponica, @HideakiImamura, @Obliquedbishop, @RektPunk, @TTRh, @aaravm, @aisha-partha, @alxhslm, @c-bata, @caleb-kaiser, @contramundum53, @eukaryo, @gen740, @kAIto47802, @karthikkurella, @keisuke-umezawa, @keita-sa, @kz4killua, @nabenabe0928, @neel04, @not522, @nzw0301, @porink0424, @sgerloff, @toshihikoyanase, @virendrapatil24, @y0z

Optuna is sponsored by the following sponsors on GitHub.

@AlphaImpact, @dec1costello, @dubovikmaster, @shu65, @raquelhortab

- Python
Published by not522 almost 2 years ago

optuna - v4.0.0-b0

This is the release note of v4.0.0-b0.

If you want to update your existing projects from Optuna v3.x to Optuna v4, please see the migration guide and try out Optuna v4.

Highlights

OptunaHub Beta Release

The Optuna team released the beta version of OptunaHub, the feature-sharing platform for Optuna. Registered features can be easily implemented on users’ code and contributors can register the features they implement. The beta version of OptunaHub is now ready to accept contributions from all over the world. Visit hub.optuna.org!

The following code shows an example to use a sampler registered on OptunaHub.

bash % pip install optunahub

```python import optunahub import optuna

def objective(trial): x = trial.suggest_float("x", 0, 1) return x

mod = optunahub.loadmodule("samplers/simulatedannealing")

sampler = mod.SimulatedAnnealingSampler() study = optuna.createstudy(sampler=sampler) study.optimize(objective, ntrials=20) ```

Stabilization of Artifact

The stable version of the artifact module is available, now equipped with several new APIs. This module introduces capabilities for managing the relatively large-sized data such as model snapshots in hyperparameter tuning, training/validation datasets, and etc. Compared to third-party libraries for experiment tracking, the advantage of using Optuna’s artifact module is a tight integration of Optuna Dashboard. This allows users to see artifacts (files) associated with the Optuna trial or study.

Here is a list of new APIs: - download_artifact: Download an artifact from the artifact store. - get_all_artifact_meta: List the associated artifact information of the provided trial or study.

Stabilization of JournalStorage

The stable version of JournalStorage is available. This implies we have decided to maintain backward compatibility of the log format in the future releases.

Please note that this release introduces the following API changes to improve the clarity of class names and the module structure.

| Deprecated APIs | Corresponding active APIs | -|- | optuna.storages.JournalFileStorage | optuna.storages.journal.JournalFileBackend | | optuna.storages.JournalFileSymlinkLock | optuna.storages.journal.JournalFileSymlinkLock | | optuna.storages.JournalFileOpenLock | optuna.storages.journal.JournalFileOpenLock | | optuna.storages.JournalRedisStorage | optuna.storages.journal.JournalRedisBackend |

Breaking Changes

• Delete deprecated three integrations, skopt, catalyst, and fastaiv1 (https://github.com/optuna/optuna-integration/pull/114)
• Remove deprecated CmaEsSampler from integration (https://github.com/optuna/optuna-integration/pull/116)
• Remove verbosity of LightGBMTuner (https://github.com/optuna/optuna-integration/pull/136)
• Move positional args of LightGBM tuner (https://github.com/optuna/optuna-integration/pull/138)
• Remove multi_objective (#5390)
• Delete deprecated _ask and _tell (#5398)
• Delete deprecated --direction(s) arguments in the ask command (#5405)
• Delete deprecated three integrations, skopt, catalyst, and fastaiv1 (#5407)
• Remove the default normalization of importance in f-ANOVA (#5411)
• Remove samplers.intersection (#5414)
• Drop implicit create-study in ask command (#5415)
• Remove deprecated study optimize CLI command (#5416)
• Remove deprecated CmaEsSampler from integration (#5417)
• Support constrained optimization in best_trial (#5426)
• Drop --study in cli.py (#5430)
• Deprecate constraints_func in plot_pareto_front function (#5455)
• Rename some classnames related to JournalStorage (#5539)

New Features

• Add Comet ML integration (https://github.com/optuna/optuna-integration/pull/63, thanks @caleb-kaiser!)
• Add Knowledge Gradient candidates functions (https://github.com/optuna/optuna-integration/pull/125, thanks @alxhslm!)
• Add is_exhausted() function in the GridSampler class (#5306, thanks @aaravm!)
• Remove experimental from plot (#5413)
• Implement download_artifact (#5448)
• Add a function to list linked artifact information (#5467)
• Stabilize artifact APIs (#5567)
• Stabilize JournalStorage (#5568)

Enhancements

• Pass two arguments to the forward of ConstrainedMCObjective to support botorch=0.10.0 (https://github.com/optuna/optuna-integration/pull/106)
• Speed up non-dominated sort (#5302)
• Make 2d hypervolume computation twice faster (#5303)
• Reduce the time complexity of HSSP 2d from O(NK^2 log K) to O((N - K)K) (#5346)
• Introduce lazy hypervolume calculations in HSSP for speedup (#5355)
• Make plot_contour faster (#5369)
• Speed up to_internal_repr in CategoricalDistribution (#5400)
• Allow users to modify categorical distance more easily (#5404)
• Speed up WFG by NumPy vectorization (#5424)
• Check whether the study is multi-objective in sample_independent of GPSampler (#5428)
• Suppress warnings from numpy in hypervolume computation (#5432)
• Make an option to assume Pareto optimality in WFG (#5433)
• Adapt multi objective to NumPy v2.0.0 (#5493)
• Enhance the error message for integration installation (#5498)
• Reduce journal size of JournalStorage (#5526)
• Simplify a SQL query for getting the trial_id of best_trial (#5537)

Bug Fixes

• Log None objective trials correctly (https://github.com/optuna/optuna-integration/pull/119, thanks @neel04!)
• Pass two arguments to the forward of ConstrainedMCObjective in qnei_candidates_func (https://github.com/optuna/optuna-integration/pull/124, thanks @alxhslm!)
• Allow single split cv in OptunaSearchCV (https://github.com/optuna/optuna-integration/pull/128, thanks @sgerloff!)
• Fix WilcoxonPruner bug when best_trial has no intermediate value (#5354)
• Debug an error caused by convergence in GPSampler (#5359)
• Fix average_is_best implementation in WilcoxonPruner (#5366)
• Create an unique renaming filename for each release operation in lock systems of JournalStorage (#5389)
• Fix _normalize_value for incomplete trials (#5422)
• Fix heartbeat for race condition (#5431)
• Guarantee weights_below to be finite in MOTPE (#5435)
• Fix _log_complete_trial for constrained optimization (#5462)
• Convert step to int in report (#5488)
• Use a fixed seed value when seed=None in GridSampler (#5490)
• Acquire session lock only for write in heartbeat (#5496)
• Fix a bug in _create_new_trial and refactor it (#5497)
• Add rdb create new trial test (#5525)
• Fix a bug in sample_normalized_param for GPSampler (#5543)

Installation

• Add an option to install integration dependencies via pip (https://github.com/optuna/optuna-integration/pull/130)
• Add version constraint to numpy (https://github.com/optuna/optuna-integration/pull/131)

Documentation

• Enhance README.md (https://github.com/optuna/optuna-integration/pull/126)
• Add document page and fix docstring and comments (https://github.com/optuna/optuna-integration/pull/134)
• Use sphinx rst syntax (#5345)
• Fix typo (#5351)
• Update docs about show_progress_bar (#5393)
• Fix artifact tutorial (#5451)
• Revise docs to specify ArtifactStore methods as non-public (#5474)
• Improve the docstring of JournalFileStorage (#5475)
• Improve make clean in docs (#5487)
• Improve the example of chemical structures in optuna artifact tutorial (#5491)
• Add FAQ for artifact store remove API (#5501)
• Add a documentation for ArtifactMeta (#5511)
• Change docs version from latest to stable (#5518)
• Add a list of supported storage for artifact store (#5532)
• Rename filenames for JournalFileStorage in documents and a tutorial (#5535)
• Add explanation for lock_obj to JournalFileStorage docstring (#5540)
• Fix storages document sections (#5553)
• Add Returns to the docstring of load_study (#5554, thanks @kAIto47802!)
• Fixing paths related to storages.journal (#5560)
• Rename filename for JournalFileBackend in examples (#5562)

Examples

• Support tensorflow 2.16.1 and separate CI run for tensorflow estimator (https://github.com/optuna/optuna-examples/pull/248)
• Replace deprecated jax function (https://github.com/optuna/optuna-examples/pull/250)
• Drop Python 3.7 support for wandb (https://github.com/optuna/optuna-examples/pull/252)
• Support python 3.12 in the CIs (https://github.com/optuna/optuna-examples/pull/253)
• Remove dask version constraint (https://github.com/optuna/optuna-examples/pull/254)
• Update GitHub actions versions to actions/checkout@v4 and actions/setup-python@v5 (https://github.com/optuna/optuna-examples/pull/255)
• Delete an example using deprecated fastaiv1 (https://github.com/optuna/optuna-examples/pull/256)
• Rename fastai v2 to fastai (https://github.com/optuna/optuna-examples/pull/257)
• Fix CI (https://github.com/optuna/optuna-examples/pull/258)
• Fix path in pruners workflow (https://github.com/optuna/optuna-examples/pull/259)
• Install tensorflow-cpu in CIs (https://github.com/optuna/optuna-examples/pull/260)
• Add python 3.12 and run terminator search cv (https://github.com/optuna/optuna-examples/pull/264)
• Organize the directory structure (https://github.com/optuna/optuna-examples/pull/266)
• Fix CI with version constraints of packages (https://github.com/optuna/optuna-examples/pull/267)
• Install numpy<2.0.0 for catboot example (https://github.com/optuna/optuna-examples/pull/268)

Tests

• Suppress ExperimentalWarnings (https://github.com/optuna/optuna-integration/pull/108)
• Add a unit test for convergence of acquisition function in GPSampler (#5365)
• Remove unused block (#5368)
• Implement backward compatibility tests for JournalStorage (#5486)

Code Fixes

• Align run name strategy between MLflowCallback and trackinmlflow method (https://github.com/optuna/optuna-integration/pull/111, thanks @TTRh!)
• Use TYPE_CHECKING for ObjectiveFuncType (https://github.com/optuna/optuna-integration/pull/113)
• Ignore ExperimentalWarning for track_in_wandb (https://github.com/optuna/optuna-integration/pull/122)
• Remove unused module (https://github.com/optuna/optuna-integration/pull/127)
• Apply formatter and follow the Optuna conventions (https://github.com/optuna/optuna-integration/pull/133)
• Fix future annotations in percentile.py (#5322, thanks @aaravm!)
• Delete examples directory (#5329)
• Simplify annotations in optuna/pruners/_hyperband.py (#5338, thanks @keita-sa!)
• Simplify annotations in optuna/pruners/_successive_halving.py and optuna/pruners/_threshold.py (#5343, thanks @keita-sa!)
• Reduce test warnings (#5344)
• Simplify type annotations for storages/_base.py (#5352, thanks @Obliquedbishop!)
• Use TYPE_CHECKING for Study in samplers (#5391)
• Refactor _normalize_objective_values in NSGA-III to supress RuntimeWarning (#5399)
• Make ObjectiveFuncType available externally (#5401)
• Replace numpy with np (#5412)
• Bundle experimental feature warning (#5434)
• Simplify annotations in _brute_force.py (#5441)
• Add __future__.annotations to base sampler (#5442)
• Introduce __future__.annotations to TPE-related modules (#5443)
• Adapt to __future__.annotations in optuna/storages/_rdb/models.py (#5452, thanks @aisha-partha!)
• Debug an unintended assertion error in GPSampler (#5484)
• Make WFG a function (#5504)
• Enhance the comments in create_new_trial (#5510)
• Replace single trailing underscore with double trailing underscore in .rst files (#5514, thanks @47aamir!)
• Fix hyperlinks to use double trailing underscores (#5515, thanks @virendrapatil24!)
• Replace _ with __ in the link in Sphinx (#5517)
• Refactor plot_parallel_coordinate() (#5527, thanks @karthikkurella!)
• Remove an obsolete TODO comment (#5528)
• Unite the argument order of artifact APIs (#5533)
• Make assume_unique_lexsorted in is_pareto_front required (#5534)
• Simplify annotations in terminator module (#5536)
• Simplify the type annotations in samplers/_qmc.py (#5538, thanks @kAIto47802!)
• Rename solution_set to loss_vals in hypervolume computation (#5541)
• Expand the type of callbacks in optimize to Iterable (#5542, thanks @kz4killua!)
• Organize the directory structure of JournalStorage (#5544)
• Add followup of journal storage module structure organization PR (#5546)
• Move RetryFailedTrialCallback to optuna.storages._callbacks (#5551)
• Change removed_version of deprecated JournalStorage classes (#5552)
• Reorganize a journal storage module structure (#5555)
• Remove unused private deprecated class BaseJournalLogSnapshot (#5561)

Continuous Integration

• Split ci (https://github.com/optuna/optuna-integration/pull/98)
• Install pipdeptree v2.16.2 to hotfix parse error (https://github.com/optuna/optuna-integration/pull/109)
• Remove dask version constraint (https://github.com/optuna/optuna-integration/pull/112)
• Rename CI name (https://github.com/optuna/optuna-integration/pull/120)
• Add deprecated arg for the comet CI job (https://github.com/optuna/optuna-integration/pull/135)
• Rename CI jobs (#5353, thanks @Obliquedbishop!)
• Update GitHub actions versions of actions/setup-python and actions/checkout (#5367)
• Add type: ignore for CI hotfix (#5419)
• Use CPU-only PyTorch wheels on GitHub Actions (#5465)
• Add a version constraint to NumPy (#5492)
• Add the note of the Hotfix in the workflow file (#5494)
• Remove unnecessary version constraint from workflow (#5495)
• Remove skipped tests for BoTorch with Python 3.12 (#5519)
• Fix the version conflicts in the workflow for the minimum version constraints (#5523)
• Replace sleep function with spin waiting to stabilize the frequent Mac test failure (#5549)

Other

• Bump up version number to 4.0.0dev (https://github.com/optuna/optuna-integration/pull/102)
• Update PyPI classifiers for Python 3.11 support (https://github.com/optuna/optuna-integration/pull/129)
• Bump up version number to 4.0.0b0 (https://github.com/optuna/optuna-integration/pull/139)
• Bump up to version number v4.0.0.dev (#5319)
• Bump up to version number 4.0.0b0 (#5572)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@47aamir, @Alnusjaponica, @HideakiImamura, @Obliquedbishop, @TTRh, @aaravm, @aisha-partha, @alxhslm, @c-bata, @caleb-kaiser, @contramundum53, @eukaryo, @gen740, @kAIto47802, @karthikkurella, @keisuke-umezawa, @keita-sa, @kz4killua, @nabenabe0928, @neel04, @not522, @nzw0301, @porink0424, @sgerloff, @toshihikoyanase, @virendrapatil24, @y0z

- Python
Published by not522 almost 2 years ago

optuna - v3.6.1

This is the release note of v3.6.1.

Bug Fixes

• [Backport] Fix Wilcoxon pruner bug when best_trial has no intermediate value #5370
• [Backport] Address issue#5358 (#5371)
• [Backport] Fix average_is_best implementation in WilcoxonPruner (#5373)

Other

• Bump up version number to v3.6.1 (#5372)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @eukaryo, @nabenabe0928

- Python
Published by eukaryo about 2 years ago

optuna - v3.6.0

This is the release note of v3.6.0.

Highlights

Optuna 3.6 newly supports the following new features. See our release blog for more detailed information. - Wilcoxon Pruner: New Pruner Based on Wilcoxon Signed-Rank Test - Lightweight Gaussian Process (GP)-Based Sampler - Speeding up Importance Evaluation with PED-ANOVA - Stricter Verification Logic for FrozenTrial - Refactoring the Optuna Dashboard - Migration to Optuna Integration

Breaking Changes

• Implement optuna.terminator using optuna._gp (#5241)

These migration-related PRs do not break the backward compatibility as long as optuna-integration v3.6.0 or later is installed in your environment.

• Move TensorBoard Integration (https://github.com/optuna/optuna-integration/pull/56, thanks @dheemantha-bhat!)
• Delete TensorBoard integration for migration to optuna-integration (#5161, thanks @dheemantha-bhat!)
• Remove CatBoost integration for isolation (#5198)
• Remove PyTorch integration (#5213)
• Remove Dask integration (#5222)
• Migrate the sklearn integration (#5225)
• Remove BoTorch integration (#5230)
• Remove SkoptSampler (#5234)
• Remove the cma integration (#5236)
• Remove the wandb integration (#5237)
• Remove XGBoost Integration (#5239)
• Remove MLflow integration (#5246)
• Migrate LightGBM integration (#5249)
• Add CatBoost integration (https://github.com/optuna/optuna-integration/pull/61)
• Add PyTorch integration (https://github.com/optuna/optuna-integration/pull/62)
• Add XGBoost integration (https://github.com/optuna/optuna-integration/pull/65, thanks @buruzaemon!)
• Add sklearn integration (https://github.com/optuna/optuna-integration/pull/66)
• Move Dask integration (https://github.com/optuna/optuna-integration/pull/67)
• Migrate BoTorch integration (https://github.com/optuna/optuna-integration/pull/72)
• Move SkoptSampler (https://github.com/optuna/optuna-integration/pull/74)
• Migrate pycma integration (https://github.com/optuna/optuna-integration/pull/77)
• Migrate the Weights & Biases integration (https://github.com/optuna/optuna-integration/pull/79)
• Add LightGBM integration (https://github.com/optuna/optuna-integration/pull/81, thanks @DanielAvdar!)
• Migrate MLflow integration (https://github.com/optuna/optuna-integration/pull/84)

New Features

• Backport the change of the timeline plot in Optuna Dashboard (#5168)
• Wilcoxon pruner (#5181)
• Add GPSampler (#5185)
• Add a super quick f-ANOVA algorithm named PED-ANOVA (#5212)

Enhancements

• Add formats.sh based on optuna/master (https://github.com/optuna/optuna-integration/pull/75)
• Use vectorization for categorical distance (#5147)
• Unify implementation of fast non-dominated sort (#5160)
• Raise TypeError if params is not a dict in enqueue_trial (#5164, thanks @adjeiv!)
• Upgrade FrozenTrial._validate() (#5211)
• Import SQLAlchemy lazily (#5215)
• Add UCB for optuna._gp (#5224)
• Enhance performance of GPSampler (#5274)
• Fix inconsistencies between terminator and its visualization (#5276, thanks @SimonPop!)
• Enhance GPSampler performance other than introducing local search (#5279)

Bug Fixes

• Fix import path (https://github.com/optuna/optuna-integration/pull/83)
• Fix README.md (https://github.com/optuna/optuna-integration/pull/88)
• Fix LightGBMTuner test (https://github.com/optuna/optuna-integration/pull/89)
• Fix JSONDecodeError in JournalStorage (#5195)
• Fix trial validation (#5229)
• Make gp.fit_kernel_params more robust (#5247)
• Fix checking value in study.tell (#5269, thanks @ryota717!)
• Fix _split_trials of TPESampler for constrained optimization with constant liar (#5298)
• Make each importance evaluator compatible with doc (#5311)

Documentation

• Remove study optimize from CLI tutorial page (#5152)
• Clarify the GridSampler with ask-and-tell interface (#5153)
• Clean-up faq.rst (#5170)
• Make Methods section hidden from Artifact Docs (#5188)
• Enhance README (#5189)
• Add a new section explaing how to customize figures (#5194)
• Replace legacy plotly.graph_objs with plotly.graph_objects (#5223)
• Add a note section to explain that reseed affects reproducibility (#5233)
• Update links to papers (#5235)
• adding link for module's example to documetation for the optuna.terminator module (#5243, thanks @HarshitNagpal29!)
• Replace the old example directory (#5244)
• Add Optuna Dashboard section to docs (#5250, thanks @porink0424!)
• Add a safety guard to Wilcoxon pruner, and modify the docstring (#5256)
• Replace LightGBM with PyTorch-based example to remove lightgbm dependency in visualization tutorial (#5257)
• Remove unnecessary comment in Specify Hyperparameters Manually tutorial page (#5258)
• Add a tutorial of Wilcoxon pruner (#5266)
• Clarify that pruners module does not support multi-objective optimization (#5270)
• Minor fixes (#5275)
• Add a guide to PED-ANOVA for n_trials>10000 (#5310)
• Minor fixes of docs and code comments for PedAnovaImportanceEvaluator (#5312)
• Fix doc for WilcoxonPruner (#5313)
• Fix doc example in WilcoxonPruner (#5315)

Examples

• Remove Python 3.7 and 3.8 from tensorboard CI (https://github.com/optuna/optuna-examples/pull/231)
• Specify black version in the CI (https://github.com/optuna/optuna-examples/pull/232)
• Apply Black 2024 to codebase (https://github.com/optuna/optuna-examples/pull/236)
• Remove MXNet examples (https://github.com/optuna/optuna-examples/pull/237)
• Add an example of Wilcoxon pruner (https://github.com/optuna/optuna-examples/pull/238)
• Make Keras examples Keras 3 friendly (https://github.com/optuna/optuna-examples/pull/239)
• Remove a comment for keras that is not used anymore in this file (https://github.com/optuna/optuna-examples/pull/240)
• Use Keras 3 friendly syntax in MLflow example (https://github.com/optuna/optuna-examples/pull/242)
• Remove -pre option in the rl integration (https://github.com/optuna/optuna-examples/pull/243)
• Hotfix CI by adding version constraints to dask and tensorflow (https://github.com/optuna/optuna-examples/pull/245)

Tests

• Unify the implementation of _create_frozen_trial() under testing module (#5157)
• Remove the Python version constraint for PyTorch (#5278)

Code Fixes

• Fix unused (and unintended) import (https://github.com/optuna/optuna-integration/pull/68)
• Add Dask to __init__.py and fix its documentation generation (https://github.com/optuna/optuna-integration/pull/71)
• Replace optuna.integration with optuna_integration in the doc and the issue template (https://github.com/optuna/optuna-integration/pull/73)
• Fix the doc for TensorFlow (https://github.com/optuna/optuna-integration/pull/76)
• Add skopt dependency (https://github.com/optuna/optuna-integration/pull/78)
• Fastai readme fix (https://github.com/optuna/optuna-integration/pull/82, thanks @DanielAvdar!)
• Fix __init__.py (https://github.com/optuna/optuna-integration/pull/86)
• Apply Black 2024 to codebase (https://github.com/optuna/optuna-integration/pull/87)
• Change the order of dependencies by name (https://github.com/optuna/optuna-integration/pull/92)
• Remove the deprecated decorator of KerasPruningCallback (https://github.com/optuna/optuna-integration/pull/93)
• Remove UserWarning by tests/test_keras.py (https://github.com/optuna/optuna-integration/pull/94)
• Refactor TPESampler for more clarity before c-TPE integration (#5117)
• Fix Checks(integration) failure (#5167)
• Fix type annotation of logging (#5176)
• Update NamedTuple in _ParzenEstimatorParameters to more modern style (#5193)
• Apply Black 2024 to codebase (#5252)
• Simplify annotations in optuna/study/_optimize.py (#5261, thanks @shahpratham!)
• Unify and refactor plot_timeline test (#5281)

Continuous Integration

• Remove non oldest and latest Python versions from tests (https://github.com/optuna/optuna-integration/pull/44)
• Fix flake8 failure in CI (https://github.com/optuna/optuna-integration/pull/55)
• Delete workflow dispatch input (https://github.com/optuna/optuna-integration/pull/57)
• Fix default branch (https://github.com/optuna/optuna-integration/pull/58)
• Fix coverage source path (https://github.com/optuna/optuna-integration/pull/60)
• Not use black 24.* (https://github.com/optuna/optuna-integration/pull/64)
• Simplify integration test (https://github.com/optuna/optuna-integration/pull/95)
• Hotfix the version of botorch<0.10. for CI failures (https://github.com/optuna/optuna-integration/pull/96)
• Hotfix the CI error by adding version constraint to dask (https://github.com/optuna/optuna-integration/pull/99)
• Fix tests with MPI (#5166)
• Fix Checks (Integration) CI for NumPy 1.23.5 (#5177)
• Add version constraint for black (#5210)
• Skip the reproducibility tests for lightgbm (#5214)
• Fix the errors in mypy for the Checks (Integration) CI (#5217)
• Add a version constraint for Torch (#5221)
• Hotfix mypy error in integration (#5232)
• Skip test_reproducible_in_other_process for GPSampler with Python 3.12 (#5251)
• Add CI settings to test Matplotlib without Plotly (#5263, thanks @DanielAvdar!)
• Unify indent size, two in toml file (#5271)
• Follow up for split integrations (#5277)
• Add a version constraint to fakeredis (#5307)

Other

• Bump up version number to 3.6.0.dev (https://github.com/optuna/optuna-integration/pull/53)
• Bump up version number to 3.6.0 (https://github.com/optuna/optuna-integration/pull/100)
• Bump the version up to v3.6.0.dev (#5143)
• Ignore auto generated files by Sphinx (#5192)
• Delete labeler.yml to disable the triage action (#5240)
• Bump up to version number 3.6.0 (#5318)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @DanielAvdar, @HarshitNagpal29, @HideakiImamura, @SimonPop, @adjeiv, @buruzaemon, @c-bata, @contramundum53, @dheemantha-bhat, @eukaryo, @gen740, @hrntsm, @knshnb, @nabenabe0928, @not522, @nzw0301, @porink0424, @ryota717, @shahpratham, @toshihikoyanase, @y0z

- Python
Published by HideakiImamura over 2 years ago

optuna - v3.5.0

This is the release note of v3.5.0.

Highlights

This is a maintenance release with various bug fixes and improvements to the documentation and more.

Breaking Changes

• Isolate the fast.ai module from optuna (https://github.com/optuna/optuna-integration/pull/49, thanks @sousu4!)
• Change n_objectives condition to be greater than 4 in candidates functions (#5121, thanks @adjeiv!)

New Features

• Support constraints in plot contour (#4975, thanks @y-kamiya!)
• Support infeasible coloring for plot_timeline (#5014)
• Support constant_liar in multi-objective TPESampler (#5021)
• Add optuna study-names cli (#5029)
• Use ExpectedHypervolumeImprovement candidates function for BotorchSampler (#5065, thanks @adjeiv!)
• Fix logeicandidatesfunc in botorch.py (#5094, thanks @sousu4!)
• Report CV scores from within OptunaSearchCV (#5098, thanks @adjeiv!)

Enhancements

• Support constant_liar in multi-objective TPESampler (#5021)
• Make positional args to kwargs in suggest_int (#5044)
• Ensure n_below is never negative in TPESampler (#5074, thanks @p1kit!)
• Improve visibility of infeasible trials in plot_contour (#5107)

Bug Fixes

• Fix random number generator of NSGAIIChildGenerationStrategy (#5003)
• Return trials for above in MO split when n_below=0 (#5079)
• Enable loading of read-only files (#5103, thanks @Guillaume227!)
• Fix logpdf for scaled truncnorm (#5110)
• Fix the bug of matplotlib's plot_rank function (#5133)

Documentation

• Add the table of dependencies in each integration module (#5005)
• Enhance the documentation of LightGBM tuner and separate train() from __init__.py (#5010)
• Update link to reference (#5064)
• Update the FAQ on reproducible optimization results to remove note on HyperbandPruner (#5075, thanks @felix-cw!)
• Remove MOTPESampler from index.rst file (#5084, thanks @Ashhar-24!)
• Add a note about the deprecation of MOTPESampler to the doc (#5086)
• Add the TPE tutorial paper to the doc-string (#5096)
• Update README.md to fix the installation and integration (#5126)
• Clarify that Recommended budgets include n_startup_trials (#5137)

Examples

• Update version syntax for PyTorch and PyTorch Lightning examples (https://github.com/optuna/optuna-examples/pull/205, thanks @JustinGoheen!)
• Update import path (https://github.com/optuna/optuna-examples/pull/213)
• Bump up python versions (https://github.com/optuna/optuna-examples/pull/214)
• Add the simplest example directly to README (https://github.com/optuna/optuna-examples/pull/215)
• Add simples examples for multi-objective and constrained optimizations (https://github.com/optuna/optuna-examples/pull/216)
• Revise the comment to describe the problem (https://github.com/optuna/optuna-examples/pull/217)
• Modify simple examples based on the Optuna code conventions (https://github.com/optuna/optuna-examples/pull/218)
• Remove version specification of jax and jaxlib (https://github.com/optuna/optuna-examples/pull/223)
• Import examples from optuna/optuna-dashboard (https://github.com/optuna/optuna-examples/pull/224)
• Add OptunaSearchCV with terminator (https://github.com/optuna/optuna-examples/pull/225)
• Drop python 3.8 from haiku test (https://github.com/optuna/optuna-examples/pull/227)
• Run MXNet in Python 3.11 (https://github.com/optuna/optuna-examples/pull/228)

Tests

• Remove tests for allennlp and chainer (https://github.com/optuna/optuna-integration/pull/47)
• Reduce the warning in tests/study_tests/test_study.py (#5070, thanks @sousu4!)

Code Fixes

• Implement NSGA-III elite population selection strategy (#5027)
• Fix import path of PyTorchLightning (#5028)
• Fix Any with float in _TreeNode.children (#5040, thanks @aanghelidi!)
• Fix future annotation in typing.py (#5054, thanks @jot-s-bindra!)
• Add future annotations to callback and terminator files inside terminator folder (#5055, thanks @jot-s-bindra!)
• Fix future annotations to edf python file (#5056, thanks @Vaibhav101203!)
• Fix future annotations in hypervolumehistory.py (#5057, thanks @Vaibhav101203!)
• Reduce the warning in tests/storages_tests/test_heartbeat.py (#5066, thanks @sousu4!)
• Fix future annotation to frozen.py (#5080, thanks @Vaibhav101203!)
• Fix annotation for dataframe.py (#5081, thanks @Vaibhav101203!)
• Fix future annotation (#5083, thanks @Vaibhav101203!)
• Fix type annotation (#5105)
• Fix mypy error in CI (#5106)
• Isolate the fast.ai module (#5120, thanks @sousu4!)
• Clean up workflow file (#5122)

Continuous Integration

• Run test_tensorflow in Python 3.11 (https://github.com/optuna/optuna-integration/pull/46)
• Exclude mypy checks for chainer (https://github.com/optuna/optuna-integration/pull/48)
• Support Python 3.12 on tests for core modules (#5018)
• Fix the issue where formats.sh does not handle tutorial/ (#5023, thanks @sousu4!)
• Skip slow integration tests (#5033)
• Install PyTorch for CPU on CIs (#5042)
• Remove unused type: ignore (#5047)
• Reduce tests-mpi to the oldest and latest Python versions (#5067)
• Add workflow matrices for the tests to reduce GitHub check runtime (#5093)
• Remove the skip of Python 3.11 in tests-mpi (#5100)
• Downgrade kaleido to 0.1.0post1 for fixing Windows CI (#5101)
• Rename should-skip to test-trigger-type for more clarity (#5134)
• Pin the version of PyQt6-Qt6 (#5135)
• Revert Pin the version of PyQt6-Qt6 (#5140)

Other

• Bump up version to v3.5.0.dev (https://github.com/optuna/optuna-integration/pull/43)
• Bump up version number to 3.5.0 (https://github.com/optuna/optuna-integration/pull/52)
• Bump the version up to v3.5.0.dev (#5032)
• Remove email of authors (#5078)
• Update the integration sections in README.md (#5108)
• Pin mypy version to 1.6.* (#5123)
• Remove !examples from .dockerignore (#5129)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @Ashhar-24, @Guillaume227, @HideakiImamura, @JustinGoheen, @Vaibhav101203, @aanghelidi, @adjeiv, @c-bata, @contramundum53, @eukaryo, @felix-cw, @gen740, @jot-s-bindra, @keisuke-umezawa, @knshnb, @nabenabe0928, @not522, @nzw0301, @p1kit, @sousu4, @toshihikoyanase, @y-kamiya

- Python
Published by HideakiImamura over 2 years ago

optuna - v3.4.0

This is the release note of v3.4.0.

Highlights

Optuna 3.4 newly supports the following new features. See our release blog for more detailed information.

• Preferential Optimization (Optuna Dashboard)
• Optuna Artifact
• Jupyter Lab Extension
• VS Code Extension
• User-defined Distance for Categorical Parameters in TPE
• Constrained Optimization Support for Visualization Functions
• User-Defined Plotly’s Figure Support (Optuna Dashboard)
• 3D Model Viewer Support (Optuna Dashboard)

Breaking Changes

• Remove deprecated arguments with regard to LightGBM>=4.0 (#4844)
• Deprecate SkoptSampler (#4913)

New Features

• Support constraints for intermediate values plot (#4851, thanks @adjeiv!)
• Display all objectives on hyperparameter importances plot (#4871)
• Implement get_all_study_names() (#4898)
• Support constraints plot_rank (#4899, thanks @ryota717!)
• Support Study Artifacts (#4905)
• Support specifying distance between categorical choices in TPESampler (#4926)
• Add metric_names getter to study (#4930)
• Add artifact middleware for exponential backoff retries (#4956)
• Add GCSArtifactStore (#4967, thanks @semiexp!)
• Add BestValueStagnationEvaluator (#4974, thanks @smygw72!)
• Allow user-defined objective names in hyperparameter importance plots (#4986)

Enhancements

• CHG constrained param displayed in #cccccc (#4877, thanks @louis-she!)
• Faster implementation of fANOVA (#4897)
• Support constraint in plot slice (#4906, thanks @hrntsm!)
• Add mimetype input (#4910, thanks @hrntsm!)
• Show all ticks in _parallel_coordinate.py when log scale (#4911)
• Speed up multi-objective TPE (#5017)

Bug Fixes

• Fix numpy indexing bugs and named tuple comparing (#4874, thanks @ryota717!)
• Fix fail_stale_trials with race condition (#4886)
• Fix alias handler (#4887)
• Add lazy random state and use it in RandomSampler (#4970, thanks @shu65!)
• Fix TensorBoard error on categorical choices of mixed types (#4973, thanks @ciffelia!)
• Use lazy random state in samplers (#4976, thanks @shu65!)
• Fix an error that does not consider min_child_samples (#5007)
• Fix BruteForceSampler in parallel optimization (#5022)

Documentation

• Fix typo in _filesystem.py (#4909)
• Mention a pruner instance is not stored in a storage in resuming tutorial (#4927)
• Add introduction of optuna-fast-fanova in documents (#4943)
• Add artifact tutorial (#4954)
• Fix an example code in Boto3ArtifactStore's docstring (#4957)
• Add tutorial for JournalStorage (#4980, thanks @semiexp!)
• Fix document regarding ArtifactNotFound (#4982, thanks @smygw72!)
• Add the workaround for duplicated samples to FAQ (#5006)

Examples

• Add huggingface's link to external projects (https://github.com/optuna/optuna-examples/pull/201)
• Fix samplers CI (https://github.com/optuna/optuna-examples/pull/202)
• Set version constraint on aim (https://github.com/optuna/optuna-examples/pull/206)
• Add an example of Optuna Terminator for LightGBM (https://github.com/optuna/optuna-examples/pull/210, thanks @hamster-86!)

Tests

• Reduce n_trials in test_combination_of_different_distributions_objective (#4950)
• Replaces California housing dataset with iris dataset (#4953)
• Fix numpy duplication warning (#4978, thanks @torotoki!)
• Make test order deterministic for pytest-xdist (#4999)

Code Fixes

• Move shap (https://github.com/optuna/optuna-integration/pull/32)
• Remove shap (#4791)
• Use isinstance instead of if type() is ... (#4896)
• Make cmaes dependency optional (#4901)
• Call internal sampler's before_trial (#4914)
• Refactor _grid.py (#4918)
• Fix the checks-integration errors on LightGBMTuner (#4923)
• Replace deprecated botorch method to remove warning (#4940)
• Fix type annotation (#4941)
• Add _split_trials instead of _get_observation_pairs and _split_observation_pairs (#4947)
• Use __future__.annotations in optuna/visualization/_optimization_history.py (#4964, thanks @YuigaWada!)
• Fix #4508 for optuna/visualization/_hypervolume_history.py (#4965, thanks @RuTiO2le!)
• Use future annotation in optuna/_convert_positional_args.py (#4966, thanks @hamster-86!)
• Fix type annotation of SQLAlchemy (#4968)
• Use collections.abc in optuna/visualization/_edf.py (#4969, thanks @g-tamaki!)
• Use collections.abc in plot pareto front (#4971)
• Remove experimental_func from metric_names property (#4983, thanks @semiexp!)
• Add __future__.annotations to progress_bar.py (#4992)
• Fix annotations in optuna/optuna/visualization/matplotlib/_optimization_history.py (#5015, thanks @sousu4!)

Continuous Integration

• Fix checks integration (#4869)
• Remove fakeredis version constraint (#4873)
• Support asv 0.6.0 (#4882)
• Fix speed-benchmarks CI (#4903)
• Fix Tests (MPI) CI (#4904)
• Fix xgboost pruning callback (#4921)
• Enhance speed benchmark (#4981, thanks @g-tamaki!)
• Drop Python 3.7 on tests-mpi (#4998)
• Remove Python 3.7 from the development docker image build (#5009)
• Use CPU version of PyTorch in Docker image (#5019)

Other

• Bump up version number to v3.4.0.dev (https://github.com/optuna/optuna-integration/pull/37)
• Update python shield in README.md (https://github.com/optuna/optuna-integration/pull/39)
• Replace deprecated mypy option (https://github.com/optuna/optuna-integration/pull/40)
• Bump up version to v3.4.0 (https://github.com/optuna/optuna-integration/pull/42)
• Bump the version up to v3.4.0.dev (#4861)
• Use OIDC (#4867)
• Add FUNDING.yml (#4912)
• Update optional-dependencies and document deselecting integration tests in CONTRIBUTING.md (#4962)
• Bump the version up to v3.4.0 (#5031)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @HideakiImamura, @RuTiO2le, @YuigaWada, @adjeiv, @c-bata, @ciffelia, @contramundum53, @cross32768, @eukaryo, @g-tamaki, @g-votte, @gen740, @hamster-86, @hrntsm, @hvy, @keisuke-umezawa, @knshnb, @lucasmrdt, @louis-she, @moririn2528, @nabenabe0928, @not522, @nzw0301, @ryota717, @semiexp, @shu65, @smygw72, @sousu4, @torotoki, @toshihikoyanase, @xadrianzetx

- Python
Published by contramundum53 over 2 years ago

optuna - v3.3.0

This is the release note of v3.3.0.

Highlights

CMA-ES with Learning Rate Adaptation

A new variant of CMA-ES has been added. By setting the lr_adapt argument to True in CmaEsSampler, you can utilize it. For multimodal and/or noisy problems, adapting the learning rate can help avoid getting trapped in local optima. For more details, please refer to #4817. We want to thank @nomuramasahir0, one of the authors of LRA-CMA-ES, for his great work and the development of cmaes library.

Hypervolume History Plot for Multiobjective Optimization

In multiobjective optimization, the history of hypervolume is commonly used as an indicator of performance. Optuna now supports this feature in the visualization module. Thanks to @y0z for your great work!

Constrained Optimization Support for Visualization Functions

| Plotly | matplotlib | | --- | --- | | | |

Some samplers support constrained optimization, however, many other features cannot handle it. We are continuously enhancing support for constraints. In this release, plot_optimization_history starts to consider constraint violations. Thanks to @hrntsm for your great work!

```python import optuna

def objective(trial): x = trial.suggestfloat("x", -15, 30) y = trial.suggestfloat("y", -15, 30) v0 = 4 * x2 + 4 * y2 trial.setuserattr("constraint", [1000 - v0]) return v0

def constraintsfunc(trial): return trial.userattrs["constraint"]

sampler = optuna.samplers.TPESampler(constraintsfunc=constraintsfunc) study = optuna.createstudy(sampler=sampler) study.optimize(objective, ntrials=100) fig = optuna.visualization.plotoptimizationhistory(study) fig.show() ```

Streamlit Integration for Human-in-the-loop Optimization

Optuna Dashboard v0.11.0 provides the tight integration with Streamlit framework. By using this feature, you can create your own application for human-in-the-loop optimization. Please check out the documentation and the example for details.

Breaking Changes

• Move mxnet (https://github.com/optuna/optuna-integration/pull/31)
• Remove mxnet (#4790)
• Remove ordered_dict argument from IntersectionSearchSpace (#4846)

New Features

• Add logei_candidate_func and make it default when available (#4667)
• Support JournalFileStorage and JournalRedisStorage on CLI (#4696)
• Implement hypervolume history plot for matplotlib backend (#4748, thanks @y0z!)
• Add cv_results_ to OptunaSearchCV (#4751, thanks @jckkvs!)
• Add optuna.integration.botorch.qnei_candidates_func (#4753, thanks @kstoneriv3!)
• Add hypervolume history plot for plotly backend (#4757, thanks @y0z!)
• Add FileSystemArtifactStore (#4763)
• Sort params on fetch (#4775)
• Add constraints support to _optimization_history_plot (#4793, thanks @hrntsm!)
• Bump up LightGBM version to v4.0.0 (#4810)
• Add constraints support to matplotlib._optimization_history_plot (#4816, thanks @hrntsm!)
• Introduce CMA-ES with Learning Rate Adaptation (#4817)
• Add upload_artifact api (#4823)
• Add before_trial (#4825)
• Add Boto3ArtifactStore (#4840)
• Display best objective value in contour plot for a given param pair, not the value from the most recent trial (#4848)

Enhancements

• Speed up logpdf in _truncnorm.py (#4712)
• Speed up erf (#4713)
• Speed up get_all_trials in InMemoryStorage (#4716)
• Add a warning for a progress bar not being displayed #4679 (#4728, thanks @rishabsinghh!)
• Make BruteForceSampler consider failed trials (#4747)
• Use shallow copy in _get_latest_trial (#4774)
• Speed up plot_hypervolume_history (#4776)

Bug Fixes

• Solve issue #4557 - error_score (#4642, thanks @jckkvs!)
• Fix BruteForceSampler for pruned trials (#4720)
• Fix plot_slice bug when some of the choices are numeric (#4724)
• Make LightGBMTuner reproducible (#4795)

Installation

• Bump up python version (https://github.com/optuna/optuna-integration/pull/34)

Documentation

• Remove jquery-extension (#4691)
• Add FAQ on combinatorial search space (#4723)
• Fix docs (#4732)
• Add plot_rank and plot_timeline plots to visualization tutorial (#4735)
• Fix typos found in integration/sklearn.py (#4745)
• Remove study.n_objectives from document (#4796)
• Add lower version constraint for sphinx_rtd_theme (#4853)
• Artifact docs (#4855)

Examples

• Run DaskML example with Python 3.11 (https://github.com/optuna/optuna-examples/pull/188)
• Show more information in terminator examples (https://github.com/optuna/optuna-examples/pull/192)
• Drop support for Python 3.7 on Haiku (https://github.com/optuna/optuna-examples/pull/198)
• Add LICENSE file (https://github.com/optuna/optuna-examples/pull/200)

Tests

• Remove unnecessary pytestmark (https://github.com/optuna/optuna-integration/pull/29)
• Add GridSampler test for failed trials (#4721)
• Follow up PR #4642 by adding a unit test to confirm OptunaSearchCV behavior (#4758)
• Fix test_log_gass_mass with SciPy 1.11.0 (#4766)
• Fix Pytorch lightning unit test (#4780)
• Remove skopt (#4792)
• Rename test directory (#4839)

Code Fixes

• Simplify the type annotations in benchmarks (#4703, thanks @caprest!)
• Unify sampling implementation in TPESampler (#4717)
• Get values after _get_observation_pairs (#4742)
• Remove unnecessary period (#4746)
• Handle deprecated argument early_stopping_rounds (#4752)
• Separate dominate function from _fast_non_dominated_sort() (#4759)
• Separate after_trial strategy (#4760)
• Remove unused attributes in TPESampler (#4769)
• Remove pkg_resources (#4770)
• Use trials as argument of _calculate_weights_below_for_multi_objective (#4773)
• Fix type annotation (#4797, thanks @taniokay!)
• Follow up separation of after trial strategy (#4803)
• Loose coupling nsgaii child generation (#4806)
• Remove _study_id parameter from Trial class (#4811, thanks @adjeiv!)
• Loose coupling nsgaii elite population selection (#4821)
• Fix checks integration (#4826)
• Remove OrderedDict (#4838, thanks @taniokay!)
• Fix typo (#4842, thanks @wouterzwerink!)
• Followup child generation strategy (#4856)
• Remove samplers._search_space.IntersectionSearchSpace (#4857)
• Add experimental decorators to artifacts functionalities (#4858)

Continuous Integration

• Output dependency tree (https://github.com/optuna/optuna-integration/pull/9)
• Use OIDC (https://github.com/optuna/optuna-integration/pull/33)
• Drop Python 3.7 support (https://github.com/optuna/optuna-integration/pull/35)
• Enhance speed benchmark for storages (#4778)
• Drop Python 3.7 on tests-integration (#4784)
• Remove unused type:ignores (#4787)
• Restrict numpy version < 1.24 (#4788)
• Upgrade redis version (#4805)
• Add version constraints on LightGBM (#4807)
• Follow-up #4807 : Fix windows-tests and mac-tests (#4809)
• Support 3.11 integration (#4820)
• Support flake8 6.1.0 (#4847)

Other

• Bump up version number to 3.3.0dev (https://github.com/optuna/optuna-integration/pull/27)
• Bump up version number to 3.3.0 (https://github.com/optuna/optuna-integration/pull/36)
• Bump up version number to 3.3.0dev (#4710)
• Bump the version up to v3.3.0 (#4860)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @HideakiImamura, @adjeiv, @c-bata, @caprest, @contramundum53, @cross32768, @eukaryo, @gen740, @hrntsm, @jckkvs, @knshnb, @kstoneriv3, @nomuramasahir0, @not522, @nzw0301, @rishabsinghh, @taniokay, @toshihikoyanase, @wouterzwerink, @xadrianzetx, @y0z

- Python
Published by c-bata almost 3 years ago

optuna - v3.2.0

This is the release note of v3.2.0.

Highlights

Human-in-the-loop optimization

With the latest release, we have incorporated support for human-in-the-loop optimization. It enables an interactive optimization process between users and the optimization algorithm. As a result, it opens up new opportunities for the application of Optuna in tuning Generative AI. For further details, please check out our human-in-the-loop optimization tutorial.

Overview of human-in-the-loop optimization. Generated images and sounds are displayed on Optuna Dashboard, and users can directly evaluate them there.

Automatic optimization terminator(Optuna Terminator)

Optuna Terminator is a new feature that quantitatively estimates room for optimization and automatically stops the optimization process. It is designed to alleviate the burden of figuring out an appropriate value for the number of trials (n_trials), or unnecessarily consuming computational resources by indefinitely running the optimization loop. See #4398 and optuna-examples#190.

Transition of estimated room for improvement. It steadily decreases towards the level of cross-validation errors.

New sampling algorithms

NSGA-III for many-objective optimization

We've introduced the NSGAIIISampler as a new multi-objective optimization sampler. It implements NSGA-III, which is an extended variant of NSGA-II, designed to efficiently optimize even when the dimensionality of the objective values is large (especially when it's four or more). NSGA-II had an issue where the search would become biased towards specific regions when the dimensionality of the objective values exceeded four. In NSGA-III, the algorithm is designed to distribute the points more uniformly. This feature was introduced by #4436.

Objective value space for multi-objective optimization (minimization problem). Red points represent Pareto solutions found by NSGA-II. Blue points represent those found by NSGA-III. NSGA-II shows a tendency for points to concentrate towards each axis (corresponding to the ends of the Pareto Front). On the other hand, NSGA-III displays a wider distribution across the Pareto Front.

BI-population CMA-ES

Continuing from v3.1, significant improvements have been made to the CMA-ES Sampler. As a new feature, we've added the BI-population CMA-ES algorithm, a kind of restart strategy that mitigates the problem of falling into local optima. Whether the IPOP CMA-ES, which we've been providing so far, or the new BI-population CMA-ES is better depends on the problems. If you're struggling with local optima, please try BI-population CMA-ES as well. For more details, please see #4464.

New visualization functions

Timeline plot for trial life cycle

The timeline plot visualizes the progress (status, start and end times) of each trial. In this plot, the horizontal axis represents time, and trials are plotted in the vertical direction. Each trial is represented as a horizontal bar, drawn from the start to the end of the trial. With this plot, you can quickly get an understanding of the overall progress of the optimization experiment, such as whether parallel optimization is progressing properly or if there are any trials taking an unusually long time.

Similar to other plot functions, all you need to do is pass the study object to plot_timeline. For more details, please refer to #4470 and #4538.

Rank plot to understand input-output relationship

A new visualization feature, plot_rank, has been introduced. This plot provides valuable insights into landscapes of objective functions, i.e., relationship between parameters and objective values. In this plot, the vertical and horizontal axes represent the parameter values, and each point represents a single trial. The points are colored according to their ranks.

Similar to other plot functions, all you need to do is pass the study object to plot_rank. For more details, please refer to #4427 and #4541.

Isolating integration modules

We have separated Optuna's integration module into a different package called optuna-integration. Maintaining many integrations within the Optuna package was becoming costly. By separating the integration module, we aim to improve the development speed of both Optuna itself and its integration module. As of the release of v3.2, we have migrated six integration modules: allennlp, catalyst, chainer, keras, skorch, and tensorflow (excepting for the TensorBoard integration). To use integration module, pip install optuna-integration will be necessary. See #4484.

• Move chainermn integration (https://github.com/optuna/optuna-integration/pull/1)
• Move integration/keras.py (https://github.com/optuna/optuna-integration/pull/5)
• Move integration/allennlp (https://github.com/optuna/optuna-integration/pull/8)
• Move Catalyst (https://github.com/optuna/optuna-integration/pull/19)
• Move tf.keras integration (https://github.com/optuna/optuna-integration/pull/21)
• Move skorch (https://github.com/optuna/optuna-integration/pull/22)
• Move tensorflow integration (https://github.com/optuna/optuna-integration/pull/23)
• Partially follow sklearn.model_selection.GridSearchCV's arguments (#4336)
• Delete optuna.integration.ChainerPruningExtension for migrating to optuna-integration package (#4370)
• Delete optuna.integration.ChainerMNStudy for migrating to optuna-integration package (#4497)
• Delete optuna.integration.KerasPruningCallback for migration to optuna-integration (#4558)
• Delete AllenNLP integration for migration to optuna-integration (#4579)
• DeleteCatalyst integration for migration to optuna-integration (#4644)
• Remove tf.keras integration (#4662)
• Delete skorch integration for migration to optuna-integration (#4663)
• Remove tensorflow integration (#4666)

Starting support for Mac & Windows

We have started supporting Optuna on Mac and Windows. While many features already worked in previous versions, we have fixed issues that arose in certain modules, such as Storage. See #4457 and #4458.

Breaking Changes

• Update deletion timing of system_attrs and set_system_attr (https://github.com/optuna/optuna-integration/pull/4)
• Change deletion timing of system_attrs and set_system_attr (#4550)

New Features

• Show custom objective names for multi-objective optimization (#4383)
• Support DDP in PyTorch-Lightning (#4384)
• Implement the evaluator of regret bounds and its GP backend for Optuna Terminator 🤖 (#4401)
• Implement the termination logic and APIs of Optuna Terminator 🤖 (#4405)
• Add rank plot (#4427)
• Implement NSGA-III (#4436)
• Add BIPOP-CMA-ES support in CmaEsSampler (#4464)
• Add timeline plot with plotly as backend (#4470)
• Move optuna.samplers._search_space.intersection.py to optuna.search_space.intersection.py (#4505)
• Add timeline plot with matplotlib as backend (#4538)
• Add rank plot matplotlib version (#4541)
• Support batched sampling with BoTorch (#4591, thanks @kstoneriv3!)
• Add plot_terminator_improvement as visualization of optuna.terminator (#4609)
• Add import for public API of optuna.terminator to optuna/terminator/__init__.py (#4669)
• Add matplotlib version of plot_terminator_improvement (#4701)

Enhancements

• Import cmaes package lazily (#4394)
• Make BruteForceSampler stateless (#4408)
• Sort studies by study_id (#4414)
• Add index study_id column on trials table (#4449, thanks @Ilevk!)
• Cache all trials in Study with delayed relative sampling (#4468)
• Avoid error at import time for optuna.terminator.improvement.gp.botorch (#4483)
• Avoid standardizing Yvar in _BoTorchGaussianProcess (#4488)
• Change the noise value in _BoTorchGaussianProcess to suppress warning messages (#4510)
• Change the argument of intersection_search_space from study to trials (#4514)
• Improve deprecated messages in the old suggest functions (#4562)
• Add support for distributed>=2023.3.2 (#4589, thanks @jrbourbeau!)
• Fix plot_rank marker lines (#4602)
• Sync owned trials when calling study.ask and study.get_trials (#4631)
• Ensure that the plotly version of timeline plot draws a legend even if all TrialStates are the same (#4635)

Bug Fixes

• Fix botorch dependency (#4368)
• Mitigate a blocking issue while running migrations with SQLAlchemy 2.0 (#4386)
• Fix colorlog compatibility problem (#4406)
• Validate length of values in add_trial (#4416)
• Fix RDBStorage.get_best_trial when there are infs (#4422)
• Fix bug of CMA-ES with margin on RDBStorage or JournalStorage (#4434)
• Fix CMA-ES Sampler (#4443)
• Fix param_mask for multivariate TPE with constant_liar (#4462)
• Make QMCSampler samplers reproducible with seed=0 (#4480)
• Fix noise becoming NaN for the terminator module (#4512)
• Fix metric_names on _log_completed_trial() function (#4594)
• Fix ImportError for botorch<=0.4.0 (#4626)
• Fix index of n_retries += 1 in RDBStorage (#4658)
• Fix CMA-ES with margin bug (#4661)
• Fix a logic for invalidating the cache in CachedStorage (#4670)
• Fix #4697 ValueError: Rank 0 node expects an optuna.trial.Trial instance as the trial argument (#4698, thanks @keisukefukuda!)
• Fix a bug reported in issue #4699 (#4700)
• Add tests for plot_terminator_improvement and fix some bugs (#4702)

Installation

• Remove codecov dependencies (https://github.com/optuna/optuna-integration/pull/13)
• Migration to pyproject.toml for packaging (#4164)
• [RFC] Remove specific pytorch version to support the latest stable PyTorch (#4585)

Documentation

• Create the document and run the test to create document in each PR (https://github.com/optuna/optuna-integration/pull/2)
• Fix Keras docs (https://github.com/optuna/optuna-integration/pull/12)
• Add links of documents (https://github.com/optuna/optuna-integration/pull/17)
• Load sphinxcontrib.jquery explicitly (https://github.com/optuna/optuna-integration/pull/18)
• Add docstring for the Terminator class (#4596)
• Fix the build on Read the Docs by following optuna #4659 (https://github.com/optuna/optuna-integration/pull/20)
• Add external packages to intersphinx_mapping in conf.py (#4290)
• Minor fix of documents (#4360)
• Fix a typo in MeanDecreaseImpurityImportanceEvaluator (#4385)
• Update to Sphinx 6 (#4479)
• Fix URL to the line of optuna-integration file (#4498)
• Fix typo (#4515, thanks @gituser789!)
• Resolve error in compiling PDF documents (#4605)
• Add sphinxcontrib.jquery extension to conf.py (#4615)
• Remove an example code of SkoptSampler (#4625)
• Add links to the optuna-integration document (#4638)
• Add manually written index page of tutorial (#4640)
• Fix the build on Read the Docs (#4659)
• Improve docstring of rank_plot function and its matplotlib version (#4660)
• Add a link to tutorial of human-in-the-loop optimization (#4665)
• Fix typo for progress bar in documentation (#4673, thanks @gituser789!)
• Add docstrings to optuna.termintor (#4675)
• Add docstring for plot_terminator_improvement (#4677)
• Remove versionadded directives (#4681)
• Add pareto front display example: 2D-plot from 3D-optimization including crop the scale (#4685, thanks @gituser789!)
• Embed a YouTube video in the docstring of DaskStorage (#4694)
• List Dashboard in navbar (#4708)
• Fix docstring of terminator improvement for min_n_trials (#4709)

Examples

• An example of using pytorch distributed data parallel on 1 machine with arbitrary multiple GPUs (https://github.com/optuna/optuna-examples/pull/155, thanks @li-li-github!)
• Apply black . with black 23.1.0 (https://github.com/optuna/optuna-examples/pull/168)
• Add Aim example (https://github.com/optuna/optuna-examples/pull/170)
• Resolve todo and fix docstrings in fastaiv2 example (https://github.com/optuna/optuna-examples/pull/171)
• Update pytorch-lightning version (https://github.com/optuna/optuna-examples/pull/172)
• Add python 3.11 to ray's version matrix (https://github.com/optuna/optuna-examples/pull/174)
• Minor code change suggestions to pytorch_distributed_spawn.py (https://github.com/optuna/optuna-examples/pull/175)
• Install optuna-integration in chainer CI (https://github.com/optuna/optuna-examples/pull/176)
• Add python 3.11 skimage's version matrix and remove warning for inputs data (https://github.com/optuna/optuna-examples/pull/177)
• Execute Ray example in CI (https://github.com/optuna/optuna-examples/pull/178)
• Update pytorch lightning version for ddp (https://github.com/optuna/optuna-examples/pull/179)
• Don't run evaluation twice on the last epoch (https://github.com/optuna/optuna-examples/pull/181, thanks @Jendker!)
• Use BoTorch 0.8 or higher (https://github.com/optuna/optuna-examples/pull/185)
• Run catboost example with python 3.11 (https://github.com/optuna/optuna-examples/pull/186)
• Add terminator examples (https://github.com/optuna/optuna-examples/pull/190)
• Use Gymnasium and pre-released Stable-Baselines3 (https://github.com/optuna/optuna-examples/pull/191)
• Fix the AllenNLP CI (https://github.com/optuna/optuna-examples/pull/193)

Tests

• Suppress FutureWarning about Trial.set_system_attr in storage tests (#4323)
• Add test for casting in test_nsgaii.py (#4387)
• Fix the blocking issue on test_with_server.py (#4402)
• Fix mypy error about Chainer (#4410)
• Add unit tests for the _BoTorchGaussianProcess class (#4441)
• Implement unit tests for optuna.terminator.improvement._preprocessing.py (#4506)
• Fix mypy error about PyTorch Lightning (#4520)

Code Fixes

• Simplify type annotations (https://github.com/optuna/optuna-integration/pull/10)
• Copy _imports.py from optuna (https://github.com/optuna/optuna-integration/pull/16)
• Refactor ParzenEstimator (#4183)
• Fix mypy error abut AllenNLP in Checks (integration) (#4277)
• Fix checks integration about pytorch lightning (#4322)
• Minor refactoring of tests/hypervolume_tests/test_hssp.py (#4329)
• Remove unnecessary sklearn version condition (#4379)
• Support black 23.1.0 (#4382)
• Warn unexpected search spaces for CmaEsSampler (#4395)
• Fix flake8 errors on sklearn integration (#4407)
• Fix mypy error about PyTorch Distributed (#4413)
• Use numpy.polynomial in _erf.py (#4415)
• Refactor _ParzenEstimator (#4433)
• Simplify an argument's name of RegretBoundEvaluator (#4442)
• Fix Checks(integration) about terminator/.../botorch.py (#4461)
• Add an experimental decorator to RegretBoundEvaluator (#4469)
• Add JSON serializable type (#4478)
• Move optuna.samplers._search_space.group_decomposed.py to optuna.search_space.group_decomposed.py (#4491)
• Simplify annotations in optuna.visualization (#4525, thanks @harupy!)
• Simplify annotations in tests.visualization_tests (#4526, thanks @harupy!)
• Remove unused instance variables in _BoTorchGaussianProcess (#4530)
• Avoid deepcopy in optuna.visualization.plot_timeline (#4540)
• Use SingleTaskGP for Optuna terminator (#4542)
• Change deletion timing of optuna.samplers.IntersectionSearchSpace and optuna.samplers.intersection_search_space (#4549)
• Remove IntersectionSearchSpace in optuna.terminator module (#4595)
• Change arguments of BaseErrorEvaluator and classes that inherit from it (#4607)
• Delete import Rectangle in visualization/matplotlib (#4620)
• Simplify type annotations in visualize/_rank.py and visualization_tests/ (#4628)
• Move the function _distribution_is_log to optuna.distributionsP fromoptuna/terminator/init.py` (#4668)
• Separate _fast_non_dominated_sort() from the samplers (#4671)
• Read trials from remote storage whenever get_all_trials of _CachedStorage is called (#4672)
• Remove experimental label from _ProgressBar (#4684, thanks @tungbq!)

Continuous Integration

• Fix coverage.yml (https://github.com/optuna/optuna-integration/pull/3)
• Delete labeler.yaml (https://github.com/optuna/optuna-integration/pull/6)
• Fix pypi publish.yaml (https://github.com/optuna/optuna-integration/pull/11)
• Test on an arbitrary branch (https://github.com/optuna/optuna-integration/pull/15)
• Fix the CI with AllenNLP (https://github.com/optuna/optuna-integration/pull/24)
• Update actions/setup-python@v2 -> v4 (#4307, thanks @Kaushik-Iyer!)
• Update action versions (#4328)
• Update actions/setup-python in mac-tests (follow-up for #4307) (#4343)
• Add type ignore to ProcessGroup import from torch.distributed (#4347)
• Fix label of pypigh-action-pypi-publish (#4359)
• [Hotfix] Avoid to install SQLAlchemy 2.0 on checks (#4364)
• [Hotfix] Add version constriant on SQLAlchemy for tests storage with server (#4372)
• Disable colored log when NO_COLOR env or not tty (#4376)
• Output installed packages in Tests CI (#4381)
• Output installed packages in mac-test CI (#4397)
• Use ubuntu-latest in PyPI publish CI (#4400)
• Output installed packages in Checks CI (#4417, thanks @Kaushik-Iyer!)
• Output installed packages in Coverage CI (#4423, thanks @Kaushik-Iyer!)
• Fix mypy error on checks-integration CI (#4424)
• Fix mac-test cache path (#4425)
• Add minimum version tests of numpy, tqdm, colorlog, PyYAML (#4428)
• Remove ignore testpytorchlightning (#4432)
• Use PyYAML==5.1 on tests-with-minimum-dependencies (#4435)
• Remove trailing spaces in CI configs (#4439)
• Output installed packages in all remaining CIs (#4445, thanks @Kaushik-Iyer!)
• Add windows ci check (#4457)
• Make mac-test executed on PRs (#4458)
• Add sqlalchemy<2.0.0 in Checks(integration) (#4482)
• Fix ci test conditions (#4496)
• Deploy results of visual regression test on Netlify (#4507)
• Pin pytorch lightning version (#4522)
• Securely deploy results of visual regression test on Netlify (#4532)
• Pin Distributed version (#4545)
• Delete fragile heartbeat test (#4551)
• Ignore AllenNLP test from Mac-CI (#4561)
• Delete visual-regression.yml (#4597)
• Remove dependency on codecov (#4606)
• Install test in checks-integration CI (#4612)
• Fix checks integration (#4617)
• Add Output dependency tree by pipdeptree to Actions (#4624)
• Add a version constraint on fakeredis (#4637)
• Hotfix and run catboost test w/ python 3.11 except for MacOS (#4646)
• Run mlflow with Python 3.11 (#4647)

Other

• Update repository settings as in optuna/optuna (https://github.com/optuna/optuna-integration/pull/7)
• Bump up version to v3.2.0.dev (#4345)
• Remove cached-path from setup.py (#4357)
• Revert a merge commit for #4183 (#4429)
• Include both venv and .venv in the exclude setting of the formatters (#4476)
• Replace hacking with flake8 (#4556)
• Fix Codecov link (#4564)
• Add lightning_logs to .gitignore (#4565)
• Fix targets of black and isort in formats.sh (#4610)
• Install benchmark, optional, and test in dev Docker image (#4611)
• Provide kind error massage for missing optuna-integration (#4636)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @HideakiImamura, @Ilevk, @Jendker, @Kaushik-Iyer, @amylase, @c-bata, @contramundum53, @cross32768, @eukaryo, @g-votte, @gen740, @gituser789, @harupy, @himkt, @hvy, @jrbourbeau, @keisuke-umezawa, @keisukefukuda, @knshnb, @kstoneriv3, @li-li-github, @nomuramasahir0, @not522, @nzw0301, @toshihikoyanase, @tungbq

- Python
Published by toshihikoyanase about 3 years ago

optuna - v3.1.1

This is the release note of v3.1.1.

Enhancements

• [Backport] Import cmaes package lazily (#4573)

Bug Fixes

• [Backport] Fix botorch dependency (#4569)
• [Backport] Fix parammask for multivariate TPE with constantliar (#4570)
• [Backport] Mitigate a blocking issue while running migrations with SQLAlchemy 2.0 (#4571)
• [Backport] Fix bug of CMA-ES with margin on RDBStorage or JournalStorage (#4572)
• [Backport] Fix RDBStorage.getbesttrial when there are infs (#4574)
• [Backport] Fix CMA-ES Sampler (#4581)

Code Fixes

• [Backport] Add types-tqdm for lint (#4566)

Other

• Update version number to v3.1.1 (#4567)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @contramundum53, @not522

- Python
Published by HideakiImamura about 3 years ago

optuna - v3.0.6

This is the release note of v3.0.6.

Installation

• Fix a project metadata for scipy version constraint (#4494)

Other

• Bump up version number to v3.0.6 (#4493)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@c-bata @HideakiImamura

- Python
Published by c-bata over 3 years ago

optuna - v3.1.0

This is the release note of v3.1.0.

This is not something you have to read from top to bottom to learn about the summary of Optuna v3.1. The recommended way is reading the release blog.

Highlights

New Features

CMA-ES with Margin

| CMA-ES | CMA-ES with Margin | | ------- | -------- | | | |

“The animation is referred from https://github.com/EvoConJP/CMA-ESwithMargin, which is distributed under the MIT license.”

CMA-ES achieves strong performance for continuous optimization, but there is still room for improvement in mixed-integer search spaces. To address this, we have added support for the "CMA-ES with Margin" algorithm to our CmaEsSampler, which makes it more efficient in these cases. You can see the benchmark results here. For more detailed information about CMA-ES with Margin, please refer to the paper “CMA-ES with Margin: Lower-Bounding Marginal Probability for Mixed-Integer Black-Box Optimization - arXiv”, which has been accepted for presentation at GECCO 2022.

```python import optuna from optuna.samplers import CmaEsSampler

def objective(trial): x = trial.suggestfloat("y", -10, 10, step=0.1) y = trial.suggestint("x", -100, 100) return x**2 + y

study = optuna.createstudy(sampler=CmaEsSampler(withmargin=True)) study.optimize(objective) ```

Distributed Optimization via NFS

JournalFileStorage, a file storage backend based on JournalStorage, supports NFS (Network File System) environments. It is the easiest option for users who wish to execute distributed optimization in environments where it is difficult to set up database servers such as MySQL, PostgreSQL or Redis (e.g. #815, #1330, #1457 and #2216).

```python import optuna from optuna.storages import JournalStorage, JournalFileStorage

def objective(trial): x = trial.suggestfloat("x", -100, 100) y = trial.suggestfloat("y", -100, 100) return x**2 + y

storage = JournalStorage(JournalFileStorage("./journal.log")) study = optuna.create_study(storage=storage) study.optimize(objective) ```

For more information on JournalFileStorage, see the blog post “Distributed Optimization via NFS Using Optuna’s New Operation-Based Logging Storage” written by @wattlebirdaz.

A Brand-New Redis Storage

We have replaced the Redis storage backend with a JournalStorage-based one. The experimental RedisStorage class has been removed in v3.1. The following example shows how to use the new JournalRedisStorage class.

```python import optuna from optuna.storages import JournalStorage, JournalRedisStorage

def objective(trial): …

storage = JournalStorage(JournalRedisStorage("redis://localhost:6379")) study = optuna.create_study(storage=storage) study.optimize(objective) ```

Dask.distributed Integration

DaskStorage, a new storage backend based on Dask.distributed, is supported. It allows you to leverage distributed capabilities in similar APIs with concurrent.futures. DaskStorage can be used with InMemoryStorage, so you don't need to set up a database server. Here's a code example showing how to use DaskStorage:

```python import optuna from optuna.storages import InMemoryStorage from optuna.integration import DaskStorage from distributed import Client, wait

def objective(trial): ...

with Client("192.168.1.8:8686") as client: study = optuna.createstudy(storage=DaskStorage(InMemoryStorage())) futures = [ client.submit(study.optimize, objective, ntrials=10, pure=False) for i in range(10) ] wait(futures) print(f"Best params: {study.best_params}") ```

Setting up a Dask cluster is easy: install dask and distributed, then run the dask scheduler and dask worker commands, as detailed in the Quick Start Guide in the Dask.distributed documentation.

```console $ pip install optuna dask distributed

$ dark scheduler INFO - Scheduler at: tcp://192.168.1.8:8686 INFO - Dashboard at: :8687 …

$ dask worker tcp://192.168.1.8:8686 $ dask worker tcp://192.168.1.8:8686 $ dask worker tcp://192.168.1.8:8686 ```

See the documentation for more information.

Brute-force Sampler

BruteForceSampler, a new sampler for brute-force search, tries all combinations of parameters. In contrast to GridSampler, it does not require passing the search space as an argument and works even with branches. This sampler constructs the search space with the define-by-run style, so it works by just adding sampler=optuna.samplers.BruteForceSampler().

```python import optuna

def objective(trial): c = trial.suggestcategorical("c", ["float", "int"]) if c == "float": return trial.suggestfloat("x", 1, 3, step=0.5) elif c == "int": a = trial.suggestint("a", 1, 3) b = trial.suggestint("b", a, 3) return a + b

study = optuna.create_study(sampler=optuna.samplers.BruteForceSampler()) study.optimize(objective) ```

Other Improvements

Bug Fix for TPE’s constant_liar Option

The constant_liar option of TPESampler is an option for the distributed optimization or batch optimization. It has been introduced in v2.8.0, but suffers from performance degradation in specific situations. In this release, we have detected the cause of the problem, and resolve it with fruitful performance verification. See #4073 for more details.

Make Scipy Dependency Optional

50% time of import optuna is consumed by SciPy-related modules. Also, it consumes 110MB of storage space, which is really problematic in environments with limited resources such as serverless computing. We decided to implement scientific functions on our own to make the SciPy dependency optional. Thanks to contributors' effort on performance optimization, our implementation is as fast as the code with SciPy although ours is written in pure Python. See #4105 for more information. Note that QMCSampler still depends on SciPy. If you use QMCSampler, please explicitly specify SciPy as your dependency.

The New UI for Optuna Dashboard

We are developing a new UI for Optuna Dashboard that is available as an opt-in feature from the beta release - simply launch the dashboard as usual and click the link to the new UI. Please try it out and share your thoughts with us.

console $ pip install "optuna-dashboard>=0.9.0b2"

Feedback Survey: The New UI for Optuna Dashboard

Change Supported Python Versions

We have changed the supported Python versions. Specifically, Python 3.6 has been removed from the supported versions and Python 3.11 has been added. See #3021 and #3964 for more details.

Breaking Changes

• Allow users to call study.optimize() in multiple threads (#4068)
• Use all trials in TPESampler even when multivariate=True (#4079)
• Drop Python 3.6 (#4150)
• Remove RedisStorage (#4156)
• Deprecate set_system_attr in Study and Trial (#4188)
• Add a directions arg to storage.create_new_study (#4189)
• Deprecate system_attrs in Study class (#4250)
• Deprecate Trial.system_attrs property method (#4264)
• Remove device argument of TorchDistributedTrial (#4266)

New Features

• Add Dask integration (#2023, thanks @jrbourbeau!)
• Add journal-style log storage (#3854)
• Support CMA-ES with margin in CmaEsSampler (#4016)
• Add journal redis storage (#4086)
• Add device argument to BoTorchSampler (#4101)
• Add the feature to JournalStorage of Redis backend to resume from a snapshot (#4102)
• TorchDistributedTrial uses group as parameter instead of device (#4106, thanks @reyoung!)
• Added user_attrs to print by Optuna studies in cli.py (#4129, thanks @gonzaload!)
• Add BruteForceSampler (#4132, thanks @semiexp!)
• Add __getstate__ and __setstate__ to RedisStorage (#4135, thanks @shu65!)
• Make journal redis storage picklable (#4139, thanks @shu65!)
• Support for qNoisyExpectedHypervolumeImprovement acquisition function from Botorch (Issue#4014) (#4186)
• Show best trial number and value in progress bar (#4205)

Enhancements

• Change the log message format for failed trials (#3857, thanks @erentknn!)
• Move default logic of get_trial_id_from_study_id_trial_number() method to BaseStorage (#3910)
• Fix the data migration script for v3 release (#4020)
• Convert search_space values of GridSampler explicitly (#4062)
• Add single exception catch to study optimize (#4098)
• Remove scipy dependencies from TPESampler (#4105)
• Add validation in enqueue_trial (#4126)
• Speed up tests/samplers_tests/test_nsgaii.py::test_fast_non_dominated_sort_with_constraints (#4128, thanks @mist714!)
• Add getstate and setstate to journal storage (#4130, thanks @shu65!)
• Support None in slice plot (#4133, thanks @belldandyxtq!)
• Add marker to matplotlib plot_intermediate_value (#4134, thanks @belldandyxtq!)
• Add overloads for type narrowing in suggest_categorical (#4143, thanks @ConnorBaker!)
• Cache study.directions to reduce the number of get_study_directions() calls (#4146)
• Add an in-memory cache in Trial class (#4240)
• Use CMAwM class even when there is no discrete params (#4289)
• Refer OPTUNA_STORAGE environment variable in Optuna CLI (#4299, thanks @Hakuyume!)
• Apply @overload to ChainerMNTrial and TorchDistributedTrial (Follow-up of #4143) (#4300)
• Make OPTUNA_STORAGE environment variable experimental (#4316)

Bug Fixes

• Fix infinite loop bug in TPESampler (#3953, thanks @gasin!)
• Fix GridSampler (#3957)
• Fix an import error of sqlalchemy.orm.declarative_base (#3967)
• Skip to add intermediate_value_type and value_type columns if exists (#4015)
• Fix duplicated sampling of SkoptSampler (#4023)
• Avoid parse errors of datetime.isoformat strings (#4025)
• Fix a concurrency bug of JournalStorage set_trial_state_values (#4033)
• Specify object type to numpy array init to avoid unintended str cast (#4035)
• Make TPESampler reproducible (#4056)
• Fix bugs in constant_liar option (#4073)
• Add a flush to JournalFileStorage.append_logs (#4076)
• Add a lock to MLflowCallback (#4097)
• Reject deprecated distributions in OptunaSearchCV (#4120)
• Stop using hash function in _get_bracket_id in HyperbandPruner (#4131, thanks @zaburo-ch!)
• Validation for the parameter enqueued in to_internal_repr of FloatDistribution and IntDistribution (#4137)
• Fix PartialFixedSampler to handle None correctly (#4147, thanks @halucinor!)
• Fix the bug of JournalFileStorage on Windows (#4151)
• Fix CmaEs system attribution key (#4184)
• Skip constraint check for running trial (#4275)
• Fix constrained optimization with TPESampler's constant_liar (#4325)
• Fix import of ProcessGroup from torch.distributed (#4344)

Installation

• Replace thop with fvcore (#3906)
• Use the latest stable scipy (#3959, thanks @gasin!)
• Remove GPyTorch version constraint (#3986)
• Make typing_extensions optional (#3990)
• Add version constraint on importlib-metadata (#4036)
• Add a version constraint of matplotlib (#4044)

Documentation

• Update cli tutorial (#3902)
• Replace thop with fvcore (#3906)
• Slightly improve docs of FrozenTrial (#3943)
• Refine docs in BaseStorage (#3948)
• Remove "Edit on GitHub" button from readthedocs (#3952)
• Mention restoring sampler in saving/resuming tutorial (#3992)
• Use log_loss instead of deprecated log since sklearn 1.1 (#3993)
• Fix script path in benchmarks/README.md (#4021)
• Ignore ConvergenceWarning in the ask-and-tell tutorial (#4032)
• Update docs to let users know the concurrency problem on SQLite3 (#4034)
• Fix the time complexity of NSGAIISampler (#4045)
• Fix sampler comparison table (#4082)
• Add BruteForceSampler in the samplers' list (#4152)
• Remove markup from NaN in FAQ (#4155)
• Remove the document of the multi_objective module (#4167)
• Fix a typo in QMCSampler (#4179)
• Introduce Optuna Dashboard in tutorial docs (#4226)
• Remove RedisStorage from docstring (#4232)
• Add the BruteForceSampler example to the document (#4244)
• Improve the document of BruteForceSampler (#4245)
• Fix an inline markup in distributed tutorial (#4247)
• Fix a typo in BruteForceSampler (#4267)
• Update FAQ (#4269)
• Fix a typo in XGBoostPruningCallback (#4270)
• Fix CMAEvolutionStrategy link in integration.PyCmaSampler document (#4284, thanks @hrntsm!)
• Resolve warnings by sphinx with nitpicky option and fix typos (#4287)
• Fix typos (#4291)
• Improve the document of JournalStorage (#4308, thanks @hrntsm!)
• Fix typo (#4332, thanks @Jasha10!)
• Fix docstring in optuna/integration/dask.py (#4333)
• Mention suggest_float in BruteForceSampler (#4334)
• Remove verbose_eval argument from lightgbm callback in tutorial pages (#4335)
• Use Sphinx 5 until sphinx_rtd_theme supports Sphinx 6 (#4341)

Examples

• Add Dask example (https://github.com/optuna/optuna-examples/pull/46, thanks @jrbourbeau!)
• Hotfix for botorch example (https://github.com/optuna/optuna-examples/pull/134)
• Replace thop with fvcore (https://github.com/optuna/optuna-examples/pull/136)
• Add Optuna-distributed to external projects (https://github.com/optuna/optuna-examples/pull/137)
• Remove the version constraint of GPyTorch (https://github.com/optuna/optuna-examples/pull/138)
• Fix a file path in CONTRIBUTING.md (https://github.com/optuna/optuna-examples/pull/139)
• Install scikit-learn instead of sklearn (https://github.com/optuna/optuna-examples/pull/141)
• Add constraint on tensorflow to <2.11.0 (https://github.com/optuna/optuna-examples/pull/146)
• Specify botorch version (https://github.com/optuna/optuna-examples/pull/151)
• Add an example of Optuna Dashboard (https://github.com/optuna/optuna-examples/pull/152)
• Pin numpy version to 1.23.x for mxnet examples (https://github.com/optuna/optuna-examples/pull/154)
• Use tensorflow 2.11 syntax to fix CI error (https://github.com/optuna/optuna-examples/pull/156)
• Add Python 3.11 to version-matrix except for integration (https://github.com/optuna/optuna-examples/pull/160)
• Use latest stable version of actions (https://github.com/optuna/optuna-examples/pull/161)
• Use Monitor to resolve stable_baselines3's warning (https://github.com/optuna/optuna-examples/pull/162)
• Add Python 3.11 to a part of CI jobs (https://github.com/optuna/optuna-examples/pull/163)

Tests

• Suppress warnings in tests/test_distributions.py (#3912)
• Suppress warnings and minor code fixes in tests/trial_tests (#3914)
• Reduce warning messages by tests/study_tests/ (#3915)
• Remove dynamic search space based objective from a parallel job test (#3916)
• Remove all warning messages from tests/integration_tests/test_sklearn.py (#3922)
• Remove out-of-range related warning messages from MLflowCallback and WeightsAndBiasesCallback (#3923)
• Ignore RuntimeWarning when nanmin and nanmax take an array only containing nan values from pruners_tests (#3924)
• Remove warning messages from test files for pytorch_distributed and chainermn modules (#3927)
• Remove warning messages from tests/integration_tests/test_lightgbm.py (#3944)
• Resolve warnings in tests/visualization_tests/test_contour.py (#3954)
• Reduced warning messages from tests/visualization_tests/test_slice.py (#3970, thanks @jmsykes83!)
• Remove warning from a few visualization tests (#3989)
• Deselect integration tests in Tests CI (#4013)
• Remove warnings from tests/visualization_tests/test_optimization_history.py (#4024)
• Unset PYTHONHASHSEED for the hash-depedenet test (#4031)
• Call study.tell from another process (#4039, thanks @Abelarm!)
• Improve test for heartbeat: Add test for the case that trial state should be kept running (#4055)
• Remove warnings in the test of Pareto front (#4072)
• Remove matplotlib's get_cmap warning from tests/visualization_tests/test_param_importances.py (#4095)
• Reduce tests' n_trials for CI time reduction (#4117)
• Skip test_pop_waiting_trial_thread_safe on RedisStorage (#4119)
• Simplify the test of BruteForceSampler for infinite search space (#4153)
• Add sep-CMA-ES in parametrize_sampler (#4154)
• Fix a broken test for dask.distributed integration (#4170)
• Add DaskStorage to existing storage tests (#4176, thanks @jrbourbeau!)
• Fix a test error in test_catboost.py (#4190)
• Remove test/integration_tests/test_sampler.py (#4204)
• Fix mypy error abut PyTorch Lightning in Checks (integration) (#4279)
• Remove OPTUNA_STORAGE environment variable to check missing storage errors (#4306)
• Use Trial not FrozenTrial in a test of WeightsAndBiasesCallback (#4309)
• Activate a test case for _set_alembic_revision (#4319)
• Add test to check error_score is stored (#4337)

Code Fixes

• Refactor _tell.py (#3841)
• Make log message user-friendly when objective returns a sequence of unsupported values (#3868)
• Gather mask of None parameter in TPESampler (#3886)
• Migrate CLI from cliff to argparse (#4100)
• Enable mypy --no-implicit-reexport option (#4110)
• Remove unused function: find_any_distribution (#4127)
• Remove object inheritance from base classes (#4161)
• Use mlflow 2.0.1 syntax (#4173)
• Simplify implementation of _preprocess_argv in CLI (#4187)
• Move _solve_hssp to _hypervolume/utils.py (#4227, thanks @jpbianchi!)
• Add tests for CmaEsSampler (#4233)
• Remove an obsoleted logic from CmaEsSampler (#4239)
• Avoid to decode log string in JournalRedisStorage (#4246)
• Fix a typo in TorchDistributedTrial (#4271)
• Fix mypy error abut Chainer in Checks (integration) (#4276)
• Fix mypy error abut BoTorch in Checks (integration) (#4278)
• Fix mypy error abut dask.py in Checks (integration) (#4280)
• Avoid to use features that will be removed in SQLAlchemy v2.0 (#4304)

Continuous Integration

• Hotfix botorch module by adding the version constraint of gpytorch (#3950)
• Drop Python 3.6 from integration CIs (#3983)
• Use PyTorch 1.11 for consistency and fix a typo (#3987)
• Support Python 3.11 (#4018)
• Remove # type: ignore for mypy 0.981 (#4019)
• Fix metric inconsistency between bayesmark plots and report (#4077)
• Pin Ubuntu version to 20.04 in Tests and Tests (Storage with server) (#4118)
• Add workflow to test Optuna with lower versions of constraints (#4125)
• Mark some tests slow and ignore in pull request trigger (#4138, thanks @mist714!)
• Allow display names to be changed in benchmark scripts (Issue #4017) (#4145)
• Disable scheduled workflow runs in forks (#4159)
• Remove the CircleCI job document (#4160)
• Stop running reproducibility tests on CI for PR (#4162)
• Stop running reproducibility tests for coverage (#4163)
• Add workflow_dispatch trigger to the integration tests (#4166)
• Fix CI errors when using mlflow==2.0.1 (#4171)
• Add fakeredis in benchmark dependencies (#4177)
• Fix asv speed benchmark (#4185)
• Skip tests with minimum version for Python 3.10 and 3.11 (#4199)
• Split normal tests and tests with minimum versions (#4200)
• Update action/checkout@v2 -> v3 (#4206)
• Update actions/cache@v2 -> v3 (#4207)
• Update actions/stale@v5 -> v6 (#4208)
• Pin botorch to avoid CI failure (#4228)
• Add the pytest dependency for asv (#4243)
• Fix mypy error about pytorch_distributed.py in Checks (integration) (#4281)
• Run test_pytorch_distributed.py again (#4301)
• Remove all CircleCI config (#4315)
• Update minimum version of cmaes (#4321)
• Add Python 3.11 to integration CI (#4327)

Other

• Bump up version number to 3.1.0.dev (#3934)
• Remove the news section on README (#3940)
• Add issue template for code fix (#3968)
• Close stale issues immediately after labeling stale (#4071)
• Remove tox.ini (#4078)
• Replace gitter with GitHub Discussions (#4083)
• Deprecate description-checked label (#4090)
• Make days-before-issue-stale 300 days (#4091)
• Unnecessary space removed (#4109, thanks @gonzaload!)
• Add note not to share pickle files in bug reports (#4212)
• Update the description of optuna-dashboard on README (#4217)
• Remove optuna.TYPE_CHECKING (#4238)
• Bump up version to v3.1.0-b0 (#4262)
• Remove the list of examples from examples/README (#4283)
• Exclude benchmark directories from the sdist package (#4318)
• Bump up version number to 3.1.0 (#4346)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Abelarm, @Alnusjaponica, @ConnorBaker, @Hakuyume, @HideakiImamura, @Jasha10, @amylase, @belldandyxtq, @c-bata, @contramundum53, @cross32768, @erentknn, @eukaryo, @g-votte, @gasin, @gen740, @gonzaload, @halucinor, @himkt, @hrntsm, @hvy, @jmsykes83, @jpbianchi, @jrbourbeau, @keisuke-umezawa, @knshnb, @mist714, @ncclementi, @not522, @nzw0301, @rene-rex, @reyoung, @semiexp, @shu65, @sile, @toshihikoyanase, @wattlebirdaz, @xadrianzetx, @zaburo-ch

- Python
Published by HideakiImamura over 3 years ago

optuna - v3.1.0-b0

This is the release note of v3.1.0-b0.

Highlights

CMA-ES with Margin support

| CMA-ES | CMA-ES with Margin | | ------- | -------- | | | |

“The animation is referred from https://github.com/EvoConJP/CMA-ESwithMargin, which is distributed under the MIT license.”

CMA-ES achieves strong performance for continuous optimization, but there is still room for improvement in mixed-integer search spaces. To address this, we have added support for the "CMA-ES with Margin" algorithm to our CmaEsSampler, which makes it more efficient in these cases. You can see the benchmark results here. For more detailed information about CMA-ES with Margin, please refer to the paper “CMA-ES with Margin: Lower-Bounding Marginal Probability for Mixed-Integer Black-Box Optimization - arXiv”, which has been accepted for presentation at GECCO 2022.

```python import optuna from optuna.samplers import CmaEsSampler

def objective(trial): x = trial.suggestfloat("y", -10, 10, step=0.1) y = trial.suggestint("x", -100, 100) return x**2 + y

study = optuna.createstudy(sampler=CmaEsSampler(withmargin=True)) study.optimize(objective) ```

Distributed Optimization via NFS

JournalFileStorage, a file storage backend based on JournalStorage, supports NFS (Network File System) environments. It is the easiest option for users who wish to execute distributed optimization in environments where it is difficult to set up database servers such as MySQL, PostgreSQL or Redis (e.g. #815, #1330, #1457 and #2216).

```python import optuna from optuna.storages import JournalStorage, JournalFileStorage

def objective(trial): x = trial.suggestfloat("x", -100, 100) y = trial.suggestfloat("y", -100, 100) return x**2 + y

storage = JournalStorage(JournalFileStorage("./journal.log")) study = optuna.create_study(storage=storage) study.optimize(objective) ```

For more information on JournalFileStorage, see the blog post “Distributed Optimization via NFS Using Optuna’s New Operation-Based Logging Storage” written by @wattlebirdaz.

Dask Integration

DaskStorage, a new storage backend based on Dask.distributed, is supported. It enables distributed computing in similar APIs with concurrent.futures. An example code is like the following (The full example code is available in the optuna-examples repository).

```python import optuna from optuna.storages import InMemoryStorage from optuna.integration import DaskStorage from distributed import Client, wait

def objective(trial): ...

with Client("192.168.1.8:8686") as client: study = optuna.createstudy(storage=DaskStorage(InMemoryStorage())) futures = [ client.submit(study.optimize, objective, ntrials=10, pure=False) for i in range(10) ] wait(futures) print(f"Best params: {study.best_params}") ```

One of the interesting aspects is the availability of InMemoryStorage. You don’t need to set up database servers for distributed optimization. Although you still need to set up the Dask.distributed cluster, it’s quite easy like the following. See Quickstart of the Dask.distributed documentation for more details.

```python $ pip install optuna dask distributed

$ dark-scheduler INFO - Scheduler at: tcp://192.168.1.8:8686 INFO - Dashboard at: :8687 …

$ dask-worker tcp://192.168.1.8:8686 $ dask-worker tcp://192.168.1.8:8686 $ dask-worker tcp://192.168.1.8:8686

$ python dask_simple.py ```

A brand-new Redis storage

We have replaced the Redis storage backend with a JournalStorage-based one. The experimental RedisStorage class has been removed in v3.1. The following example shows how to use the new JournalRedisStorage class.

```python import optuna from optuna.storages import JournalStorage, JournalRedisStorage

def objective(trial): …

storage = JournalStorage(JournalRedisStorage("redis://localhost:6379")) study = optuna.create_study(storage=storage) study.optimize(objective) ```

Sampler for brute-force search

BruteForceSampler, a new sampler for brute-force search, tries all combinations of parameters. In contrast to GridSampler, it does not require passing the search space as an argument and works even with branches. This sampler constructs the search space with the define-by-run style, so it works by just adding sampler=optuna.samplers.BruteForceSampler().

```python import optuna

def objective(trial): c = trial.suggestcategorical("c", ["float", "int"]) if c == "float": return trial.suggestfloat("x", 1, 3, step=0.5) elif c == "int": a = trial.suggestint("a", 1, 3) b = trial.suggestint("b", a, 3) return a + b

study = optuna.create_study(sampler=optuna.samplers.BruteForceSampler()) study.optimize(objective) ```

Breaking Changes

• Allow users to call study.optimize() in multiple threads (#4068)
• Use all trials in TPESampler even when multivariate=True (#4079)
• Drop Python 3.6 (#4150)
• Remove RedisStorage (#4156)
• Deprecate set_system_attr in Study and Trial (#4188)
• Deprecate system_attrs in Study class (#4250)

New Features

• Add Dask integration (#2023, thanks @jrbourbeau!)
• Add journal-style log storage (#3854)
• Support CMA-ES with margin in CmaEsSampler (#4016)
• Add journal redis storage (#4086)
• Add device argument to BoTorchSampler (#4101)
• Add the feature to JournalStorage of Redis backend to resume from a snapshot (#4102)
• Added user_attrs to print by optuna studies in cli.py (#4129, thanks @gonzaload!)
• Add BruteForceSampler (#4132, thanks @semiexp!)
• Add __getstate__ and __setstate__ to RedisStorage (#4135, thanks @shu65!)
• Support pickle in JournalRedisStorage (#4139, thanks @shu65!)
• Support for qNoisyExpectedHypervolumeImprovement acquisition function from BoTorch (Issue#4014) (#4186)

Enhancements

• Change the log message format for failed trials (#3857, thanks @erentknn!)
• Move default logic of get_trial_id_from_study_id_trial_number() method to BaseStorage (#3910)
• Fix the data migration script for v3 release (#4020)
• Convert search_space values of GridSampler explicitly (#4062)
• Add single exception catch to study optimize (#4098)
• Add validation in enqueue_trial (#4126)
• Speed up tests/samplers_tests/test_nsgaii.py::test_fast_non_dominated_sort_with_constraints (#4128, thanks @mist714!)
• Add getstate and setstate to journal storage (#4130, thanks @shu65!)
• Support None in slice plot (#4133, thanks @belldandyxtq!)
• Add marker to matplotlib plot_intermediate_value (#4134, thanks @belldandyxtq!)
• Cache study.directions to reduce the number of get_study_directions() calls (#4146)
• Add an in-memory cache in Trial class (#4240)

Bug Fixes

• Fix infinite loop bug in TPESampler (#3953, thanks @gasin!)
• Fix GridSampler (#3957)
• Fix an import error of sqlalchemy.orm.declarative_base (#3967)
• Skip to add intermediate_value_type and value_type columns if exists (#4015)
• Fix duplicated sampling of SkoptSampler (#4023)
• Avoid parse errors of datetime.isoformat strings (#4025)
• Fix a concurrency bug of JournalStorage set_trial_state_values (#4033)
• Specify object type to numpy array init to avoid unintended str cast (#4035)
• Make TPESampler reproducible (#4056)
• Fix bugs in constant_liar option (#4073)
• Add a flush to JournalFileStorage.append_logs (#4076)
• Add a lock to MLflowCallback (#4097)
• Reject deprecated distributions in OptunaSearchCV (#4120)
• Stop using hash function in _get_bracket_id in HyperbandPruner (#4131, thanks @zaburo-ch!)
• Validation for the parameter enqueued in to_internal_repr of FloatDistribution and IntDistribution (#4137)
• Fix PartialFixedSampler to handle None correctly (#4147, thanks @halucinor!)
• Fix the bug of JournalFileStorage on Windows (#4151)
• Fix CmaEs system attribution key (#4184)

Installation

• Replace thop with fvcore (#3906)
• Use the latest stable scipy (#3959, thanks @gasin!)
• Remove GPyTorch version constraint (#3986)
• Make typing_extensions optional (#3990)
• Add version constraint on importlib-metadata (#4036)
• Add a version constraint of matplotlib (#4044)

Documentation

• Update cli tutorial (#3902)
• Replace thop with fvcore (#3906)
• Slightly improve docs of FrozenTrial (#3943)
• Refine docs in BaseStorage (#3948)
• Remove "Edit on GitHub" button from readthedocs (#3952)
• Mention restoring sampler in saving/resuming tutorial (#3992)
• Use log_loss instead of deprecated log since sklearn 1.1 (#3993)
• Fix script path in benchmarks/README.md (#4021)
• Ignore ConvergenceWarning in the ask-and-tell tutorial (#4032)
• Update docs to let users know the concurrency problem on SQLite3 (#4034)
• Fix the time complexity of NSGAIISampler (#4045)
• Fix sampler comparison table (#4082)
• Add BruteForceSampler in the samplers' list (#4152)
• Remove markup from NaN in FAQ (#4155)
• Remove the document of the multi_objective module (#4167)
• Fix a typo in QMCSampler (#4179)
• Introduce Optuna Dashboard in tutorial docs (#4226)
• Remove RedisStorage from docstring (#4232)
• Add the BruteForceSampler example to the document (#4244)
• Improve the document of BruteForceSampler (#4245)
• Fix an inline markup in distributed tutorial (#4247)

Examples

• Add Dask example (https://github.com/optuna/optuna-examples/pull/46, thanks @jrbourbeau!)
• Hotfix for botorch example (https://github.com/optuna/optuna-examples/pull/134)
• Replace thop with fvcore (https://github.com/optuna/optuna-examples/pull/136)
• Add Optuna-distributed to external projects (https://github.com/optuna/optuna-examples/pull/137)
• Remove the version constraint of GPyTorch (https://github.com/optuna/optuna-examples/pull/138)
• Fix a file path in CONTRIBUTING.md (https://github.com/optuna/optuna-examples/pull/139)
• Install scikit-learn instead of sklearn (https://github.com/optuna/optuna-examples/pull/141)
• Add constraint on tensorflow to <2.11.0 (https://github.com/optuna/optuna-examples/pull/146)
• Specify botorch version (https://github.com/optuna/optuna-examples/pull/151)
• Pin numpy version to 1.23.x for mxnet examples (https://github.com/optuna/optuna-examples/pull/154)

Tests

• Suppress warnings in tests/test_distributions.py (#3912)
• Suppress warnings and minor code fixes in tests/trial_tests (#3914)
• Reduce warning messages by tests/study_tests/ (#3915)
• Remove dynamic search space based objective from a parallel job test (#3916)
• Remove all warning messages from tests/integration_tests/test_sklearn.py (#3922)
• Remove out-of-range related warning messages from MLflowCallback and WeightsAndBiasesCallback (#3923)
• Ignore RuntimeWarning when nanmin and nanmax take an array only containing nan values from pruners_tests (#3924)
• Remove warning messages from test files for pytorch_distributed and chainermn modules (#3927)
• Remove warning messages from tests/integration_tests/test_lightgbm.py (#3944)
• Resolve warnings in tests/visualization_tests/test_contour.py (#3954)
• Reduced warning messages from tests/visualization_tests/test_slice.py (#3970, thanks @jmsykes83!)
• Remove warning from a few visualizaiton tests (#3989)
• Deselect integration tests in Tests CI (#4013)
• Remove warnings from tests/visualization_tests/test_optimization_history.py (#4024)
• Unset PYTHONHASHSEED for the hash-depedenet test (#4031)
• Test: calling study.tell from another process (#4039, thanks @Abelarm!)
• Improve test for heartbeat: Add test for the case that trial state should be kept running (#4055)
• Remove warnings in the test of Paretopereto front (#4072)
• Remove matplotlib get_cmap warning from tests/visualization_tests/test_param_importances.py (#4095)
• Reduce tests' n_trials for CI time reduction (#4117)
• Skip test_pop_waiting_trial_thread_safe on RedisStorage (#4119)
• Simplify the test of BruteForceSampler for infinite search space (#4153)
• Add sep-CMA-ES in parametrize_sampler (#4154)
• Fix a broken test for dask.distributed integration (#4170)
• Add DaskStorage to existing storage tests (#4176, thanks @jrbourbeau!)
• Fix a test error in test_catboost.py (#4190)
• Remove test/integration_tests/test_sampler.py (#4204)

Code Fixes

• Refactor _tell.py (#3841)
• Make log message user-friendly when objective returns a sequence of unsupported values (#3868)
• Gather mask of None parameter in TPESampler (#3886)
• Update cli tutorial (#3902)
• Migrate CLI from cliff to argparse (#4100)
• Enable mypy --no-implicit-reexport option (#4110)
• Remove unused function: find_any_distribution (#4127)
• Remove object inheritance from base classes (#4161)
• Use mlflow 2.0.1 syntax (#4173)
• Simplify implementation of _preprocess_argv in CLI (#4187)
• Move _solve_hssp to _hypervolume/utils.py (#4227, thanks @jpbianchi!)
• Avoid to decode log string in JournalRedisStorage (#4246)

Continuous Integration

• Hotfix botorch module by adding the version constraint of gpytorch (#3950)
• Drop python 3.6 from integration CIs (#3983)
• Use PyTorch 1.11 for consistency and fix a typo (#3987)
• Support Python 3.11 (#4018)
• Remove # type: ignore for mypy 0.981 (#4019)
• Fix metric inconsistency between bayesmark plots and report (#4077)
• Pin Ubuntu version to 20.04 in Tests and Tests (Storage with server) (#4118)
• Add workflow to test Optuna with lower versions of constraints (#4125)
• Mark some tests slow and ignore in pull request trigger (#4138, thanks @mist714!)
• Allow display names to be changed in benchmark scripts (Issue #4017) (#4145)
• Disable scheduled workflow runs in forks (#4159)
• Remove the CircleCI job document (#4160)
• Stop running reproducibility tests on CI for PR (#4162)
• Stop running reproducibility tests for coverage (#4163)
• Add workflow_dispatch trigger to the integration tests (#4166)
• [hotfix] Fix CI errors when using mlflow==2.0.1 (#4171)
• Add fakeredis in benchmark deps (#4177)
• Fix asv speed benchmark (#4185)
• Skip tests with minimum version for Python 3.10 and 3.11 (#4199)
• Split normal tests and tests with minimum versions (#4200)
• Update action/checkout@v2 -> v3 (#4206)
• Update actions/stale@v5 -> v6 (#4208)
• Pin botorch to avoid CI failure (#4228)
• Add the pytest dependency for asv (#4243)

Other

• Bump up version number to 3.1.0.dev (#3934)
• Remove the news section on README (#3940)
• Add issue template for code fix (#3968)
• Close stale issues immediately after labeling stale (#4071)
• Remove tox.ini (#4078)
• Replace gitter with GitHub Discussions (#4083)
• Deprecate description-checked label (#4090)
• Make days-before-issue-stale 300 days (#4091)
• Unnecessary space removed (#4109, thanks @gonzaload!)
• Add note not to share pickle files in bug reports (#4212)
• Update the description of optuna-dashboard on README (#4217)
• Remove optuna.TYPE_CHECKING (#4238)
• Bump up version to v3.1.0-b0 (#4262)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Abelarm, @Alnusjaponica, @HideakiImamura, @amylase, @belldandyxtq, @c-bata, @contramundum53, @cross32768, @erentknn, @eukaryo, @g-votte, @gasin, @gen740, @gonzaload, @halucinor, @himkt, @hvy, @jmsykes83, @jpbianchi, @jrbourbeau, @keisuke-umezawa, @knshnb, @mist714, @ncclementi, @not522, @nzw0301, @rene-rex, @semiexp, @shu65, @sile, @toshihikoyanase, @wattlebirdaz, @xadrianzetx, @zaburo-ch

- Python
Published by contramundum53 over 3 years ago

optuna - v3.0.5

This is the release note of v3.0.5.

Bug Fixes

• [Backport] Fix bugs in constant_liar option (#4257)

Other

• Bump up version number to 3.0.5 (#4256)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @eukaryo, @toshihikoyanase

- Python
Published by toshihikoyanase over 3 years ago

optuna - v3.0.4

This is the release note of v3.0.4.

Bug Fixes

• [Backport] Specify object type to numpy array init to avoid unintended str cast (#4218)

Other

• Bump up version to v3.0.4 (#4214)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @contramundum53

- Python
Published by HideakiImamura over 3 years ago

optuna - v3.0.3

This is the release note of v3.0.3.

Enhancements

• [Backport] Fix the data migration script for v3 release (#4053)

Bug Fixes

• [Backport] Skip to add intermediate_value_type and value_type columns if exists (#4052)

Installation

• Backport #4036 and #4044 to pass tests on release-v3.0.3 branch (#4043)

Other

• Bump up version to v3.0.3 (#4041)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@c-bata, @contramundum53

- Python
Published by c-bata over 3 years ago

optuna - v3.0.2

This is the release note of v3.0.2.

Highlights

Bug fix for DB migration with SQLAlchemy v1.3

In v3.0.0 or v3.0.1, DB migration fails with SQLAlchemy v1.3. We fixed this issue in v3.0.2.

Removing typing-extensions from dependency

In v3.0.0, typing-extensions was used for fine-grained type checking. However, that resulted in import failures when using older versions of typing-extensions. We made the dependency optional in v3.0.2.

Bug Fixes

• [Backport] Merge pull request #3967 from c-bata/fix-issue-3966 (#4004)

Installation

• [Backport] Merge pull request #3990 from c-bata/make-typing-extensions-optional (#4005)

Others

• Bump up version number to v3.0.2 (#3991)

Thanks to All the Contributors!

@contramundum53, @c-bata

This release was made possible by the authors and the people who participated in the reviews and discussions.

- Python
Published by c-bata almost 4 years ago

optuna - v3.0.1

This is the release note of v3.0.1.

Highlights

Bug fix for GridSampler with RDB

In v3.0.0, GridSampler with RDB raises an error. This patch fixes this combination.

Bug Fixes

• Backport #3957 (#3972)

Others

• Bump up version number to v3.0.1 (#3973)

Thanks to All the Contributors!

@HideakiImamura, @contramundum53, @not522

This release was made possible by the authors and the people who participated in the reviews and discussions.

- Python
Published by contramundum53 almost 4 years ago

optuna - v3.0.0

This is the release note of v3.0.0.

This is not something you have to read from top to bottom to learn about the summary of Optuna v3. The recommended way is reading the release blog.

If you want to update your existing projects from Optuna v2.x to Optuna v3, please see the migration guide and try out Optuna v3.

Highlights

New Features

New NSGA-II Crossover Options

New crossover options are added to NSGA-II sampler, the default multi-objective algorithm of Optuna. The performance for floating point parameters are improved. Please visit #2903, #3221, and the document for more information.

A New Algorithm: Quasi-Monte Carlo Sampler

Quasi-Monte Carlo sampler is now supported. It can be used in place of RandomSampler, and can improve performance especially for high dimensional problems. See #2423, #2964, and the document for more information.

Constrained Optimization Support for TPE

TPESampler now supports constraint-aware optimization. For more information on this feature, please visit #3506 and the document.

| Without constraints | With constraints | | - | - | | | |

Constraints Support for Pareto-front Plot

Pareto-front plot now shows which trials satisfy the constraints and which do not. For more information, please see the following PRs (#3128, #3497, and #3389) and the document.

A New Importance Evaluator: ShapleyImportanceEvaluator

We introduced a new importance evaluator, optuna.integration.ShapleyImportanceEvaluator, which uses SHAP. See #3507 and the document for more information.

New History Visualization with Multiple Studies

Optimization history plot can now compare multiple studies or display the mean and variance of multiple studies optimized with the same settings. For more information, please see the following multiple PRs (#2807, #3062, #3122, and #3736) and the document.

Improved Stability

Optuna has a number of core APIs. One being the suggest API and the optuna.Study class. The visualization module is also frequently used to analyze results. Many of these have been simplified, stabilized, and refactored in v3.0.

Simplified Suggest API

The suggest API has been aggregated into 3 APIs: suggest_float for floating point parameters, suggest_int for integer parameters, and suggest_catagorical for categorical parameters. For more information, see #2939, #2941, and PRs submitted for those issues.

Introduction of a Test Policy

We have developed and published a test policy in v3.0 that defines how tests for Optuna should be written. Based on the published test policy, we have improved many unit tests. For more information, see https://github.com/optuna/optuna/issues/2974 and PRs with test label.

Visualization Refactoring

Optuna's visualization module had a deep history and various debts. We have worked throughout v3.0 to eliminate this debt with the help of many contributors. See #2893, #2913, #2959 and PRs submitted for those issues.

Stabilized Features

Through the development of v3.0, we have decided to provide many experimental features as stable features by going through their behavior, fixing bugs, and analyzing use cases. The following is a list of features that have been stabilized in v3.0.

• optuna.study.MaxTrialsCallback
• optuna.study.Study.enqueue_trial
• optuna.study.Study.add_trial
• optuna.study.Study.add_trials
• optuna.study.copy_study
• optuna.trial.create_trial
• optuna.visualization.plotparetofront

Performance Verification

Optuna has many algorithms implemented, but many of their behaviors and characteristics are unknown to the user. We have developed the following table to inform users of empirically known behaviors and characteristics. See #3571 and #3593 for more details.

To quantitatively assess the performance of our algorithms, we have developed a benchmarking environment. We also evaluated the performance of the algorithms by conducting actual benchmarking experiments using this environment. See here, #2964, and #2906 for more details.

Breaking Changes

Changes to the RDB schema:

• To use Optuna v3.0.0 with RDBStorage that was created in the previous versions of Optuna, please execute optuna storage upgrade to migrate your database (#3113, #3559, #3603, #3668).

Features deprecated in 3.0:

• suggest_uniform(), suggest_loguniform(), and suggest_discrete_uniform() UniformDistribution, LogUniformDistribution, DiscreteUniformDistribution, IntUniformDistribution, and IntLogUniformDistribution (#3246, #3420)
• Positional arguments of create_study(), load_study(), delete_study(), and create_study() (#3270)
• axis_order argument of plot_pareto_front() (#3341)

Features removed in 3.0:

• optuna dashboard command (#3058)
• optuna.structs module (#3057)
• best_booster property of LightGBMTuner (#3057)
• type_checking module (#3235)

Minor breaking changes:

• Add option to exclude best trials from study summaries (#3109)
• Move validation logic from _run_trial to study.tell (#3144)
• Use an enqueued parameter that is out of range from suggest API (#3298)
• Fix distribution compatibility for linear and logarithmic distribution (#3444)
• Remove get_study_id_from_trial_id, the method of BaseStorage (#3538)

New Features

• Add interval for LightGBM callback (#2490)
• Allow multiple studies and add error bar option to plot_optimization_history (#2807)
• Support PyTorch-lightning DDP training (#2849, thanks @tohmae!)
• Add crossover operators for NSGA-II (#2903, thanks @yoshinobc!)
• Add abbreviated JSON formats of distributions (#2905)
• Extend MLflowCallback interface (#2912, thanks @xadrianzetx!)
• Support AllenNLP distributed pruning (#2977)
• Make trial.user_attrs logging optional in MLflowCallback (#3043, thanks @xadrianzetx!)
• Support multiple input of studies when plot with Matplotlib (#3062, thanks @TakuyaInoue-github!)
• Add IntDistribution & FloatDistribution (#3063, thanks @nyanhi!)
• Add trial.user_attrs to pareto_front hover text (#3082, thanks @kasparthommen!)
• Support error bar for Matplotlib (#3122, thanks @TakuyaInoue-github!)
• Add optuna tell with --skip-if-finished (#3131)
• Add QMC sampler (#2423, thanks @kstoneriv3!)
• Refactor pareto front and support constraints_func in plot_pareto_front (#3128, thanks @semiexp!)
• Add skip_if_finished flag to Study.tell (#3150, thanks @xadrianzetx!)
• Add user_attrs argument to Study.enqueue_trial (#3185, thanks @knshnb!)
• Option to inherit intermediate values in RetryFailedTrialCallback (#3269, thanks @knshnb!)
• Add setter method for DiscreteUniformDistribution.q (#3283)
• Stabilize allennlp integrations (#3228)
• Stabilize create_trial (#3196)
• Add CatBoostPruningCallback (#2734, thanks @tohmae!)
• Create common API for all NSGA-II crossover operations (#3221)
• Add a history of retried trial numbers in Trial.system_attrs (#3223, thanks @belltailjp!)
• Convert all positional arguments to keyword-only (#3270, thanks @higucheese!)
• Stabilize study.py (#3309)
• Add targets and deprecate axis_order in optuna.visualization.matplotlib.plot_pareto_front (#3341, thanks @shu65!)
• Add targets argument to plot_pareto_plont of plotly backend (#3495, thanks @TakuyaInoue-github!)
• Support constraints_func in plot_pareto_front in matplotlib visualization (#3497, thanks @fukatani!)
• Calculate the feature importance with mean absolute SHAP values (#3507, thanks @liaison!)
• Make GridSampler reproducible (#3527, thanks @gasin!)
• Replace ValueError with warning in GridSearchSampler (#3545)
• Implement callbacks argument of OptunaSearchCV (#3577)
• Add option to skip table creation to RDBStorage (#3581)
• Add constraints option to TPESampler (#3506)
• Add skip_if_exists argument to enqueue_trial (#3629)
• Remove experimental from plot_pareto_front (#3643)
• Add popsize argument to CmaEsSampler (#3649)
• Add seed argument for BoTorchSampler (#3756)
• Add seed argument for SkoptSampler (#3791)
• Revert AllenNLP integration back to experimental (#3822)
• Remove abstractmethod decorator from get_trial_id_from_study_id_trial_number (#3909)

Enhancements

• Add single distribution support to BoTorchSampler (#2928)
• Speed up import optuna (#3000)
• Fix _contains of IntLogUniformDistribution (#3005)
• Render importance scores next to bars in matplotlib.plot_param_importances (#3012, thanks @xadrianzetx!)
• Make default value of verbose_eval NoneN forLightGBMTuner/LightGBMTunerCV` to avoid conflict (#3014, thanks @chezou!)
• Unify colormap of plot_contour (#3017)
• Relax FixedTrial and FrozenTrial allowing not-contained parameters during suggest_* (#3018)
• Raise errors if optuna ask CLI receives --sampler-kwargs without --sampler (#3029)
• Remove _get_removed_version_from_deprecated_version function (#3065, thanks @nuka137!)
• Reformat labels for small importance scores in plotly.plot_param_importances (#3073, thanks @xadrianzetx!)
• Speed up Matplotlib backend plot_contour using SciPy's spsolve (#3092)
• Remove updates in cached storage (#3120, thanks @shu65!)
• Reduce number of queries to fetch directions, user_attrs and system_attrs of study summaries (#3108)
• Support FloatDistribution across codebase (#3111, thanks @xadrianzetx!)
• Use json.loads to decode pruner configuration loaded from environment variables (#3114)
• Show progress bar based on timeout (#3115, thanks @xadrianzetx!)
• Support IntDistribution across codebase (#3126, thanks @nyanhi!)
• Make progress bar available with n_jobs!=1 (#3138, thanks @masap!)
• Wrap RedisStorage in CachedStorage (#3204, thanks @masap!)
• Use functools.wraps in track_in_mlflow decorator (#3216)
• Make RedisStorage fast when running multiple trials (#3262, thanks @masap!)
• Reduce database query result for Study.ask() (#3274, thanks @masap!)
• Enable cache for study.tell() (#3265, thanks @masap!)
• Warn if heartbeat is used with ask-and-tell (#3273)
• Make optuna.study.get_all_study_summaries() of RedisStorage fast (#3278, thanks @masap!)
• Improve Ctrl-C interruption handling (#3374, thanks @CorentinNeovision!)
• Use same colormap among plotly visualization methods (#3376)
• Make EDF plots handle trials with nonfinite values (#3435)
• Make logger message optional in filter_nonfinite (#3438)
• Set precision of sqlalchemy.Float in RDBStorage table definition (#3327)
• Accept nan in trial.report (#3348, thanks @belldandyxtq!)
• Lazy import of alembic, sqlalchemy, and scipy (#3381)
• Unify pareto front (#3389, thanks @semiexp!)
• Make set_trial_param() of RedisStorage faster (#3391, thanks @masap!)
• Make _set_best_trial() of RedisStorage faster (#3392, thanks @masap!)
• Make set_study_directions() of RedisStorage faster (#3393, thanks @masap!)
• Make optuna compatible with wandb sweep panels (#3403, thanks @captain-pool!)
• Change "#Trials" to "Trial" in plot_slice, plot_pareto_front, and plot_optimization_history (#3449, thanks @dubey-anshuman!)
• Make contour plots handle trials with nonfinite values (#3451)
• Query studies for trials only once in EDF plots (#3460)
• Make Parallel-Coordinate plots handle trials with nonfinite values (#3471, thanks @divyanshugit!)
• Separate heartbeat functionality from BaseStorage (#3475)
• Remove torch.distributed calls from TorchDistributedTrial properties (#3490, thanks @nlgranger!)
• Remove the internal logic that calculates the interaction of two or more variables in fANOVA (#3543)
• Handle inf/-inf for trial_values table in RDB (#3559)
• Add intermediate_value_type column to represent inf/-inf on RDBStorage (#3564)
• Move is_heartbeat_enabled from storage to heartbeat (#3596)
• Refactor ImportanceEvaluators (#3597)
• Avoid maximum limit when MLflow saves information (#3651)
• Control metric decimal digits precision in bayesmark benchmark report (#3693)
• Support inf values for crowding distance (#3743)
• Normalize importance values (#3828)

Bug Fixes

• Add tests of sample_relative and fix type of return values of SkoptSampler and PyCmaSampler (#2897)
• Fix GridSampler with RetryFailedTrialCallback or enqueue_trial (#2946)
• Fix the type of trial.values in MLflow integration (#2991)
• Fix to raise ValueError for invalid q in DiscreteUniformDistribution (#3001)
• Do not call trial.report during sanity check (#3002)
• Fix matplotlib.plot_contour bug (#3046, thanks @IEP!)
• Handle single distributions in fANOVA evaluator (#3085, thanks @xadrianzetx!)
• Fix bug of nondeterministic behavior of TPESampler when group=True (#3187, thanks @xuzijian629!)
• Handle non-numerical params in matplotlib.contour_plot (#3213, thanks @xadrianzetx!)
• Fix log scale axes padding in matplotlib.contour_plot (#3218, thanks @xadrianzetx!)
• Handle -inf and inf values in RDBStorage (#3238, thanks @xadrianzetx!)
• Skip limiting the value if it is nan (#3286)
• Make TPE work with a categorical variable with different choice types (#3190, thanks @keisukefukuda!)
• Fix axis range issue in matplotlib contour plot (#3249, thanks @harupy!)
• Allow fail_state_trials show warning when heartbeat is enabled (#3301)
• Clip untransformed values sampled from int uniform distributions (#3319)
• Fix missing user_attrs and system_attrs in study summaries (#3352)
• Fix objective scale in parallel coordinate of Matplotlib (#3369)
• Fix matplotlib.plot_parallel_coordinate with log distributions (#3371)
• Fix parallel coordinate with missing value (#3373)
• Add utility to filter trials with inf values from visualizations (#3395)
• Return the best trial number, not worst trial number by best_index_ (#3410)
• Avoid using px.colors.sequential.Blues that introduces pandas dependency (#3422)
• Fix _is_reverse_scale (#3424)
• Import COLOR_SCALE inside import util context (#3492)
• Remove -v option of optuna study set-user-attr command (#3499, thanks @nyanhi!)
• Filter trials with nonfinite value in optuna.visualization.plot_param_importances and optuna.visualization.matplotlib.plot_param_importance (#3500, thanks @takoika!)
• Fix --verbose and --quiet options in CLI (#3532, thanks @nyanhi!)
• Replace ValueError with RuntimeError in get_best_trial (#3541)
• Take the same search space as in CategoricalDistribution by GridSampler (#3544)
• Fix CategoricalDistribution with NaN (#3567)
• Fix NaN comparison in grid sampler (#3592)
• Fix bug in IntersectionSearchSpace (#3666)
• Remove trial_values records whose values are None (#3668)
• Fix PostgreSQL primary key unsorted problem (#3702, thanks @wattlebirdaz!)
• Raise error on NaN in _constrained_dominates (#3738)
• Fix inf-related issue on implementation of _calculate_nondomination_rank (#3739)
• Raise errors for NaN in constraint values (#3740)
• Fix _calculate_weights such that it throws ValueError on invalid weights (#3742)
• Change warning for axis_order of plot_pareto_front (#3802)
• Fix check for number of objective values (#3808)
• Raise ValueError when waiting trial is told (#3814)
• Fix Study.tell with invalid values (#3819)
• Fix infeasible case in NSGAII test (#3839)

Installation

• Support scikit-learn v1.0.0 (#3003)
• Pin tensorflow and tensorflow-estimator versions to <2.7.0 (#3059)
• Add upper version constraint of PyTorchLightning (#3077)
• Pin keras version to <2.7.0 (#3078)
• Remove version constraints of tensorflow (#3084)
• Bump to torch related packages (#3156)
• Use pytorch-lightning>=1.5.0 (#3157)
• Remove testoutput from doctest of mlflow integration (#3170)
• Restrict nltk version (#3201)
• Add version constraints of setuptools (#3207)
• Remove version constraint of setuptools (#3231)
• Remove Sphinx version constraint (#3237)
• Drop TensorFlow support for Python 3.6 (#3296)
• Pin AllenNLP version (#3367)
• Skip run fastai job on Python 3.6 (#3412)
• Avoid latest click==8.1.0 that removed a deprecated feature (#3413)
• Avoid latest PyTorch lightning until integration is updated (#3417)
• Revert "Avoid latest click==8.1.0 that removed a deprecated feature" (#3430)
• Partially support Python 3.10 (#3353)
• Clean up setup.py (#3517)
• Remove duplicate requirements from document section (#3613)
• Add a version constraint of cached-path (#3665)
• Relax version constraint of fakeredis (#3905)
• Add version constraint for typing_extensions to use ParamSpec (#3926)

Documentation

• Add note of the behavior when calling multiple trial.report (#2980)
• Add note for DDP training of pytorch-lightning (#2984)
• Add note to OptunaSearchCV about direction (#3007)
• Clarify n_trials in the docs (#3016, thanks @Rohan138!)
• Add a note to use pickle with different optuna versions (#3034)
• Unify the visualization docs (#3041, thanks @sidshrivastav!)
• Fix a grammatical error in FAQ doc (#3051, thanks @belldandyxtq!)
• Less ambiguous documentation for optuna tell (#3052)
• Add example for logging.set_verbosity (#3061, thanks @drumehiron!)
• Mention the tutorial of 002_configurations.py in the Trial API page (#3067, thanks @makkimaki!)
• Mention the tutorial of 003_efficient_optimization_algorithms.py in the Trial API page (#3068, thanks @makkimaki!)
• Add link from set_user_attrs in Study to the user_attrs entry in Tutorial (#3069, thanks @MasahitoKumada!)
• Update description for missing samplers and pruners (#3087, thanks @masaaldosey!)
• Simplify the unit testing explanation (#3089)
• Fix range description in suggest_float docstring (#3091, thanks @xadrianzetx!)
• Fix documentation for the package installation procedure on different OS (#3118, thanks @masap!)
• Add description of ValueError and TypeErorr to Raises section of Trial.report (#3124, thanks @MasahitoKumada!)
• Add a note logging_callback only works in single process situation (#3143)
• Correct FrozenTrial's docstring (#3161)
• Promote to use of v3.0.0a0 in README.md (#3167)
• Mention tutorial of callback for Study.optimize from API page (#3171, thanks @xuzijian629!)
• Add reference to tutorial page in study.enqueue_trial (#3172, thanks @knshnb!)
• Fix typo in specify_params (#3174, thanks @knshnb!)
• Guide to tutorial of Multi-objective Optimization in visualization tutorial (#3182, thanks @xuzijian629!)
• Add explanation about Parallelize Optimization at FAQ (#3186, thanks @MasahitoKumada!)
• Add order in tutorial (#3193, thanks @makinzm!)
• Fix inconsistency in distributions documentation (#3222, thanks @xadrianzetx!)
• Add FAQ entry for heartbeat (#3229)
• Replace AUC with accuracy in docs (#3242)
• Fix Raises section of FloatDistribution docstring (#3248, thanks @xadrianzetx!)
• Add {Float,Int}Distribution to docs (#3252)
• Update explanation for metrics of AllenNLPExecutor (#3253)
• Add missing cli methods to the list (#3268)
• Add docstring for property DiscreteUniformDistribution.q (#3279)
• Add reference to tutorial page in CLI (#3267, thanks @tsukudamayo!)
• Carry over notes on step behavior to new distributions (#3276)
• Correct the disable condition of show_progress_bar (#3287)
• Add a document to lead FAQ and example of heartbeat (#3294)
• Add a note for copy_study: it creates a copy regardless of its state (#3295)
• Add note to recommend Python 3.8 or later in documentation build with artifacts (#3312)
• Fix crossover references in Raises doc section (#3315)
• Add reference to QMCSampler in tutorial (#3320)
• Fix layout in tutorial (with workaround) (#3322)
• Scikit-learn required for plot_param_importances (#3332, thanks @ll7!)
• Add a link to multi-objective tutorial from a pareto front page (#3339, thanks @kei-mo!)
• Add reference to tutorial page in visualization (#3340, thanks @Hiroyuki-01!)
• Mention tutorials of User-Defined Sampler/Pruner from the API reference pages (#3342, thanks @hppRC!)
• Add reference to saving/resuming study with RDB backend (#3345, thanks @Hiroyuki-01!)
• Fix a typo (#3360)
• Remove deprecated command optuna study optimize in FAQ (#3364)
• Fix nit typo (#3380)
• Add see also section for best_trial (#3396, thanks @divyanshugit!)
• Updates the tutorial page for re-use the best trial (#3398, thanks @divyanshugit!)
• Add explanation about Study.best_trials in multi-objective optimization tutorial (#3443)
• Clean up exception docstrings (#3429)
• Revise docstring in MLFlow and WandB callbacks (#3477)
• Change the parameter name from classifier to regressor in the code snippet of README.md (#3481)
• Add link to Minituna in CONTRIBUTING.md (#3482)
• Fix benchmarks/README.md for the bayesmark section (#3496)
• Mention Study.stop as a criteria to stop creating trials in document (#3498, thanks @takoika!)
• Fix minor English errors in the docstring of study.optimize (#3505)
• Add Python 3.10 in supported version in README.md (#3508)
• Remove articles at the beginning of sentences in crossovers (#3509)
• Correct FronzenTrial's docstring (#3514)
• Mention specify hyperparameter tutorial (#3515)
• Fix typo in MLFlow callback (#3533)
• Improve docstring of GridSampler's seed option (#3568)
• Add the samplers comparison table (#3571)
• Replace youtube.com with youtube-nocookie.com (#3590)
• Fix time complexity of the samplers comparison table (#3593)
• Remove language from docs configuration (#3594)
• Add documentation of SHAP integration (#3623)
• Remove news entry on Optuna user survey (#3645)
• Introduce optuna-fast-fanova (#3647)
• Add github discussions link (#3660)
• Fix a variable name of ask-and-tell tutorial (#3663)
• Clarify which trials are used for importance evaluators (#3707)
• Fix typo in Study.optimize (#3720, thanks @29Takuya!)
• Update link to plotly's jupyterlab-support page (#3722, thanks @29Takuya!)
• Update CONTRIBUTING.md (#3726)
• Remove "Edit on Github" button (#3777, thanks @cfkazu!)
• Remove duplicated period at the end of copyright (#3778)
• Add note for deprecation of plot_pareto_front's axis_order (#3803)
• Describe the purpose of prepare_study_with_trials (#3809)
• Fix a typo in docstring of ShapleyImportanceEvaluator (#3810)
• Add a reference for MOTPE (#3838, thanks @y0z!)
• Minor fixes of sampler comparison table (#3850)
• Fix typo: Replace trail with trial (#3861)
• Add .. seealso:: in Study.get_trials and Study.trials (#3862, thanks @jmsykes83!)
• Add docstring of TrialState.is_finished (#3869)
• Fix docstring in FrozenTrial (#3872, thanks @wattlebirdaz!)
• Add note to explain when colormap reverses (#3873)
• Make NSGAIISampler docs informative (#3880)
• Add note for constant_liar with multi-objective function (#3881)
• Use copybutton_prompt_text not to copy the bash prompt (#3882)
• Fix typo in HyperbandPruner (#3894)
• Improve HyperBand docs (#3900)
• Mention reproducibility of HyperBandPruner (#3901)
• Add a new note to mention unsupported GPU case for CatBoostPruningCallback (#3903)

Examples

• Use RetryFailedTrialCallback in pytorch_checkpoint example (https://github.com/optuna/optuna-examples/pull/59, thanks @xadrianzetx!)
• Add Python 3.9 to CI yaml files (https://github.com/optuna/optuna-examples/pull/61)
• Replace suggest_uniform with suggest_float (https://github.com/optuna/optuna-examples/pull/63)
• Remove deprecated warning message in lightgbm (https://github.com/optuna/optuna-examples/pull/64)
• Pin tensorflow and tensorflow-estimator versions to <2.7.0 (https://github.com/optuna/optuna-examples/pull/66)
• Restrict upper version of pytorch-lightning (https://github.com/optuna/optuna-examples/pull/67)
• Add an external resource to README.md (https://github.com/optuna/optuna-examples/pull/68, thanks @solegalli!)
• Add pytorch-lightning DDP example (https://github.com/optuna/optuna-examples/pull/43, thanks @tohmae!)
• Install latest AllenNLP (https://github.com/optuna/optuna-examples/pull/73)
• Restrict nltk version (https://github.com/optuna/optuna-examples/pull/75)
• Add version constraints of setuptools (https://github.com/optuna/optuna-examples/pull/76)
• Remove constraint of setuptools (https://github.com/optuna/optuna-examples/pull/79)
• Remove Python 3.6 from haiku's CI (https://github.com/optuna/optuna-examples/pull/83)
• Apply black 22.1.0 & run checks daily (https://github.com/optuna/optuna-examples/pull/84)
• Add hiplot example (https://github.com/optuna/optuna-examples/pull/86)
• Stop running jobs using TF with Python3.6 (https://github.com/optuna/optuna-examples/pull/87)
• Pin AllenNLP version (https://github.com/optuna/optuna-examples/pull/89)
• Add Medium link (https://github.com/optuna/optuna-examples/pull/91)
• Use official CatBoostPruningCallback (https://github.com/optuna/optuna-examples/pull/92)
• Stop running fastai job on Python 3.6 (https://github.com/optuna/optuna-examples/pull/93)
• Specify Python version using str in workflow files (https://github.com/optuna/optuna-examples/pull/95)
• Introduce upper version constraint of PyTorchLightning (https://github.com/optuna/optuna-examples/pull/96)
• Update SimulatedAnnealingSampler to support FloatDistribution (https://github.com/optuna/optuna-examples/pull/97)
• Fix version of JAX (https://github.com/optuna/optuna-examples/pull/99)
• Remove constraints by #99 (https://github.com/optuna/optuna-examples/pull/100)
• Replace some methods in the sklearn example (https://github.com/optuna/optuna-examples/pull/102, thanks @MasahitoKumada!)
• Add Python3.10 in allennlp.yml (https://github.com/optuna/optuna-examples/pull/104)
• Remove numpy (https://github.com/optuna/optuna-examples/pull/105)
• Add python 3.10 to fastai CI (https://github.com/optuna/optuna-examples/pull/106)
• Add python 3.10 to non-integration examples CIs (https://github.com/optuna/optuna-examples/pull/107)
• Add python 3.10 to Hiplot CI (https://github.com/optuna/optuna-examples/pull/108)
• Add a comma to visualization.yml (https://github.com/optuna/optuna-examples/pull/109)
• Rename WandB example to follow naming rules (https://github.com/optuna/optuna-examples/pull/110)
• Add scikit-learn version constraint for Dask-ML (https://github.com/optuna/optuna-examples/pull/112)
• Add python 3.10 to sklearn CI (https://github.com/optuna/optuna-examples/pull/113)
• Set version constraint of protobuf in PyTorch Lightning example (https://github.com/optuna/optuna-examples/pull/116)
• Introduce stale bot (https://github.com/optuna/optuna-examples/pull/119)
• Use Hydra 1.2 syntax (https://github.com/optuna/optuna-examples/pull/122)
• Fix CI due to thop (https://github.com/optuna/optuna-examples/pull/123)
• Hotfix allennlp dependency (https://github.com/optuna/optuna-examples/pull/124)
• Remove unreferenced variable in pytorch_simple.py (https://github.com/optuna/optuna-examples/pull/125)
• set OMPI_MCA_rmaps_base_oversubscribe=yes before mpirun (https://github.com/optuna/optuna-examples/pull/126)
• Add python 3.10 to python-version (https://github.com/optuna/optuna-examples/pull/127)
• Remove upper version constraint of sklearn (https://github.com/optuna/optuna-examples/pull/128)
• Move catboost integration line to integration section from pruning section (https://github.com/optuna/optuna-examples/pull/129)
• Simplify skimage example (https://github.com/optuna/optuna-examples/pull/130)
• Remove deprecated warning in PyTorch Lightning example (https://github.com/optuna/optuna-examples/pull/131)
• Resolve TODO task in ray example (https://github.com/optuna/optuna-examples/pull/132)
• Remove version constraint of cached-path (https://github.com/optuna/optuna-examples/pull/133)

Tests

• Add test case of samplers for conditional objective function (#2904)
• Test int distributions with default step (#2924)
• Be aware of trial preparation when checking heartbeat interval (#2982)
• Simplify the DDP model definition in the test of pytorch-lightning (#2983)
• Wrap data with np.asarray in lightgbm test (#2997)
• Patch calls to deprecated suggest APIs across codebase (#3027, thanks @xadrianzetx!)
• Make return_cvbooster of LightGBMTuner consistent to the original value (#3070, thanks @abatomunkuev!)
• Fix parametrize_sampler (#3080)
• Fix verbosity for tests/integration_tests/lightgbm_tuner_tests/test_optimize.py (#3086, thanks @nyanhi!)
• Generalize empty search space test case to all hyperparameter importance evaluators (#3096, thanks @xadrianzetx!)
• Check if texts in legend by order agnostic way (#3103)
• Add tests for axis scales to matplotlib.plot_slice (#3121)
• Add tests for transformer with upper bound parameter (#3163)
• Add tests in visualization_tests/matplotlib_tests/test_slice.py (#3175, thanks @keisukefukuda!)
• Add test case of the value in optimization history with matplotlib (#3176, thanks @TakuyaInoue-github!)
• Add tests for generated plots of matplotlib.plot_edf (#3178, thanks @makinzm!)
• Improve pareto front figure tests for matplotlib (#3183, thanks @akawashiro!)
• Add tests for generated plots of plot_edf (#3188, thanks @makinzm!)
• Match contour tests between Plotly and Matplotlib (#3192, thanks @belldandyxtq!)
• Implement missing matplotlib.contour_plot test (#3232, thanks @xadrianzetx!)
• Unify the validation function of edf value between visualization backends (#3233)
• Add test for default grace period (#3263, thanks @masap!)
• Add the missing tests of Plotly's plot_parallel_coordinate (#3266, thanks @MasahitoKumada!)
• Switch function order progbar tests (#3280, thanks @BasLaa!)
• Add plot value tests to matplotlib_tests/test_param_importances (#3180, thanks @belldandyxtq!)
• Make tests of plot_optimization_history methods consistent (#3234)
• Add integration test for RedisStorage (#3258, thanks @masap!)
• Change the order of arguments in the catalyst integration test (#3308)
• Cleanup MLflowCallback tests (#3378)
• Test serialize/deserialize storage on parametrized conditions (#3407)
• Add tests for parameter of 'None' for TPE (#3447)
• Improve matplotlib parallel coordinate test (#3368)
• Save figures for all matplotlib tests (#3414, thanks @divyanshugit!)
• Add inf test to intermediate values test (#3466)
• Add test cases for test_storages.py (#3480)
• Improve the tests of optuna.visualization.plot_pareto_front (#3546)
• Move heartbeat-related tests in test_storages.py to another file (#3553)
• Use seed method of np.random.RandomState for reseeding and fix test_reseed_rng (#3569)
• Refactor test_get_observation_pairs (#3574)
• Add tests for inf/nan objectives for ShapleyImportanceEvaluator (#3576)
• Add deprecated warning test to the multi-objective sampler test file (#3601)
• Simplify multi-objective TPE tests (#3653)
• Add edge cases to multi-objective TPE tests (#3662)
• Remove tests on TypeError (#3667)
• Add edge cases to the tests of the parzen estimator (#3673)
• Add tests for _constrained_dominates (#3683)
• Refactor tests of constrained TPE (#3689)
• Add inf and NaN tests for test_constraints_func (#3690)
• Fix calling storage API in study tests (#3695, thanks @wattlebirdaz!)
• DRY test_frozen.py (#3696)
• Unify the tests of plot_contours (#3701)
• Add test cases for crossovers of NSGAII (#3705)
• Enhance the tests of NSGAIISampler._crowding_distance_sort (#3706)
• Unify edf test files (#3730)
• Fix test_calculate_weights_below (#3741)
• Refactor test_intermediate_plot.py (#3745)
• Test samplers are reproducible (#3757)
• Add tests for _dominates function (#3764)
• DRY importance tests (#3785)
• Move tests for create_trial (#3794)
• Remove with_c_d option from prepare_study_with_trials (#3799)
• Use DeterministicRelativeSampler in test_trial.py (#3807)
• Add tests for _fast_non_dominated_sort (#3686)
• Unify slice plot tests (#3784)
• Unify the tests of plot_parallel_coordinates (#3800)
• Unify optimization history tests (#3806)
• Suppress warnings in tests for multi_objective module (#3911)
• Remove warnings: UserWarning from tests/visualization_tests/test_utils.py (#3919, thanks @jmsykes83!)

Code Fixes

• Add test case of samplers for conditional objective function (#2904)
• Fix #2949, remove BaseStudy (#2986, thanks @twsl!)
• Use optuna.load_study in optuna ask CLI to omit direction/directions option (#2989)
• Fix typo in Trial warning message (#3008, thanks @xadrianzetx!)
• Replaces boston dataset with california housing dataset (#3011, thanks @avats-dev!)
• Fix deprecation version of suggest APIs (#3054, thanks @xadrianzetx!)
• Add remove_version to the missing @deprecated argument (#3064, thanks @nuka137!)
• Add example of optuna.logging.get_verbosity (#3066, thanks @MasahitoKumada!)
• Support {Float|Int}Distribution in NSGA-II crossover operators (#3139, thanks @xadrianzetx!)
• Black fix (#3147)
• Switch to FloatDistribution (#3166, thanks @xadrianzetx!)
• Remove deprecated decorator of the feature of n_jobs (#3173, thanks @MasahitoKumada!)
• Fix black and blackdoc errors (#3260, thanks @masap!)
• Remove experimental label from MaxTrialsCallback (#3261, thanks @knshnb!)
• Remove redundant _check_trial_id (#3264, thanks @masap!)
• Make existing int/float distributions wrapper of {Int,Float}Distribution (#3244)
• Switch to IntDistribution (#3181, thanks @nyanhi!)
• Fix type hints for Python 3.8 (#3240)
• Remove UniformDistribution, LogUniformDistribution and DiscreteUniformDistribution code paths (#3275)
• Merge set_trial_state() and set_trial_values() into one function (#3323, thanks @masap!)
• Follow up for {Float, Int}Distributions (#3337, thanks @nyanhi!)
• Move the get_trial_xxx abstract functions to base (#3338, thanks @belldandyxtq!)
• Update type hints of states (#3359, thanks @BasLaa!)
• Remove unused function from RedisStorage (#3394, thanks @masap!)
• Remove unnecessary string concatenation (#3406)
• Follow coding style and fix typos in tests/integration_tests (#3408)
• Fix log message formatting in filter_nonfinite (#3436)
• Add RetryFailedTrialCallback to optuna.storages.* (#3441)
• Unify fail_stale_trials in each storage implementation (#3442, thanks @knshnb!)
• Ignore incomplete trials in matplotlib.plot_parallel_coordinate (#3415)
• Update warning message and add a test when a trial fails with exception (#3454)
• Remove old distributions from NSGA-II sampler (#3459)
• Remove duplicated DB access in _log_completed_trial (#3551)
• Reduce the number of copy.deepcopy() calls in importance module (#3554)
• Remove duplicated check_trial_is_updatable (#3557)
• Replace optuna.testing.integration.create_running_trial with study.ask (#3562)
• Refactor test_get_observation_pairs (#3574)
• Update label of feasible trials if constraints_func is specified (#3587)
• Replace unused variable name with underscore (#3588)
• Enable no-implicit-optional for mypy (#3599, thanks @harupy!)
• Enable warn_redundant_casts for mypy (#3602, thanks @harupy!)
• Refactor the type of value of TrialIntermediateValueModel (#3603)
• Fix broken mypy checks of Alembic's get_current_head() method (#3608)
• Move heartbeat-related thread operation in _optimize.py to _heartbeat.py (#3609)
• Sort dependencies by name (#3614)
• Add typehint for deprecated and experimental (#3575)
• Remove useless object inheritance (#3628, thanks @harupy!)
• Remove useless except clauses (#3632, thanks @harupy!)
• Rename optuna.testing.integration with optuna.testing.pruner (#3638)
• Cosmetic fix in Optuna CLI (#3641)
• Enable strict_equality for mypy #3579 (#3648, thanks @wattlebirdaz!)
• Make file names in testing consistent with optuna module (#3657)
• Remove the implementation of read_trials_from_remote_storage in the all storages apart from CachedStorage (#3659)
• Remove unnecessary deep copy in Redis storage (#3672, thanks @wattlebirdaz!)
• Workaround mypy bug (#3679)
• Unify plot_contours (#3682)
• Remove storage.get_all_study_summaries(include_best_trial: bool) (#3697, thanks @wattlebirdaz!)
• Unify the logic of edf functions (#3698)
• Unify the logic of plot_param_importances functions (#3700)
• Enable disallow_untyped_calls for mypy (#3704, thanks @29Takuya!)
• Use get_trials with states argument to filter trials depending on trial state (#3708)
• Return Python's native float values (#3714)
• Simplify bayesmark benchmark report rendering (#3725)
• Unify the logic of intermediate plot (#3731)
• Unify the logic of slice plot (#3732)
• Unify the logic of plot_parallel_coordinates (#3734)
• Unify implementation of plot_optimization_history between plotly and matplotlib (#3736)
• Extract fail_objective and pruned_objective for tests (#3737)
• Remove deprecated storage functions (#3744, thanks @29Takuya!)
• Remove unnecessary optionals from visualization/_pareto_front.py (#3752)
• Change types inside _ParetoInfoType (#3753)
• Refactor pareto front (#3754)
• Use _ContourInfo to plot in plot_contour (#3755)
• Follow up #3465 (#3763)
• Refactor importances plot (#3765)
• Remove no_trials option of prepare_study_with_trials (#3766)
• Follow the coding style of comments in plot_contour files (#3767)
• Raise ValueError for invalid returned type of target in _filter_nonfinite (#3768)
• Fix value error condition in plot_contour (#3769)
• DRY constraints in Sampler.after_trial (#3775)
• DRY stop_objective (#3786)
• Refactor non-exist param test in plot_contour test (#3787)
• Remove less_than_two and more_than_three options from prepare_study_with_trials (#3789)
• Fix return value's type of _get_node_value (#3818)
• Remove unused type: ignore (#3832)
• Fix typos and remove unused argument in QMCSampler (#3837)
• Unify tests for plot_param_importances (#3760)
• Refactor test_pareto_front (#3798)
• Remove duplicated definition of CategoricalChoiceType from optuna.distributions (#3846)
• Revert most of changes by 3651 (#3848)
• Attach abstractmethod decorator to BaseStorage.get_trial_id_from_study_id_trial_number (#3870, thanks @wattlebirdaz!)
• Refactor BaseStorage.get_best_trial (#3871, thanks @wattlebirdaz!)
• Simplify IntersectionSearchSpace.calculate (#3887)
• Replace q with step in private function and warning message (#3913)
• Reduce warnings in storage tests (#3917)
• Reduce trivial warning messages from tests/sampler_tests (#3921)

Continuous Integration

• Install botorch to CI jobs on mac (#2988)
• Use libomp 11.1.0 for Mac (#3024)
• Run mac-tests CI at a scheduled time (#3028)
• Set concurrency to github workflows (#3095)
• Skip CLI tests when calculating the coverage (#3097)
• Migrate mypy version to 0.910 (#3123)
• Avoid installing the latest MLfow to prevent doctests from failing (#3135)
• Use python 3.8 for CI and docker (#3026)
• Add performance benchmarks using kurobako (#3155)
• Use Python 3.7 in checks CI job (#3239)
• Add performance benchmarks using bayesmark (#3354)
• Fix speed benchmarks (#3362)
• Pin setuptools (#3427)
• Introduce the benchmark for multi-objectives samplers (#3271, thanks @drumehiron!)
• Use coverage directly (#3347, thanks @higucheese!)
• Add WFG benchmark test (#3349, thanks @kei-mo!)
• Add workflow to use reviewdog (#3357)
• Add NASBench201 from NASLib (#3465)
• Fix speed benchmarks CI (#3470)
• Support PyTorch 1.11.0 (#3510)
• Install 3rd party libraries in CI for lint (#3580)
• Make bayesmark benchmark results comparable to kurobako (#3584)
• Restore virtualenv for benchmark extras (#3585)
• Use protobuf<4.0.0 to resolve Sphinx CI error (#3591)
• Unpin protobuf (#3598, thanks @harupy!)
• Extract MPI tests from integration CI as independent CI (#3606)
• Enable warn_unused_ignores for mypy (#3627, thanks @harupy!)
• Add onnx and version constrained protobuf to document dependencies (#3658)
• Add mo-kurobako benchmark to CI (#3691)
• Enable mypy's strict configs (#3710)
• Run visual regression tests to find regression bugs of visualization module (#3721)
• Remove downloading old libomp for mac tests (#3728)
• Match Python versions between bayesmark CI jobs (#3750)
• Set OMPI_MCA_rmaps_base_oversubscribe=yes before mpirun (#3758)
• Add budget option to benchmarks (#3774)
• Add n_concurrency option to benchmarks (#3776)
• Use n-runs instead of repeat to represent the number of studies in the bayesmark benchmark (#3780)
• Fix type hints for mypy 0.971 (#3797)
• Pin scipy to avoid the CI failure (#3834)
• Extract float value from tensor for trial.report in PyTorchLightningPruningCallback (#3842)

Other

• Bump up version to 2.11.0dev (#2976)
• Add roadmap news to README.md (#2999)
• Bump up version number to 3.0.0a1.dev (#3006)
• Add Python 3.9 to tox.ini (#3025)
• Fix version number to 3.0.0a0 (#3140)
• Bump up version to v3.0.0a1.dev (#3142)
• Introduce a form to make TODOs explicit when creating issues (#3169)
• Bump up version to v3.0.0b0.dev (#3289)
• Add description field for question-and-help-support (#3305)
• Update README to inform v3.0.0a2 (#3314)
• Add Optuna-related URLs for PyPi (#3355, thanks @andriyor!)
• Bump Optuna to v3.0.0-b0 (#3458)
• Bump up version to v3.0.0b1.dev (#3457)
• Fix kurobako benchmark code to run it locally (#3468)
• Fix label of issue template (#3493)
• Improve issue templates (#3536)
• Hotfix for fakeredis 1.7.4 release (#3549)
• Remove the version constraint of fakeredis (#3561)
• Relax version constraint of fakeredis (#3607)
• Shorten the durations of the stale bot for PRs (#3611)
• Clarify the criteria to assign reviewers in the PR template (#3619)
• Bump up version number to v3.0.0rc0.dev (#3621)
• Make tox.ini consistent with checking (#3654)
• Avoid to stale description-checked issues (#3816)
• Bump up version to v3.0.0.dev (#3852)
• Bump up version to v3.0.0 (#3933)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@29Takuya, @BasLaa, @CorentinNeovision, @Crissman, @HideakiImamura, @Hiroyuki-01, @IEP, @MasahitoKumada, @Rohan138, @TakuyaInoue-github, @abatomunkuev, @akawashiro, @andriyor, @avats-dev, @belldandyxtq, @belltailjp, @c-bata, @captain-pool, @cfkazu, @chezou, @contramundum53, @divyanshugit, @drumehiron, @dubey-anshuman, @fukatani, @g-votte, @gasin, @harupy, @higucheese, @himkt, @hppRC, @hvy, @jmsykes83, @kasparthommen, @kei-mo, @keisuke-umezawa, @keisukefukuda, @knshnb, @kstoneriv3, @liaison, @ll7, @makinzm, @makkimaki, @masaaldosey, @masap, @nlgranger, @not522, @nuka137, @nyanhi, @nzw0301, @semiexp, @shu65, @sidshrivastav, @sile, @solegalli, @takoika, @tohmae, @toshihikoyanase, @tsukudamayo, @tupui, @twsl, @wattlebirdaz, @xadrianzetx, @xuzijian629, @y0z, @yoshinobc, @ytsmiling

- Python
Published by HideakiImamura almost 4 years ago

optuna - v3.0.0-rc0

This is the release note of v3.0.0-rc0. This is a release candidate of Optuna V3. We plan to release the major version within a few weeks. Please try this version and report bugs!

Highlights

Constrained Optimization Support for TPE

TPESampler, the default sampler of Optuna, now supports constrained optimization. It takes a function constraints_func as an argument, and examines whether trials are feasible or not. Feasible trials are prioritized over infeasible ones similarly to NSGAIISampler. See #3506 for more details.

```python def objective(trial): # Binh and Korn function with constraints. x = trial.suggestfloat("x", -15, 30) y = trial.suggestfloat("y", -15, 30)

# Store the constraints as user attributes so that they can be restored after optimization.
c0 = (x - 5) ** 2 + y ** 2 - 25
c1 = -((x - 8) ** 2) - (y + 3) ** 2 + 7.7
trial.set_user_attr("constraints", (c0, c1))

v0 = 4 * x ** 2 + 4 * y ** 2
v1 = (x - 5) ** 2 + (y - 5) ** 2

return v0, v1

def constraints(trial): return trial.user_attrs["constraints"]

if name == "main": sampler = optuna.samplers.TPESampler( constraintsfunc=constraints, ) study = optuna.createstudy( directions=["minimize", "minimize"], sampler=sampler, ) study.optimize(objective, n_trials=1000)

optuna.visualization.plot_pareto_front(study, constraints_func=constraints).show()

```

| MOTPE without constraints | MOTPE with constraints | | - | - | | | |

A Major Refactoring of Visualization Module

We have undertaken major refactoring of the visualization features as one of the major tasks of Optuna V3. The current situation is as follows.

Unification of implementations of different backends: plotly and matplotlib

Historically, the implementations of Optuna's visualization features were split between two different backends, plotly and matplotlib. Many of these implementations were duplicated and unmaintainable, and many were implemented as a single large function, resulting in poor testability and, as a result, becoming the cause of many bugs. We clarified the specifications that each visualization function in Optuna must meet and defined the backend-independent information needed to perform the visualization. By using this information commonly across different backends, we achieved a highly maintainable and testable implementation, and improved the stability of the visualization functions dramatically. We are currently rewriting the unit tests, and the resulting tests will be simple yet powerful.

Visual Regression Test

It is very important to detect hidden bugs in the implementation through PR reviews. However, visualizations are likely to contain bugs that are difficult to find just by reading the code, and many of these bugs are only revealed when the visualization is actually performed. Therefore, we introduced the Visual Regression Test to improve the review process. In the PR for visualization features, you can jump to the Visual Regression Test link by clicking on the link generated from within the PR. Reviewers can verify that the PR implementation is performing the visualization properly.

Improve Code Quality Including Many Bugfix

In the latter development cycle of Optuna v3, we put emphasis on improving the overall code quality of the library. We fixed several bugs and possible corruption of internal data structures on e.g. handling Inf/NaN values (#3567, #3592, #3738, #3739, #3740) and invalid inputs (#3668, #3808, #3814, #3819). For example, there had been bugs before v3 when NaN values were used in a CategoricalDistribution or GridSampler. In several other functions, NaN values were unacceptable but the library failed silently without any warning or error. Such bugs are fixed in this release.

New Features

• Add constraints option to TPESampler (#3506)
• Add skip_if_exists argument to enqueue_trial (#3629)
• Remove experimental from plot_pareto_front (#3643)
• Add popsize argument to CmaEsSampler (#3649)
• Add seed argument for BoTorchSampler (#3756)
• Add seed argument for SkoptSampler (#3791)
• Revert AllenNLP integration back to experimental (#3822)

Enhancements

• Move is_heartbeat_enabled from storage to heartbeat (#3596)
• Refactor ImportanceEvaluators (#3597)
• Avoid maximum limit when MLflow saves information (#3651)
• Control metric decimal digits precision in bayesmark benchmark report (#3693)
• Support inf values for crowding distance (#3743)
• Normalize importance values (#3828)

Bug Fixes

• Fix CategoricalDistribution with NaN (#3567)
• Fix NaN comparison in grid sampler (#3592)
• Fix bug in IntersectionSearchSpace (#3666)
• Remove trial_values records whose values are None (#3668)
• Fix PostgreSQL primary key unsorted problem (#3702, thanks @wattlebirdaz!)
• Raise error on NaN in _constrained_dominates (#3738)
• Fix inf-related issue on implementation of _calculate_nondomination_rank (#3739)
• Raise errors for NaN in constraint values (#3740)
• Fix _calculate_weights such that it throws ValueError on invalid weights (#3742)
• Change warning for axis_order of plot_pareto_front (#3802)
• Fix check for number of objective values (#3808)
• Raise ValueError when waiting trial is told (#3814)
• Fix Study.tell with invalid values (#3819)
• Fix infeasible case in NSGAII test (#3839)

Installation

• Add a version constraint of cached-path (#3665)

Documentation

• Add documentation of SHAP integration (#3623)
• Remove news entry on Optuna user survey (#3645)
• Introduce optuna-fast-fanova (#3647)
• Add github discussions link (#3660)
• Fix a variable name of ask-and-tell tutorial (#3663)
• Clarify which trials are used for importance evaluators (#3707)
• Fix typo in Study.optimize (#3720, thanks @29Takuya!)
• Update link to plotly's jupyterlab-support page (#3722, thanks @29Takuya!)
• Update CONTRIBUTING.md (#3726)
• Remove "Edit on Github" button (#3777, thanks @cfkazu!)
• Remove duplicated period at the end of copyright (#3778)
• Add note for deprecation of plot_pareto_front's axis_order (#3803)
• Describe the purpose of prepare_study_with_trials (#3809)
• Fix a typo in docstring of ShapleyImportanceEvaluator (#3810)
• Add a reference for MOTPE (#3838, thanks @y0z!)

Examples

• Introduce stale bot (https://github.com/optuna/optuna-examples/pull/119)
• Use Hydra 1.2 syntax (https://github.com/optuna/optuna-examples/pull/122)
• Fix CI due to thop (https://github.com/optuna/optuna-examples/pull/123)
• Hotfix allennlp dependency (https://github.com/optuna/optuna-examples/pull/124)
• Remove unreferenced variable in pytorch_simple.py (https://github.com/optuna/optuna-examples/pull/125)
• set OMPI_MCA_rmaps_base_oversubscribe=yes before mpirun (https://github.com/optuna/optuna-examples/pull/126)

Tests

• Simplify multi-objective TPE tests (#3653)
• Add edge cases to multi-objective TPE tests (#3662)
• Remove tests on TypeError (#3667)
• Add edge cases to the tests of the parzen estimator (#3673)
• Add tests for _constrained_dominates (#3683)
• Refactor tests of constrained TPE (#3689)
• Add inf and NaN tests for test_constraints_func (#3690)
• Fix calling storage API in study tests (#3695, thanks @wattlebirdaz!)
• DRY test_frozen.py (#3696)
• Unify the tests of plot_contours (#3701)
• Add test cases for crossovers of NSGAII (#3705)
• Enhance the tests of NSGAIISampler._crowding_distance_sort (#3706)
• Unify edf test files (#3730)
• Fix test_calculate_weights_below (#3741)
• Refactor test_intermediate_plot.py (#3745)
• Test samplers are reproducible (#3757)
• Add tests for _dominates function (#3764)
• DRY importance tests (#3785)
• Move tests for create_trial (#3794)
• Remove with_c_d option from prepare_study_with_trials (#3799)
• Use DeterministicRelativeSampler in test_trial.py (#3807)

Code Fixes

• Add typehint for deprecated and experimental (#3575)
• Move heartbeat-related thread operation in _optimize.py to _heartbeat.py (#3609)
• Remove useless object inheritance (#3628, thanks @harupy!)
• Remove useless except clauses (#3632, thanks @harupy!)
• Rename optuna.testing.integration with optuna.testing.pruner (#3638)
• Cosmetic fix in Optuna CLI (#3641)
• Enable strict_equality for mypy #3579 (#3648, thanks @wattlebirdaz!)
• Make file names in testing consistent with optuna module (#3657)
• Remove the implementation of read_trials_from_remote_storage in the all storages apart from CachedStorage (#3659)
• Remove unnecessary deep copy in Redis storage (#3672, thanks @wattlebirdaz!)
• Workaround mypy bug (#3679)
• Unify plot_contours (#3682)
• Remove storage.get_all_study_summaries(include_best_trial: bool) (#3697, thanks @wattlebirdaz!)
• Unify the logic of edf functions (#3698)
• Unify the logic of plot_param_importances functions (#3700)
• Enable disallow_untyped_calls for mypy (#3704, thanks @29Takuya!)
• Use get_trials with states argument to filter trials depending on trial state (#3708)
• Return Python's native float values (#3714)
• Simplify bayesmark benchmark report rendering (#3725)
• Unify the logic of intermediate plot (#3731)
• Unify the logic of slice plot (#3732)
• Unify the logic of plot_parallel_coordinates (#3734)
• Unify implementation of plot_optimization_history between plotly and matplotlib (#3736)
• Extract fail_objective and pruned_objective for tests (#3737)
• Remove deprecated storage functions (#3744, thanks @29Takuya!)
• Remove unnecessary optionals from visualization/_pareto_front.py (#3752)
• Change types inside _ParetoInfoType (#3753)
• Refactor pareto front (#3754)
• Use _ContourInfo to plot in plot_contour (#3755)
• Follow up #3465 (#3763)
• Refactor importances plot (#3765)
• Remove no_trials option of prepare_study_with_trials (#3766)
• Follow the coding style of comments in plot_contour files (#3767)
• Raise ValueError for invalid returned type of target in _filter_nonfinite (#3768)
• Fix value error condition in plot_contour (#3769)
• DRY constraints in Sampler.after_trial (#3775)
• DRY stop_objective (#3786)
• Refactor non-exist param test in plot_contour test (#3787)
• Remove less_than_two and more_than_three options from prepare_study_with_trials (#3789)
• Fix return value's type of _get_node_value (#3818)
• Remove unused type: ignore (#3832)
• Fix typos and remove unused argument in QMCSampler (#3837)

Continuous Integration

• Use coverage directly (#3347, thanks @higucheese!)
• Install 3rd party libraries in CI for lint (#3580)
• Make bayesmark benchmark results comparable to kurobako (#3584)
• Enable warn_unused_ignores for mypy (#3627, thanks @harupy!)
• Add onnx and version constrained protobuf to document dependencies (#3658)
• Add mo-kurobako benchmark to CI (#3691)
• Enable mypy's strict configs (#3710)
• Run visual regression tests to find regression bugs of visualization module (#3721)
• Remove downloading old libomp for mac tests (#3728)
• Match Python versions between bayesmark CI jobs (#3750)
• Set OMPI_MCA_rmaps_base_oversubscribe=yes before mpirun (#3758)
• Add budget option to benchmarks (#3774)
• Add n_concurrency option to benchmarks (#3776)
• Use n-runs instead of repeat to represent the number of studies in the bayesmark benchmark (#3780)
• Fix type hints for mypy 0.971 (#3797)
• Pin scipy to avoid the CI failure (#3834)
• Extract float value from tensor for trial.report in PyTorchLightningPruningCallback (#3842)

Other

• Clarify the criteria to assign reviewers in the PR template (#3619)
• Bump up version number to v3.0.0rc0.dev (#3621)
• Make tox.ini consistent with checking (#3654)
• Avoid to stale description-checked issues (#3816)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@29Takuya, @HideakiImamura, @c-bata, @cfkazu, @contramundum53, @g-votte, @harupy, @higucheese, @himkt, @hvy, @keisuke-umezawa, @knshnb, @not522, @nzw0301, @sile, @toshihikoyanase, @wattlebirdaz, @xadrianzetx, @y0z

- Python
Published by HideakiImamura almost 4 years ago

optuna - v2.10.1

This is the release note of v2.10.1.

This is a patch release to resolve the issues on the document build. No feature updates are included.

Installation

• Fix document build of v2.10.1 (#3642)

Documentation

• Backport #3590: Replace youtube.com with youtube-nocookie.com (#3633)

Other

• Bump up version to v2.10.1 (#3635)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@contramundum53, @toshihikoyanase

- Python
Published by contramundum53 about 4 years ago

optuna - v3.0.0-b1

This is the release note of v3.0.0-b1.

Highlights

A Samplers Comparison Table

We added a sampler comparison table on the samplers' documentation page. It includes supported options (parameter types, pruning, multi-objective optimization, constrained optimization, etc.), time complexity, and recommended budgets for each sampler. Please use this to select appropriate samplers for your tasks! See #3571 and #3593 for more details.

A New Importance Evaluator: ShapleyImportanceEvaluator

Optuna now supports mean absolute SHAP value for evaluating parameter importances through integration with the SHAP library. SHAP value is a game-theoretic measure of parameter importance featuring nice theoretical properties (See paper for more information).

To use mean absolute SHAP importances, an object of optuna.integration.shap.ShapleyImportanceEvaluator can be passed to evaluator argument in optuna.visualization.plot_param_importances or optuna.importance.get_param_importances.

``` import optuna from optuna.integration.shap import ShapleyImportanceEvaluator

study = optuna.createstudy() study.optimize(objective, ntrials=100)

optuna.visualization.plotparamimportances(study, evaluator=ShapleyImportanceEvaluator()) ```

See the #3507 for more details.

A New Benchmarking Task

The benchmarking environment for black-box optimization algorithms on the GitHub Actions was introduced in the previous release. We have further enhanced its capabilities. The benchmarking functionality introduced can be run on all users' forks using GitHub Actions. You can also freely customize and run benchmarks on more computationally powerful clusters, for example, AWS, using the code in the optuna/benchmarks directory.

Neural Architecture Search Benchmark Support

Optuna's algorithms can now be benchmarked using NASLib, the Neural Architecture Search benchmark library. For now we only support one dataset, NASBench 201, which deals with image recognition. Larger datasets and datasets from other areas such as natural language processing will be supported in the future.

| cifar10 | cifar100 | imagenet16-120| | ---- | ---- | ---- | | | | |

See README and #3465 for more information.

Multi-objective Optimization Benchmark Support

We are now able to benchmark our multi-objective optimization algorithms. They are not yet available on GitHub Actions, but you can use optuna/benchmarks/run_mo_kurobakmo.py directly. They will be available on GitHub Actions in the next release, so stay tuned! See #3271 and #3349 for more details.

Python 3.10 Support

This is the first version to officially support Python 3.10. All tests are passed including integration modules, with a few exceptions.

Storage Database Migration

To use Optuna v3.0.0-b1 with RDBStorage that was created in the previous versions of Optuna, please run optuna storage upgrade to migrate your database.

```sh

YOUR_RDB_URL is the URL of your database.

optuna storage upgrade –storage YOURRDBURL ```

If you use RedisStorage, copy your study with RDBStorage using copy_study with the Optuna you used to create the study, thenrun optuna storage upgrade with Optuna v3.0.0-b0. After upgrading the storage, copy the study back as a new RedisStorage.

sh python -c ‘import optuna; optuna.copy_study(from_study_name=”example”, from_storage=”redis://localhost:6379”, to_storage=”sqlite:///upgrade.db”) pip install –pre -U optuna optuna storage upgrade –storage sqlite:///upgrade.db python -c ‘import optuna; optuna.copy_study(from_study_name="example", from_storage="sqlite:///upgrade.db", to_study_name="new-example", to_storage="redis://localhost:6379")’

Breaking Changes

• Fix distribution compatibility for linear and logarithmic distribution (#3444)
• Remove get_study_id_from_trial_id (#3538)

New Features

• Add targets argument to plot_pareto_plont of plotly backend (#3495, thanks @TakuyaInoue-github!)
• Support constraints_func in plot_pareto_front in matplotlib visualization (#3497, thanks @fukatani!)
• Calculate the feature importance with mean absolute SHAP values (#3507, thanks @liaison!)
• Make GridSampler reproducible (#3527, thanks @gasin!)
• Replace ValueError with warning in GridSearchSampler (#3545)
• Implement callbacks argument of OptunaSearchCV (#3577)
• Add option to skip table creation to RDBStorage (#3581)

Enhancements

• Set precision of sqlalchemy.Float in RDBStorage table definition (#3327)
• Accept nan in trial.report (#3348, thanks @belldandyxtq!)
• Lazy import of alembic, sqlalchemy, and scipy (#3381)
• Unify pareto front (#3389, thanks @semiexp!)
• Make set_trial_param() of RedisStorage faster (#3391, thanks @masap!)
• Make _set_best_trial() of RedisStorage faster (#3392, thanks @masap!)
• Make set_study_directions() of RedisStorage faster (#3393, thanks @masap!)
• Make optuna compatible with wandb sweep panels (#3403, thanks @captain-pool!)
• Change "#Trials" to "Trial" in plot_slice, plot_pareto_front, and plot_optimization_history (#3449, thanks @dubey-anshuman!)
• Make contour plots handle trials with nonfinite values (#3451)
• Query studies for trials only once in EDF plots (#3460)
• Make Parallel-Coordinate plots handle trials with nonfinite values (#3471, thanks @divyanshugit!)
• Separate heartbeat functionality from BaseStorage (#3475)
• Remove torch.distributed calls from TorchDistributedTrial properties (#3490, thanks @nlgranger!)
• Remove the internal logic that calculates the interaction of two or more variables in fANOVA (#3543)
• Handle inf/-inf for trial_values table in RDB (#3559)
• Add intermediate_value_type column to represent inf/-inf on RDBStorage (#3564)

Bug Fixes

• Import COLOR_SCALE inside import util context (#3492)
• Remove -v option of optuna study set-user-attr command (#3499, thanks @nyanhi!)
• Filter trials with nonfinite value in optuna.visualization.plot_param_importances and optuna.visualization.matplotlib.plot_param_importance (#3500, thanks @takoika!)
• Fix --verbose and --quiet options in CLI (#3532, thanks @nyanhi!)
• Replace ValueError with RuntimeError in get_best_trial (#3541)
• Take the same search space as in CategoricalDistribution by GridSampler (#3544)

Installation

• Partially support Python 3.10 (#3353)
• Clean up setup.py (#3517)
• Remove duplicate requirements from document section (#3613)

Documentation

• Clean up exception docstrings (#3429)
• Revise docstring in MLFlow and WandB callbacks (#3477)
• Change the parameter name from classifier to regressor in the code snippet of README.md (#3481)
• Add link to Minituna in CONTRIBUTING.md (#3482)
• Fix benchmarks/README.md for the bayesmark section (#3496)
• Mention Study.stop as a criteria to stop creating trials in document (#3498, thanks @takoika!)
• Fix minor English errors in the docstring of study.optimize (#3505)
• Add Python 3.10 in supported version in README.md (#3508)
• Remove articles at the beginning of sentences in crossovers (#3509)
• Correct FronzenTrial's docstring (#3514)
• Mention specify hyperparameter tutorial (#3515)
• Fix typo in MLFlow callback (#3533)
• Improve docstring of GridSampler's seed option (#3568)
• Add the samplers comparison table (#3571)
• Replace youtube.com with youtube-nocookie.com (#3590)
• Fix time complexity of the samplers comparison table (#3593)
• Remove language from docs configuration (#3594)

Examples

• Fix version of JAX (https://github.com/optuna/optuna-examples/pull/99)
• Remove constraints by #99 (https://github.com/optuna/optuna-examples/pull/100)
• Replace some methods in the sklearn example (https://github.com/optuna/optuna-examples/pull/102, thanks @MasahitoKumada!)
• Add Python3.10 in allennlp.yml (https://github.com/optuna/optuna-examples/pull/104)
• Remove numpy (https://github.com/optuna/optuna-examples/pull/105)
• Add python 3.10 to fastai CI (https://github.com/optuna/optuna-examples/pull/106)
• Add python 3.10 to non-integration examples CIs (https://github.com/optuna/optuna-examples/pull/107)
• Add python 3.10 to Hiplot CI (https://github.com/optuna/optuna-examples/pull/108)
• Add a comma to visualization.yml (https://github.com/optuna/optuna-examples/pull/109)
• Rename WandB example to follow naming rules (https://github.com/optuna/optuna-examples/pull/110)
• Add scikit-learn version constraint for Dask-ML (https://github.com/optuna/optuna-examples/pull/112)
• Add python 3.10 to sklearn CI (https://github.com/optuna/optuna-examples/pull/113)
• Set version constraint of protobuf in PyTorch Lightning example (https://github.com/optuna/optuna-examples/pull/116)

Tests

• Improve matplotlib parallel coordinate test (#3368)
• Save figures for all matplotlib tests (#3414, thanks @divyanshugit!)
• Add inf test to intermediate values test (#3466)
• Add test cases for test_storages.py (#3480)
• Improve the tests of optuna.visualization.plot_pareto_front (#3546)
• Move heartbeat-related tests in test_storages.py to another file (#3553)
• Use seed method of np.random.RandomState for reseeding and fix test_reseed_rng (#3569)
• Refactor test_get_observation_pairs (#3574)
• Add tests for inf/nan objectives for ShapleyImportanceEvaluator (#3576)
• Add deprecated warning test to the multi-objective sampler test file (#3601)

Code Fixes

• Ignore incomplete trials in matplotlib.plot_parallel_coordinate (#3415)
• Update warning message and add a test when a trial fails with exception (#3454)
• Remove old distributions from NSGA-II sampler (#3459)
• Remove duplicated DB access in _log_completed_trial (#3551)
• Reduce the number of copy.deepcopy() calls in importance module (#3554)
• Remove duplicated check_trial_is_updatable (#3557)
• Replace optuna.testing.integration.create_running_trial with study.ask (#3562)
• Refactor test_get_observation_pairs (#3574)
• Update label of feasible trials if constraints_func is specified (#3587)
• Replace unused variable name with underscore (#3588)
• Enable no-implicit-optional for mypy (#3599, thanks @harupy!)
• Enable warn_redundant_casts for mypy (#3602, thanks @harupy!)
• Refactor the type of value of TrialIntermediateValueModel (#3603)
• Fix broken mypy checks of Alembic's get_current_head() method (#3608)
• Sort dependencies by name (#3614)

Continuous Integration

• Introduce the benchmark for multi-objectives samplers (#3271, thanks @drumehiron!)
• Add WFG benchmark test (#3349, thanks @kei-mo!)
• Add workflow to use reviewdog (#3357)
• Add NASBench201 from NASLib (#3465)
• Fix speed benchmarks CI (#3470)
• Support PyTorch 1.11.0 (#3510)
• Restore virtualenv for benchmark extras (#3585)
• Use protobuf<4.0.0 to resolve Sphinx CI error (#3591)
• Unpin protobuf (#3598, thanks @harupy!)
• Extract MPI tests from integration CI as independent CI (#3606)

Other

• Bump up version to v3.0.0b1.dev (#3457)
• Fix kurobako benchmark code to run it locally (#3468)
• Fix label of issue template (#3493)
• Improve issue templates (#3536)
• Hotfix for fakeredis 1.7.4 release (#3549)
• Remove the version constraint of fakeredis (#3561)
• Relax version constraint of fakeredis (#3607)
• Shorten the durations of the stale bot for PRs (#3611)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @MasahitoKumada, @TakuyaInoue-github, @belldandyxtq, @c-bata, @captain-pool, @contramundum53, @divyanshugit, @drumehiron, @dubey-anshuman, @fukatani, @g-votte, @gasin, @harupy, @himkt, @hvy, @kei-mo, @keisuke-umezawa, @knshnb, @liaison, @masap, @nlgranger, @not522, @nyanhi, @nzw0301, @semiexp, @sile, @takoika, @toshihikoyanase, @xadrianzetx

- Python
Published by HideakiImamura about 4 years ago

optuna - v3.0.0-b0

This is the release note of v3.0.0-b0.

Highlights

Simplified Distribution Classes: Float, Int and Categorical

Search space definitions, which consist of BaseDistribution and its child classes in Optuna, are greatly simplified. We have introduced FloatDistribution, IntDistribution, and CategoricalDistribution. If you use the suggest API and Study.optimize, the search space information is stored as these three distributions. Previous UniformDistribution, LogUniformDistribution, DiscreteUniformDistribution, IntUniformDistribution, and IntLogUniformDistribution are deprecated. If you pass deprecated distributions to APIs such as Study.ask or create_trial, they are internally converted to corresponding FloatDistribution or IntDistribution.

Storage Database Migration

To use Optuna v3.0.0-b0 with RDBStorage that was created in the previous versions of Optuna, please run optuna storage upgrade to migrate your database.

If you use RedisStorage, copy your study with RDBStorage using copy_study with the Optuna you used to create the study, thenrun optuna storage upgrade with Optuna v3.0.0-b0. After upgrading the storage, copy the study back as a new RedisStorage.

sh python -c ‘import optuna; optuna.copy_study(from_study_name=”example”, from_storage=”redis://localhost:6379”, to_storage=”sqlite:///upgrade.db”) pip install –pre -U optuna optuna storage upgrade –storage sqlite:///upgrade.db python -c ‘import optuna; optuna.copy_study(from_study_name="example", from_storage="sqlite:///upgrade.db", to_study_name="new-example", to_storage="redis://localhost:6379")’

Consistent Ask-and-Tell Interface with Study.optimize

Study.tell fails a trial when it is called with certain invalid combinations of state and values, instead of raising an error. This change aims to make Study.tell consistent with Study.optimize, which continues an optimization even if an objective returns an invalid value.

Study.tell now also returns the resulting trial (FrozenTrial) in order to allow inspecting how the arguments were interpreted.

Before

Study.tell raises an exception when it is called with an invalid combination of state and values.

```python study.tell(study.ask(), values=None)

Traceback (most recent call last):

File "", line 1, in

File "/…/optuna/optuna/study/study.py", line 579, in tell

raise ValueError(

ValueError: No values were told. Values are required when state is TrialState.COMPLETE.

```

After

Study.tell automatically fails the trial.

python trial: FrozenTrial = study.tell(study.ask(), value=None) assert trial.state == TrialState.FAIL

See #3144 for more details.

Stable Study APIs

We are converting all positional arguments of create_study, delete_study, load_study, and copy_study to keyword-only arguments since the order of arguments were inconsistent. This is not yet a breaking-change, but if you use these features with positional arguments, then you will get a warning message to use them with keyword-only arguments.

In addition, we have fixed all of problems described in #2955, so we have stabled the Study APIs. Specifically, Study.add_trial, Study.add_trials, Study.enqueue_trial, and copy_study have been stabled.

See #3270 and #2955 for more details.

Improved Visualization

Several bugs in the visualization module have been resolved. For instance, the parallel coordinates plot ignores trials with missing parameters (#3373) and the scale of the objective value is fixed (#3369). The edf plot filters trials with inf values (#3395 and #3435).

Before: Trials with missing parameters are wrongly connected to each other.

After: Trials with missing parameters are removed from the plot.

Breaking Changes

• Add option to exclude best trials from study summaries (#3109)
• Migrate to {Float,Int}Distribution using alembic (#3113)
• Move validation logic from _run_trial to study.tell (#3144)
• Enable FloatDistribution and IntDistribution (#3246)
• Use an enqueued parameter that is out of range from suggest API (#3298)
• Convert deprecated distribution to new distribution internally (#3420)

New Features

• Add CatBoostPruningCallback (#2734, thanks @tohmae!)
• Create common API for all NSGA-II crossover operations (#3221)
• Add a history of retried trial numbers in Trial.system_attrs (#3223, thanks @belltailjp!)
• Convert all positional arguments to keyword-only (#3270, thanks @higucheese!)
• Stabilize study.py (#3309)
• Add targets and deprecate axis_order in optuna.visualization.matplotlib.plot_pareto_front (#3341, thanks @shu65!)

Enhancements

• Enable cache for study.tell() (#3265, thanks @masap!)
• Warn if heartbeat is used with ask-and-tell (#3273)
• Make optuna.study.get_all_study_summaries() of RedisStorage fast (#3278, thanks @masap!)
• Improve Ctrl-C interruption handling (#3374, thanks @CorentinNeovision!)
• Use same colormap among plotly visualization methods (#3376)
• Make EDF plots handle trials with nonfinite values (#3435)
• Make logger message optional in filter_nonfinite (#3438)

Bug Fixes

• Make TPE work with a categorical variable with different choice types (#3190, thanks @keisukefukuda!)
• Fix axis range issue in matplotlib contour plot (#3249, thanks @harupy!)
• Allow fail_state_trials show warning when heartbeat is enabled (#3301)
• Clip untransformed values sampled from int uniform distributions (#3319)
• Fix missing user_attrs and system_attrs in study summaries (#3352)
• Fix objective scale in parallel coordinate of Matplotlib (#3369)
• Fix matplotlib.plot_parallel_coordinate with log distributions (#3371)
• Fix parallel coordinate with missing value (#3373)
• Add utility to filter trials with inf values from visualizations (#3395)
• Return the best trial number, not worst trial number by best_index_ (#3410)
• Avoid using px.colors.sequential.Blues that introduces pandas dependency (#3422)
• Fix _is_reverse_scale (#3424)

Installation

• Drop TensorFlow support for Python 3.6 (#3296)
• Pin AllenNLP version (#3367)
• Skip run fastai job on Python 3.6 (#3412)
• Avoid latest click==8.1.0 that removed a deprecated feature (#3413)
• Avoid latest PyTorch lightning until integration is updated (#3417)
• Revert "Avoid latest click==8.1.0 that removed a deprecated feature" (#3430)

Documentation

• Add reference to tutorial page in CLI (#3267, thanks @tsukudamayo!)
• Carry over notes on step behavior to new distributions (#3276)
• Correct the disable condition of show_progress_bar (#3287)
• Add a document to lead FAQ and example of heartbeat (#3294)
• Add a note for copy_study: it creates a copy regardless of its state (#3295)
• Add note to recommend Python 3.8 or later in documentation build with artifacts (#3312)
• Fix crossover references in Raises doc section (#3315)
• Add reference to QMCSampler in tutorial (#3320)
• Fix layout in tutorial (with workaround) (#3322)
• Scikit-learn required for plot_param_importances (#3332, thanks @ll7!)
• Add a link to multi-objective tutorial from a pareto front page (#3339, thanks @kei-mo!)
• Add reference to tutorial page in visualization (#3340, thanks @Hiroyuki-01!)
• Mention tutorials of User-Defined Sampler/Pruner from the API reference pages (#3342, thanks @hppRC!)
• Add reference to saving/resuming study with RDB backend (#3345, thanks @Hiroyuki-01!)
• Fix a typo (#3360)
• Remove deprecated command optuna study optimize in FAQ (#3364)
• Fix nit typo (#3380)
• Add see also section for best_trial (#3396, thanks @divyanshugit!)
• Updates the tutorial page for re-use the best trial (#3398, thanks @divyanshugit!)
• Add explanation about Study.best_trials in multi-objective optimization tutorial (#3443)

Examples

• Remove Python 3.6 from haiku's CI (https://github.com/optuna/optuna-examples/pull/83)
• Apply black 22.1.0 & run checks daily (https://github.com/optuna/optuna-examples/pull/84)
• Add hiplot example (https://github.com/optuna/optuna-examples/pull/86)
• Stop running jobs using TF with Python3.6 (https://github.com/optuna/optuna-examples/pull/87)
• Pin AllenNLP version (https://github.com/optuna/optuna-examples/pull/89)
• Add Medium link (https://github.com/optuna/optuna-examples/pull/91)
• Use official CatBoostPruningCallback (https://github.com/optuna/optuna-examples/pull/92)
• Stop running fastai job on Python 3.6 (https://github.com/optuna/optuna-examples/pull/93)
• Specify Python version using str in workflow files (https://github.com/optuna/optuna-examples/pull/95)
• Introduce upper version constraint of PyTorchLightning (https://github.com/optuna/optuna-examples/pull/96)
• Update SimulatedAnnealingSampler to support FloatDistribution (https://github.com/optuna/optuna-examples/pull/97)

Tests

• Add plot value tests to matplotlib_tests/test_param_importances (#3180, thanks @belldandyxtq!)
• Make tests of plot_optimization_history methods consistent (#3234)
• Add integration test for RedisStorage (#3258, thanks @masap!)
• Change the order of arguments in the catalyst integration test (#3308)
• Cleanup MLflowCallback tests (#3378)
• Test serialize/deserialize storage on parametrized conditions (#3407)
• Add tests for parameter of 'None' for TPE (#3447)

Code Fixes

• Switch to IntDistribution (#3181, thanks @nyanhi!)
• Fix type hints for Python 3.8 (#3240)
• Remove UniformDistribution, LogUniformDistribution and DiscreteUniformDistribution code paths (#3275)
• Merge set_trial_state() and set_trial_values() into one function (#3323, thanks @masap!)
• Follow up for {Float, Int}Distributions (#3337, thanks @nyanhi!)
• Move the get_trial_xxx abstract functions to base (#3338, thanks @belldandyxtq!)
• Update type hints of states (#3359, thanks @BasLaa!)
• Remove unused function from RedisStorage (#3394, thanks @masap!)
• Remove unnecessary string concatenation (#3406)
• Follow coding style and fix typos in tests/integration_tests (#3408)
• Fix log message formatting in filter_nonfinite (#3436)
• Add RetryFailedTrialCallback to optuna.storages.* (#3441)
• Unify fail_stale_trials in each storage implementation (#3442, thanks @knshnb!)

Continuous Integration

• Add performance benchmarks using bayesmark (#3354)
• Fix speed benchmarks (#3362)
• Pin setuptools (#3427)

Other

• Bump up version to v3.0.0b0.dev (#3289)
• Add description field for question-and-help-support (#3305)
• Update README to inform v3.0.0a2 (#3314)
• Add Optuna-related URLs for PyPi (#3355, thanks @andriyor!)
• Bump Optuna to v3.0.0-b0 (#3458)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@BasLaa, @CorentinNeovision, @HideakiImamura, @Hiroyuki-01, @andriyor, @belldandyxtq, @belltailjp, @contramundum53, @divyanshugit, @harupy, @higucheese, @himkt, @hppRC, @hvy, @kei-mo, @keisuke-umezawa, @keisukefukuda, @knshnb, @ll7, @masap, @not522, @nyanhi, @nzw0301, @shu65, @sile, @tohmae, @toshihikoyanase, @tsukudamayo, @xadrianzetx

- Python
Published by HideakiImamura about 4 years ago

optuna - v3.0.0-a2

This is the release note of v3.0.0-a2.

Highlights

Study.optimize Warning Configuration Fix

This is a small release that fixes a bug that the same warning message was emitted more than once when calling Study.optimize.

Bug Fixes

• [Backport] Allow fail_state_trials show warning when heartbeat is enabled (#3303)

Other

• Bump Optuna (#3302)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@HideakiImamura, @himkt

- Python
Published by himkt over 4 years ago

optuna - v3.0.0-a1

This is the release note of v3.0.0-a1.

Highlights

Second alpha pre-release in preparation for the upcoming major version update v3.

Included are several new features, improved optimization algorithms, removals of deprecated interfaces and many quality of life improvements.

To read about the entire v3 roadmap, please refer to the Wiki.

While this is a pre-release, we encourage users to keep using the latest releases of Optuna, including this one, for a smoother transition to the coming major release. Early feedback is welcome!

A New Algorithm: Quasi-Monte Carlo Sampler

Now, you can utilize a new sampling algorithm based on the Quasi-Monte Carlo method, optuna.samplers.QMCSampler. This is oftentimes a good alternative to the existing optuna.samplers.RandomSampler. The generated (sampled) sequences have lower discrepancies compared to the standard random sequences, which are sampled uniformly. The following figures show the performance comparison to other existing samplers. Note that this algorithm is only supported for python >= 3.7.

See #2423 for more details.

Parkinson in HPOBench | Slice in HPOBench -- | -- |

Constraints Support for Pareto-front plot

The Pareto front plot now supports visualization of constrained optimization. In Optuna, NSGAIISampler and BoTorchSampler allow constrained optimization by taking a function constraints_func as argument, then examine whether trials are feasible or not. The optuna.visualization.plot_pareto_front receives a similar function and uses this function to plot the trials in different colors depending on whether they violate the constraints or not.

See #3128 for more details.

```python def objective(trial): # Binh and Korn function with constraints. x = trial.suggestfloat("x", -15, 30) y = trial.suggestfloat("y", -15, 30)

# Store the constraints as user attributes so that they can be restored after optimization.
c0 = (x - 5) ** 2 + y ** 2 - 25
c1 = -((x - 8) ** 2) - (y + 3) ** 2 + 7.7
trial.set_user_attr("constraints", (c0, c1))

v0 = 4 * x ** 2 + 4 * y ** 2
v1 = (x - 5) ** 2 + (y - 5) ** 2

return v0, v1

def constraints(trial): return trial.user_attrs["constraints"]

if name == "main": sampler = optuna.samplers.NSGAIISampler( constraintsfunc=constraints, ) study = optuna.createstudy( directions=["minimize", "minimize"], sampler=sampler, ) study.optimize(objective, n_trials=1000)

optuna.visualization.plot_pareto_front(study, constraints_func=constraints).show()

```

Distribution Cleanup

We are actively working on cleaning up distributions for integer and floating-point. In Optuna v3, these distribution are unified to optuna.distributions.IntDistribution and optuna.distributions.FloatDistribution. v3.0.0-a1 contains several changes for this project and you will temporarily see UserWarning when you call Trial.suggest_int and Trial.suggest_float. We apologize for the inconvenience and the warning will be removed from the next release.

See #2941 for more information.

Stabilization of Experimental Modules

We make AllenNLP integration and FrozenTrial.create_trial stable.

See #3196 and #3228 for more information

Breaking Changes

• Remove type_checking.py (#3235)

New Features

• Add QMC sampler (#2423, thanks @kstoneriv3!)
• Refactor pareto front and support constraints_func in plot_pareto_front (#3128, thanks @semiexp!)
• Add skip_if_finished flag to Study.tell (#3150, thanks @xadrianzetx!)
• Add user_attrs argument to Study.enqueue_trial (#3185, thanks @knshnb!)
• Option to inherit intermediate values in RetryFailedTrialCallback (#3269, thanks @knshnb!)
• Add setter method for DiscreteUniformDistribution.q (#3283)
• Stabilize allennlp integrations (#3228)
• Stabilize create_trial (#3196)

Enhancements

• Reduce number of queries to fetch directions, user_attrs and system_attrs of study summaries (#3108)
• Support FloatDistribution across codebase (#3111, thanks @xadrianzetx!)
• Use json.loads to decode pruner configuration loaded from environment variables (#3114)
• Show progress bar based on timeout (#3115, thanks @xadrianzetx!)
• Support IntDistribution across codebase (#3126, thanks @nyanhi!)
• Make progress bar available with n_jobs!=1 (#3138, thanks @masap!)
• Wrap RedisStorage in CachedStorage (#3204, thanks @masap!)
• Use functools.wraps in track_in_mlflow decorator (#3216)
• Make RedisStorage fast when running multiple trials (#3262, thanks @masap!)
• Reduce database query result for Study.ask() (#3274, thanks @masap!)

Bug Fixes

• Fix bug of nondeterministic behavior of TPESampler when group=True (#3187, thanks @xuzijian629!)
• Handle non-numerical params in matplotlib.contour_plot (#3213, thanks @xadrianzetx!)
• Fix log scale axes padding in matplotlib.contour_plot (#3218, thanks @xadrianzetx!)
• Handle -inf and inf values in RDBStorage (#3238, thanks @xadrianzetx!)
• Skip limiting the value if it is nan (#3286)

Installation

• Bump to torch related packages (#3156)
• Use pytorch-lightning>=1.5.0 (#3157)
• Remove testoutput from doctest of mlflow integration (#3170)
• Restrict nltk version (#3201)
• Add version constraints of setuptools (#3207)
• Remove version constraint of setuptools (#3231)
• Remove Sphinx version constraint (#3237)

Documentation

• Add a note logging_callback only works in single process situation (#3143)
• Correct FrozenTrial's docstring (#3161)
• Promote to use of v3.0.0a0 in README.md (#3167)
• Mention tutorial of callback for Study.optimize from API page (#3171, thanks @xuzijian629!)
• Add reference to tutorial page in study.enqueue_trial (#3172, thanks @knshnb!)
• Fix typo in specify_params (#3174, thanks @knshnb!)
• Guide to tutorial of Multi-objective Optimization in visualization tutorial (#3182, thanks @xuzijian629!)
• Add explanation about Parallelize Optimization at FAQ (#3186, thanks @MasahitoKumada!)
• Add order in tutorial (#3193, thanks @makinzm!)
• Fix inconsistency in distributions documentation (#3222, thanks @xadrianzetx!)
• Add FAQ entry for heartbeat (#3229)
• Replace AUC with accuracy in docs (#3242)
• Fix Raises section of FloatDistribution docstring (#3248, thanks @xadrianzetx!)
• Add {Float,Int}Distribution to docs (#3252)
• Update explanation for metrics of AllenNLPExecutor (#3253)
• Add missing cli methods to the list (#3268)
• Add docstring for property DiscreteUniformDistribution.q (#3279)

Examples

• Add pytorch-lightning DDP example (https://github.com/optuna/optuna-examples/pull/43, thanks @tohmae!)
• Install latest AllenNLP (https://github.com/optuna/optuna-examples/pull/73)
• Restrict nltk version (https://github.com/optuna/optuna-examples/pull/75)
• Add version constraints of setuptools (https://github.com/optuna/optuna-examples/pull/76)
• Remove constraint of setuptools (https://github.com/optuna/optuna-examples/pull/79)

Tests

• Add tests for transformer with upper bound parameter (#3163)
• Add tests in visualization_tests/matplotlib_tests/test_slice.py (#3175, thanks @keisukefukuda!)
• Add test case of the value in optimization history with matplotlib (#3176, thanks @TakuyaInoue-github!)
• Add tests for generated plots of matplotlib.plot_edf (#3178, thanks @makinzm!)
• Improve pareto front figure tests for matplotlib (#3183, thanks @akawashiro!)
• Add tests for generated plots of plot_edf (#3188, thanks @makinzm!)
• Match contour tests between Plotly and Matplotlib (#3192, thanks @belldandyxtq!)
• Implement missing matplotlib.contour_plot test (#3232, thanks @xadrianzetx!)
• Unify the validation function of edf value between visualization backends (#3233)
• Add test for default grace period (#3263, thanks @masap!)
• Add the missing tests of Plotly's plot_parallel_coordinate (#3266, thanks @MasahitoKumada!)
• Switch function order progbar tests (#3280, thanks @BasLaa!)

Code Fixes

• Black fix (#3147)
• Switch to FloatDistribution (#3166, thanks @xadrianzetx!)
• Remove deprecated decorator of the feature of n_jobs (#3173, thanks @MasahitoKumada!)
• Fix black and blackdoc errors (#3260, thanks @masap!)
• Remove experimental label from MaxTrialsCallback (#3261, thanks @knshnb!)
• Remove redundant _check_trial_id (#3264, thanks @masap!)
• Make existing int/float distributions wrapper of {Int,Float}Distribution (#3244)

Continuous Integration

• Use python 3.8 for CI and docker (#3026)
• Add performance benchmarks using kurobako (#3155)
• Use Python 3.7 in checks CI job (#3239)

Other

• Bump up version to v3.0.0a1.dev (#3142)
• Introduce a form to make TODOs explicit when creating issues (#3169)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@BasLaa, @HideakiImamura, @MasahitoKumada, @TakuyaInoue-github, @akawashiro, @belldandyxtq, @g-votte, @himkt, @hvy, @keisuke-umezawa, @keisukefukuda, @knshnb, @kstoneriv3, @makinzm, @masap, @not522, @nyanhi, @nzw0301, @semiexp, @tohmae, @toshihikoyanase, @tupui, @xadrianzetx, @xuzijian629

- Python
Published by himkt over 4 years ago

optuna - v3.0.0-a0

This is the release note of v3.0.0-a0.

Highlights

First alpha pre-release in preparation for the upcoming major version update v3.

Included are several new features, improved optimization algorithms, removals of deprecated interfaces and many quality of life improvements.

To read about the entire v3 roadmap, please refer to the Wiki.

While this is a pre-release, we encourage users to keep using the latest releases of Optuna, including this one, for a smoother transition to the coming major release. Early feedback is welcome!

CLI Improvements

Optuna CLI speed and usability improvements. Previously, it took several seconds to launch a CLI command, #3000 significantly speeds up the commands by halving the module load time.

The usability of the ask-and-tell interface is also improved. The ask command allows users to define search space with short and simple JSON strings after #2905. The tell command supports --skip-if-finished which ignores duplicated reports of values and statuses instead of raising errors. It for instance improves robustness against pod retries on cluster environments. See #2905.

Before: console $ optuna ask --storage sqlite:///mystorage.db --study-name mystudy \ --search-space '{"x": {"name": "UniformDistribution", "attributes": {"low": 0.0, "high": 1.0}}}'

After: console $ optuna ask --storage sqlite:///mystorage.db --study-name mystudy \ --search-space '{"x": {"type": "float", "low": 0.0, "high": 1.0}}'

New NSGA-II Crossover Options

The optimization performance of NSGA-II has been greatly improved for real-valued problems. We introduce the crossover argument in NSGAIISampler. You can select several variants of the crossover option from uniform (default), blxalpha, sbx, vsbx, undx, and spx.

The following figure shows that the newly introduced crossover algorithms perform better than existing algorithms, that is, the uniform crossover algorithm and the Gaussian process based algorithm, in terms of biasness, convergence, and diversity. Note that the previous method, other implementations (in kurobako), and the default of the new method are based on uniform crossover.

See #2903 for more information.

New History Visualization with Multiple Studies

The optimization history plot now supports visualization of multiple studies. It receives a list of studies. If the error_bar option is False, it outputs those histories in one figure. If the error_bar option is True, it calculates and shows the means and the standard deviations of those histories.

See #2807 for more details.

```python import optuna

def objective(trial): return trial.suggest_float("x", 0, 1) ** 2

nstudies = 5 studies = [optuna.createstudy(studyname=f"{i}th-study") for i in range(nstudies)] for study in studies: study.optimize(objective, n_trials=20)

This generates the first figure.

fig = optuna.visualization.plotoptimizationhistory(studies) fig.write_image("./multiple.png")

This generates the second figure.

fig = optuna.visualization.plotoptimizationhistory(studies, errorbar=True) fig.writeimage("./error_bar.png") ```

AllenNLP Distributed Pruning

The AllenNLP integration supports pruning in distributed environments. This change enables users to use the optuna_pruner callback option along with the distributed option as can be seen in the following training configuration. See #2977.

yaml ... trainer: { optimizer: 'adam', cuda_device: -1, callbacks: [ { type: 'optuna_pruner', } ], }, distributed: { cuda_devices: [-1, -1], },

Preparations for Unification of Distributions Classes

There are several implementations of BaseDistribution in Optuna, such as UniformDistribution, DiscreteUniformDistribution, IntUniformDistribution, CategoricalDistribution, This release includes part of ongoing work in reducing the number of these distribution classes to just FloatDistribution, IntDistribution, and CategoricalDistribution, aligning the classes to the trial suggest interface (suggest_float, suggest_int, and suggest_categorical). Please note that users are not recommended to use these distributions yet, because samplers haven’t been updated to support those. See #3063 for more details.

Breaking Changes

Some deprecated features including the optuna.structs module, LightGBMTuner.best_booster, and the optuna dashboard command are removed in #3057 and #3058. If you use such features please migrate to the new ones.

| Removed APIs | Corresponding active APIs | | --- | --- | | optuna.structs.StudyDirection | optuna.study.StudyDirection | | optuna.structs.StudySummary | optuna.study.StudySummary | | optuna.structs.FrozenTrial | optuna.trial.FrozenTrial | | optuna.structs.TrialState | optuna.trial.TrialState | | optuna.structs.TrialPruned | optuna.exceptions.TrialPruned | | optuna.integration.lightgbm.LightGBMTuner.best_booster | optuna.integration.lightgbm.LightGBMTuner.get_best_booster | | optuna dashboard | optuna-dashboard |

• Unify suggest APIs for floating-point parameters (#2990, thanks @xadrianzetx!)
• Clean up deprecated features (#3057, thanks @nuka137!)
• Remove optuna dashboard (#3058)

New Features

• Add interval for LightGBM callback (#2490)
• Allow multiple studies and add error bar option to plot_optimization_history (#2807)
• Support PyTorch-lightning DDP training (#2849, thanks @tohmae!)
• Add crossover operators for NSGA-II (#2903, thanks @yoshinobc!)
• Add abbreviated JSON formats of distributions (#2905)
• Extend MLflowCallback interface (#2912, thanks @xadrianzetx!)
• Support AllenNLP distributed pruning (#2977)
• Make trial.user_attrs logging optional in MLflowCallback (#3043, thanks @xadrianzetx!)
• Support multiple input of studies when plot with Matplotlib (#3062, thanks @TakuyaInoue-github!)
• Add IntDistribution & FloatDistribution (#3063, thanks @nyanhi!)
• Add trial.user_attrs to pareto_front hover text (#3082, thanks @kasparthommen!)
• Support error bar for Matplotlib (#3122, thanks @TakuyaInoue-github!)
• Add optuna tell with --skip-if-finished (#3131)

Enhancements

• Add single distribution support to BoTorchSampler (#2928)
• Speed up import optuna (#3000)
• Fix _contains of IntLogUniformDistribution (#3005)
• Render importance scores next to bars in matplotlib.plot_param_importances (#3012, thanks @xadrianzetx!)
• Make default value of verbose_eval NoneN forLightGBMTuner/LightGBMTunerCV` to avoid conflict (#3014, thanks @chezou!)
• Unify colormap of plot_contour (#3017)
• Relax FixedTrial and FrozenTrial allowing not-contained parameters during suggest_* (#3018)
• Raise errors if optuna ask CLI receives --sampler-kwargs without --sampler (#3029)
• Remove _get_removed_version_from_deprecated_version function (#3065, thanks @nuka137!)
• Reformat labels for small importance scores in plotly.plot_param_importances (#3073, thanks @xadrianzetx!)
• Speed up Matplotlib backend plot_contour using SciPy's spsolve (#3092)
• Remove updates in cached storage (#3120, thanks @shu65!)

Bug Fixes

• Add tests of sample_relative and fix type of return values of SkoptSampler and PyCmaSampler (#2897)
• Fix GridSampler with RetryFailedTrialCallback or enqueue_trial (#2946)
• Fix the type of trial.values in MLflow integration (#2991)
• Fix to raise ValueError for invalid q in DiscreteUniformDistribution (#3001)
• Do not call trial.report during sanity check (#3002)
• Fix matplotlib.plot_contour bug (#3046, thanks @IEP!)
• Handle single distributions in fANOVA evaluator (#3085, thanks @xadrianzetx!)

Installation

• Support scikit-learn v1.0.0 (#3003)
• Pin tensorflow and tensorflow-estimator versions to <2.7.0 (#3059)
• Add upper version constraint of PyTorchLightning (#3077)
• Pin keras version to <2.7.0 (#3078)
• Remove version constraints of tensorflow (#3084)

Documentation

• Add note of the behavior when calling multiple trial.report (#2980)
• Add note for DDP training of pytorch-lightning (#2984)
• Add note to OptunaSearchCV about direction (#3007)
• Clarify n_trials in the docs (#3016, thanks @Rohan138!)
• Add a note to use pickle with different optuna versions (#3034)
• Unify the visualization docs (#3041, thanks @sidshrivastav!)
• Fix a grammatical error in FAQ doc (#3051, thanks @belldandyxtq!)
• Less ambiguous documentation for optuna tell (#3052)
• Add example for logging.set_verbosity (#3061, thanks @drumehiron!)
• Mention the tutorial of 002_configurations.py in the Trial API page (#3067, thanks @makkimaki!)
• Mention the tutorial of 003_efficient_optimization_algorithms.py in the Trial API page (#3068, thanks @makkimaki!)
• Add link from set_user_attrs in Study to the user_attrs entry in Tutorial (#3069, thanks @MasahitoKumada!)
• Update description for missing samplers and pruners (#3087, thanks @masaaldosey!)
• Simplify the unit testing explanation (#3089)
• Fix range description in suggest_float docstring (#3091, thanks @xadrianzetx!)
• Fix documentation for the package installation procedure on different OS (#3118, thanks @masap!)
• Add description of ValueError and TypeErorr to Raises section of Trial.report (#3124, thanks @MasahitoKumada!)

Examples

• Use RetryFailedTrialCallback in pytorch_checkpoint example (https://github.com/optuna/optuna-examples/pull/59, thanks @xadrianzetx!)
• Add Python 3.9 to CI yaml files (https://github.com/optuna/optuna-examples/pull/61)
• Replace suggest_uniform with suggest_float (https://github.com/optuna/optuna-examples/pull/63)
• Remove deprecated warning message in lightgbm (https://github.com/optuna/optuna-examples/pull/64)
• Pin tensorflow and tensorflow-estimator versions to <2.7.0 (https://github.com/optuna/optuna-examples/pull/66)
• Restrict upper version of pytorch-lightning (https://github.com/optuna/optuna-examples/pull/67)
• Add an external resource to README.md (https://github.com/optuna/optuna-examples/pull/68, thanks @solegalli!)

Tests

• Add test case of samplers for conditional objective function (#2904)
• Test int distributions with default step (#2924)
• Be aware of trial preparation when checking heartbeat interval (#2982)
• Simplify the DDP model definition in the test of pytorch-lightning (#2983)
• Wrap data with np.asarray in lightgbm test (#2997)
• Patch calls to deprecated suggest APIs across codebase (#3027, thanks @xadrianzetx!)
• Make return_cvbooster of LightGBMTuner consistent to the original value (#3070, thanks @abatomunkuev!)
• Fix parametrize_sampler (#3080)
• Fix verbosity for tests/integration_tests/lightgbm_tuner_tests/test_optimize.py (#3086, thanks @nyanhi!)
• Generalize empty search space test case to all hyperparameter importance evaluators (#3096, thanks @xadrianzetx!)
• Check if texts in legend by order agnostic way (#3103)
• Add tests for axis scales to matplotlib.plot_slice (#3121)

Code Fixes

• Add test case of samplers for conditional objective function (#2904)
• Fix #2949, remove BaseStudy (#2986, thanks @twsl!)
• Use optuna.load_study in optuna ask CLI to omit direction/directions option (#2989)
• Fix typo in Trial warning message (#3008, thanks @xadrianzetx!)
• Replaces boston dataset with california housing dataset (#3011, thanks @avats-dev!)
• Fix deprecation version of suggest APIs (#3054, thanks @xadrianzetx!)
• Add remove_version to the missing @deprecated argument (#3064, thanks @nuka137!)
• Add example of optuna.logging.get_verbosity (#3066, thanks @MasahitoKumada!)
• Support {Float|Int}Distribution in NSGA-II crossover operators (#3139, thanks @xadrianzetx!)

Continuous Integration

• Install botorch to CI jobs on mac (#2988)
• Use libomp 11.1.0 for Mac (#3024)
• Run mac-tests CI at a scheduled time (#3028)
• Set concurrency to github workflows (#3095)
• Skip CLI tests when calculating the coverage (#3097)
• Migrate mypy version to 0.910 (#3123)
• Avoid installing the latest MLfow to prevent doctests from failing (#3135)

Other

• Bump up version to 2.11.0dev (#2976)
• Add roadmap news to README.md (#2999)
• Bump up version number to 3.0.0a1.dev (#3006)
• Add Python 3.9 to tox.ini (#3025)
• Fix version number to 3.0.0a0 (#3140)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Crissman, @HideakiImamura, @IEP, @MasahitoKumada, @Rohan138, @TakuyaInoue-github, @abatomunkuev, @avats-dev, @belldandyxtq, @chezou, @drumehiron, @g-votte, @himkt, @hvy, @kasparthommen, @keisuke-umezawa, @makkimaki, @masaaldosey, @masap, @not522, @nuka137, @nyanhi, @nzw0301, @shu65, @sidshrivastav, @sile, @solegalli, @tohmae, @toshihikoyanase, @twsl, @xadrianzetx, @yoshinobc, @ytsmiling

- Python
Published by HideakiImamura over 4 years ago

optuna - v2.10.0

This is the release note of v2.10.0.

Highlights

New CLI Subcommand for Analyzing Studies

New subcommands optuna trials, optuna best-trial and optuna best-trials have been introduced to Optuna’s CLI for listing trials in studies with RDB storages. It allows direct interaction with trial data from the command line in various formats including human readable tables, JSON or YAML. See the following examples:

Show all trials in a study.

console $ optuna trials --storage sqlite:///example.db --study-name example +--------+---------------------+---------------------+---------------------+----------------+---------------------+----------+ | number | value | datetime_start | datetime_complete | duration | params | state | +--------+---------------------+---------------------+---------------------+----------------+---------------------+----------+ | 0 | 0.6098421143538713 | 2021-10-01 14:36:46 | 2021-10-01 14:36:46 | 0:00:00.026059 | {'x': 'A', 'y': 6} | COMPLETE | | 1 | 0.6584108953598753 | 2021-10-01 14:36:46 | 2021-10-01 14:36:46 | 0:00:00.023447 | {'x': 'A', 'y': 10} | COMPLETE | | 2 | 0.612883262548314 | 2021-10-01 14:36:46 | 2021-10-01 14:36:46 | 0:00:00.021577 | {'x': 'C', 'y': 3} | COMPLETE | | 3 | 0.09326753798819143 | 2021-10-01 14:36:46 | 2021-10-01 14:36:46 | 0:00:00.024183 | {'x': 'A', 'y': 0} | COMPLETE | | 4 | 0.7316749689191168 | 2021-10-01 14:36:46 | 2021-10-01 14:36:46 | 0:00:00.021994 | {'x': 'C', 'y': 4} | COMPLETE | +--------+---------------------+---------------------+---------------------+----------------+---------------------+----------+

Show the best trial as YAML.

console $ optuna best-trial --storage sqlite:///example.db --study-name example --format yaml datetime_complete: '2021-10-01 14:36:46' datetime_start: '2021-10-01 14:36:46' duration: '0:00:00.024183' number: 3 params: x: A y: 0 state: COMPLETE value: 0.09326753798819143

Show the best trials of multi-objective optimization and train a neural network with one of the best parameters.

```console $ STORAGE=sqlite:///example.db $ STUDYNAME=example-mo $ optuna best-trials --storage $STORAGE --study-name $STUDYNAME +--------+-------------------------------------------+---------------------+---------------------+----------------+--------------------------------------------------+----------+ | number | values | datetimestart | datetimecomplete | duration | params | state | +--------+-------------------------------------------+---------------------+---------------------+----------------+--------------------------------------------------+----------+ | 0 | [0.23884292794146034, 0.6905832476748404] | 2021-10-01 15:02:32 | 2021-10-01 15:02:32 | 0:00:00.035815 | {'lr': 0.05318673615579818, 'optimizer': 'adam'} | COMPLETE | | 2 | [0.3157886300888031, 0.05110976427394465] | 2021-10-01 15:02:32 | 2021-10-01 15:02:32 | 0:00:00.030019 | {'lr': 0.08044012012204389, 'optimizer': 'sgd'} | COMPLETE | +--------+-------------------------------------------+---------------------+---------------------+----------------+--------------------------------------------------+----------+

$ optuna best-trials --storage $STORAGE --study-name $STUDY_NAME --format json > result.json $ OPTIMIZER=jq '.[0].params.optimizer' result.json $ LR=jq '.[0].params.lr' result.json $ python train.py $OPTIMIZER $LR ```

See #2847 for more details.

Multi-objective Optimization Support of Weights & Biases and MLflow Integrations

Weights & Biases and MLflow integration modules support tracking multi-objective optimization. Now, they accept arbitrary numbers of objective values with metric names.

Weights & Biases

```python from optuna.integration import WeightsAndBiasesCallback

wandbc = WeightsAndBiasesCallback(metric_name=["mse", "mae"])

...

study = optuna.createstudy(directions=["minimize", "minimize"]) study.optimize(objective, ntrials=100, callbacks=[wandbc]) ```

MLflow

```python from optuna.integration import MLflowCallback

mlflc = MLflowCallback(metric_name=["accuracy", "latency"])

...

study = optuna.createstudy(directions=["minimize", "minimize"]) study.optimize(objective, ntrials=100, callbacks=[mlflc]) ```

See #2835 and #2863 for more details.

Breaking Changes

• Align CLI output format (#2882)
  • In particular, the return format of optuna ask has been simplified. The first layer of nesting with the key “trial” is removed. Parsing can be simplified from jq ‘.trial.params’ to jq ‘.params’.

New Features

• Support multi-objective optimization in WeightsAndBiasesCallback (#2835, thanks @xadrianzetx!)
• Introduce trials CLI (#2847)
• Support multi-objective optimization in MLflowCallback (#2863, thanks @xadrianzetx!)

Enhancements

• Add Plotly-like interpolation algorithm to optuna.visualization.matplotlib.plot_contour (#2810, thanks @xadrianzetx!)
• Sort values when the categorical values is numerical in plot_parallel_coordinate (#2821, thanks @TakuyaInoue-github!)
• Refactor MLflowCallback (#2855, thanks @xadrianzetx!)
• Minor refactoring of plot_parallel_coordinate (#2856)
• Update sklearn.py (#2966, thanks @Garve!)

Bug Fixes

• Fix datetime_complete in _CachedStorage (#2846)
• Hyperband no longer assumes it is the only pruner (#2879, thanks @cowwoc!)
• Fix method untransform of _SearchSpaceTransform with distribution.single() == True (#2947, thanks @yoshinobc!)

Installation

• Avoid keras 2.6.0 (#2851)
• Drop tensorflow and keras version constraints (#2852)
• Avoid latest allennlp==2.7.0 (#2894)
• Introduce the version constraint of scikit-learn (#2953)

Documentation

• Fix bounds' shape in the document (#2830)
• Simplify documentation of FrozenTrial (#2833)
• Fix typo: replace CirclCI with CircleCI (#2840)
• Added alternative callback function #2844 (#2845, thanks @DeviousLab!)
• Update URL of cmaes repository (#2857)
• Improve the docstring of MLflowCallback (#2883)
• Fix create_trial document (#2888)
• Fix an argument in docstring of _CachedStorage (#2917)
• Use :obj: for True, False, and None instead of inline code (#2922)
• Use inline code syntax for constraints_func (#2930)
• Add link to Weights & Biases example (#2962, thanks @xadrianzetx!)

Examples

• Do not use latest keras==2.6.0 (https://github.com/optuna/optuna-examples/pull/44)
• Fix typo in Dask-ML GitHub Action workflow (https://github.com/optuna/optuna-examples/pull/45, thanks @jrbourbeau!)
• Support Python 3.9 for TensorFlow and MLFlow (https://github.com/optuna/optuna-examples/pull/47)
• Replace deprecated argument lr with learning_rate in tf.keras (https://github.com/optuna/optuna-examples/pull/51)
• Avoid latest allennlp==2.7.0 (https://github.com/optuna/optuna-examples/pull/52)
• Save checkpoint to tmpfile and rename it (https://github.com/optuna/optuna-examples/pull/53)
• PyTorch checkpoint cosmetics (https://github.com/optuna/optuna-examples/pull/54)
• Add Weights & Biases example (https://github.com/optuna/optuna-examples/pull/55, thanks @xadrianzetx!)
• Use MLflowCallback in MLflow example (https://github.com/optuna/optuna-examples/pull/58, thanks @xadrianzetx!)

Tests

• Fixed relational operator not including 1 (#2865, thanks @Yu212!)
• Add scenario tests for samplers (#2869)
• Add test cases for storage upgrade (#2890)
• Add test cases for show_progress_bar of optimize (#2900, thanks @xadrianzetx!)
• Speed-up sampler tests by using random sampling of skopt (#2910)
• Fixes namedtuple type name (#2961, thanks @sobolevn!)

Code Fixes

• Changed y-axis and x-axis access according to matplotlib docs (#2834, thanks @01-vyom!)
• Fix a BoTorch deprecation warning (#2861)
• Relax metric name type hinting in WeightsAndBiasesCallback (#2884, thanks @xadrianzetx!)
• Fix recent alembic 1.7.0 type hint error (#2887)
• Remove old unused Trial._after_func method (#2899)
• Fixes namedtuple type name (#2961, thanks @sobolevn!)

Continuous Integration

• Enable act to run for other workflows (#2656)
• Drop tensorflow and keras version constraints (#2852)
• Avoid segmentation fault of test_lightgbm.py on macOS (#2896)

Other

• Preinstall RDB binding Python libraries in Docker image (#2818)
• Bump to v2.10.0.dev (#2829)
• Bump to v2.10.0 (#2975)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@01-vyom, @Crissman, @DeviousLab, @Garve, @HideakiImamura, @TakuyaInoue-github, @Yu212, @c-bata, @cowwoc, @himkt, @hvy, @jrbourbeau, @keisuke-umezawa, @not522, @nzw0301, @sobolevn, @toshihikoyanase, @xadrianzetx, @yoshinobc

- Python
Published by hvy over 4 years ago

optuna - v2.9.1

This is the release note of v2.9.1.

Highlights

Ask-and-Tell CLI Fix

The storage URI and the study name are no longer logged by optuna ask and optuna tell. The former could contain sensitive information.

Enhancements

• Remove storage URI from ask and tell CLI subcommands (#2838)

Other

• Bump to v2.9.1 (#2839)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@himkt, @hvy, @not522

- Python
Published by hvy almost 5 years ago

optuna - v2.9.0

This is the release note of v2.9.0.

Help us create the next version of Optuna! Please take a few minutes to fill in this survey, and let us know how you use Optuna now and what improvements you'd like. https://forms.gle/TtJuuaqFqtjmbCP67

Highlights

Ask-and-Tell CLI: Optuna from the Command Line

The built-in CLI which you can use to upgrade storages or check the installed version with optuna --version, now provides experimental subcommands for the Ask-and-Tell interface. It is now possible to optimize using Optuna entirely from the CLI, without writing a single line of Python.

Ask with optuna ask

Ask for parameters using optuna ask, specifying the search space, storage, study name, sampler and optimization direction. The parameters and the associated trial number can be output as either JSON or YAML.

The following is an example outputting and piping the results to a YAML file.

```console $ optuna ask --storage sqlite:///mystorage.db \ --study-name mystudy \ --sampler TPESampler \ --sampler-kwargs '{"multivariate": true}' \ --search-space '{"x": {"name": "UniformDistribution", "attributes": {"low": 0.0, "high": 1.0}}, "y": {"name": "CategoricalDistribution", "attributes": {"choices": ["foo", "bar"]}}}' \ --direction minimize \ --out yaml \ > out.yaml [I 2021-07-30 15:56:50,774] A new study created in RDB with name: mystudy [I 2021-07-30 15:56:50,808] Asked trial 0 with parameters {'x': 0.21492964898919975, 'y': 'foo'} in study 'mystudy' and storage 'sqlite:///mystorage.db'.

$ cat out.yaml trial: number: 0 params: x: 0.21492964898919975 y: foo ```

Specify multiple whitespace separated directions for multi-objective optimization.

Tell with optuna tell

After computing the objective value based on the output of ask, you can report the result back using optuna tell and it will be stored in the study.

console $ optuna tell --storage sqlite:///mystorage.db \ --study-name mystudy \ --trial-number 0 \ --values 1.0 [I 2021-07-30 16:01:13,039] Told trial 0 with values [1.0] and state TrialState.COMPLETE in study 'mystudy' and storage 'sqlite:///mystorage.db'.

Specify multiple whitespace separated values for multi-objective optimization.

See https://github.com/optuna/optuna/pull/2817 for details.

Weights & Biases Integration

WeightsAndBiasesCallback is a new study optimization callback that allows logging with Weights & Biases. This allows utilizing Weight & Biases’ rich visualization features to analyze studies to complement Optuna’s visualization.

```python import optuna from optuna.integration.wandb import WeightsAndBiasesCallback

def objective(trial): x = trial.suggest_float("x", -10, 10)

return (x - 2) ** 2

wandbkwargs = {"project": "my-project"} wandbc = WeightsAndBiasesCallback(wandbkwargs=wandbkwargs) study = optuna.createstudy(studyname="mystudy") study.optimize(objective, ntrials=10, callbacks=[wandbc]) ```

See https://github.com/optuna/optuna/pull/2781 for details.

TPE Sampler Refactorings

The Tree-structured Parzen Estimator (TPE) sampler has always been the default sampler in Optuna. Both it’s API and internal code has over time grown to accomodate for various needs such as independent and join parameter sampling (the multivariate parameter) , and multi-objective optimization (the MOTPESampler sampler). In this release, the TPE sampler has been refactored and its code greatly reduced. The previously experimental multi-objective TPE Sampler MOTPESampler has also been deprecated and its capabilities are now absorbed by the standard TPESampler.

This change may break code that depends on fixed seeds with this sampler. The optimization algorithms otherwise have not been changed.

Following demonstrates how you can now use the TPESampler for multi-objective optimization.

```python import optuna

def objective(trial): x = trial.suggestfloat("x", 0, 5) y = trial.suggestfloat("y", 0, 3)

v0 = 4 * x ** 2 + 4 * y ** 2
v1 = (x - 5) ** 2 + (y - 5) ** 2

return v0, v1

sampler = optuna.samplers.TPESampler() # MOTPESampler used to be required for multi-objective optimization. study = optuna.createstudy( directions=["minimize", "minimize"], sampler=sampler, ) study.optimize(objective, ntrials=100) ```

Note that omitting the sampler argument or specifying None currently defaults to the NSGAIISampler for multi-objective studies instead of the TPESampler.

See https://github.com/optuna/optuna/pull/2618 for details.

Breaking Changes

• Unify the univariate and multivariate TPE (#2618)

New Features

• MLFlow decorator for optimization function (#2670, thanks @lucafurrer!)
• Redis Heartbeat (#2780, thanks @Turakar!)
• Introduce Weights & Biases integration (#2781, thanks @xadrianzetx!)
• Function for failing zombie trials and invoke their callbacks (#2811)
• Optuna ask and tell CLI options (#2817)

Enhancements

• Unify MOTPESampler and TPESampler (#2688)
• Changed interpolation type to make numeric range consistent with Plotly (#2712, thanks @01-vyom!)
• Add the warning if an intermediate value is already reported at the same step (#2782, thanks @TakuyaInoue-github!)
• Prioritize grids that are not yet running in GridSampler (#2783)
• Fix warn_independent_sampling in TPESampler (#2786)
• Avoid applying constraint_fn to non-COMPLETE trials in NSGAII-sampler (#2791)
• Speed up TPESampler (#2816)
• Enable CLI helps for subcommands (#2823)

Bug Fixes

• Fix AllenNLPExecutor reproducibility (#2717, thanks @MagiaSN!)
• Use repr and eval to restore pruner parameters in AllenNLP integration (#2731)
• Fix Nan cast bug in TPESampler (#2739)
• Fix infer_relative_search_space of TPE with the single point distributions (#2749)

Installation

• Avoid latest numpy 1.21 (#2766)
• Fix numpy 1.21 related mypy errors (#2767)

Documentation

• Add how to suggest proportion to FAQ (#2718)
• Explain how to add a user's own logging callback function (#2730)
• Add copy_study to the docs (#2737)
• Fix link to kurobako benchmark page (#2748)
• Improve docs of constant liar (#2785)
• Fix the document of RetryFailedTrialCallback.retried_trial_number (#2789)
• Match the case of ID (#2798, thanks @belldandyxtq!)
• Rephrase RDBStorage RuntimeError description (#2802, thanks @belldandyxtq!)

Examples

• Add remaining examples to CI tests (https://github.com/optuna/optuna-examples/pull/26)
• Use hydra 1.1.0 syntax (https://github.com/optuna/optuna-examples/pull/28)
• Replace monitor value with accuracy (https://github.com/optuna/optuna-examples/pull/32)

Tests

• Count the number of calls of the wrapped method in the test of MOTPEMultiObjectiveSampler (#2666)
• Add specific test cases for visualization.matplotlib.plot_intermediate_values (#2754, thanks @asquare100!)
• Added unit tests for optimization history of matplotlib tests (#2761, thanks @01-vyom!)
• Changed unit tests for pareto front of matplotlib tests (#2763, thanks @01-vyom!)
• Added unit tests for slice of matplotlib tests (#2764, thanks @01-vyom!)
• Added unit tests for param importances of matplotlib tests (#2774, thanks @01-vyom!)
• Changed unit tests for parallel coordinate of matplotlib tests (#2778, thanks @01-vyom!)
• Use more specific assert in tests/visualization_tests/matplotlib/test_intermediate_plot.py (#2803)
• Added unit tests for contour of matplotlib tests (#2806, thanks @01-vyom!)

Code Fixes

• Create study directory (#2721)
• Dissect allennlp integration in submodules based on roles (#2745)
• Fix deprecated version of MOTPESampler (#2770)

Continuous Integration

• Daily CI of Checks (#2760)
• Use default resolver in CI's pip installation (#2779)

Other

• Bump up version to v2.9.0dev (#2723)
• Add an optional section to ask reproducible codes (#2799)
• Add survey news to README.md (#2801)
• Add python code to issue templates for making reporting runtime information easy (#2805)
• Bump to v2.9.0 (#2828)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@ytsmiling, @harupy, @asquare100, @hvy, @c-bata, @nzw0301, @lucafurrer, @belldandyxtq, @not522, @TakuyaInoue-github, @01-vyom, @himkt, @Crissman, @toshihikoyanase, @sile, @vanpelt, @HideakiImamura, @MagiaSN, @keisuke-umezawa, @Turakar, @xadrianzetx, @ytsmiling, @harupy, @asquare100, @hvy, @c-bata, @nzw0301, @lucafurrer, @belldandyxtq, @not522, @TakuyaInoue-github, @01-vyom, @himkt, @Crissman, @toshihikoyanase, @sile, @vanpelt, @HideakiImamura, @MagiaSN, @keisuke-umezawa, @Turakar, @xadrianzetx

- Python
Published by hvy almost 5 years ago

optuna - v2.8.0

This is the release note of v2.8.0.

New Examples Repository

The number of Optuna examples has grown as the number of integrations have increased, and we’ve moved them to their own repository: optuna/optuna-examples.

Highlights

TPE Sampler Improvements

Constant Liar for Distributed Optimization

In distributed environments, the TPE sampler may sample many points in a small neighborhood, because it does not have knowledge that other trials running in parallel are sampling nearby. To avoid this issue, we’ve implemented the Constant Liar (CL) heuristic to return a poor value for trials which have started but are not yet complete, to reduce search effort.

python study = optuna.create_study(sampler=optuna.samplers.TPESampler(constant_liar=True))

The following history plots demonstrate how optimization can be improved using this feature. Ten parallel workers are simultaneously trying to optimize the same function which takes about one second to compute. The first plot has constant_liar=False, and the second with constant_liar=True, uses the Constant Liar feature. We can see that with Constant Liar, the sampler does a better job of assigning different parameter configurations to different trials and converging faster.

See #2664 for details.

Tree-structured Search Space Support

The TPE sampler with multivariate=True now supports tree-structured search spaces. Previously, if the user split the search space with an if-else statement, as shown below, the TPE sampler with multivariate=True would fall back to random sampling. Now, if you set multivariate=True and group=True, the TPE sampler algorithm will be applied to each partitioned search space to perform efficient sampling.

See #2526 for more details.

```python def objective(trial): classifiername = trial.suggestcategorical("classifier", ["SVC", "RandomForest"])

if classifier_name == "SVC":
    # If `multivariate=True` and `group=True`, the following 2 parameters are sampled jointly by TPE.
    svc_c = trial.suggest_float("svc_c", 1e-10, 1e10, log=True)
    svc_kernel = trial.suggest_categorical("kernel", ["linear", "rbf", "sigmoid"])

    classifier_obj = sklearn.svm.SVC(C=svc_c, kernel=svc_kernel)
else:
    # If `multivariate=True` and `group=True`, the following 3 parameters are sampled jointly by TPE.
    rf_n_estimators = trial.suggest_int("rf_n_estimators", 1, 20)
    rf_criterion = trial.suggest_categorical("rf_criterion", ["gini", "entropy"])
    rf_max_depth = trial.suggest_int("rf_max_depth", 2, 32, log=True)

    classifier_obj = sklearn.ensemble.RandomForestClassifier(n_estimators=rf_n_estimators, criterion=rf_criterion, max_depth=rf_max_depth)

... 

sampler = optuna.samplers.TPESampler(multivariate=True, group=True) ```

Copying Studies

Studies can now be copied across storages. The trial history as well as Study.user_attrs and Study.system_attrs are preserved.

For instance, this allows dumping a study in an MySQL RDBStorage into an SQLite file. Serializing the study this way, it can be shared with other users who are unable to access the original storage.

```python study = optuna.createstudy( studyname=”my-study”, storage=”mysql+pymysql://root@localhost/optuna" ) study.optimize(..., n_trials=100)

Creates a copy of the study “my-study” in an MySQL RDBStorage to a local file named optuna.db.

optuna.copystudy( fromstudyname="my-study", fromstorage="mysql+pymysql://root@localhost/optuna", to_storage="sqlite:///optuna.db", )

study = optuna.loadstudy(studyname=”my-study”, storage=”sqlite:///optuna.db”) assert len(study.trials) >= 100 ```

See #2607 for details.

Callbacks

optuna.storages.RetryFailedTrialCallback Added

Used as a callback in RDBStorage, this allows a previously pre-empted or otherwise aborted trials that are detected by a failed heartbeat to be re-run.

python storage = optuna.storages.RDBStorage( url="sqlite:///:memory:", heartbeat_interval=60, grace_period=120, failed_trial_callback=optuna.storages.RetryFailedTrialCallback(max_retry=3), ) study = optuna.create_study(storage=storage)

See #2694 for details.

optuna.study.MaxTrialsCallback Added

Used as a callback in study.optimize, this allows setting of a maximum number of trials of a particular state, such as setting the maximum number of failed trials, before stopping the optimization.

python study.optimize( objective, callbacks=[optuna.study.MaxTrialsCallback(10, states=(optuna.trial.TrialState.COMPLETE,))], ) See #2636 for details.

Breaking Changes

• Allow None as study_name when there is only a single study in load_study (#2608)
• Relax GridSampler allowing not-contained parameters during suggest_* (#2663)

New Features

• Make LightGBMTuner and LightGBMTunerCV reproducible (#2431, thanks @tetsuoh0103!)
• Add visualization.matplotlib.plot_pareto_front (#2450, thanks @tohmae!)
• Support a group decomposed search space and apply it to TPE (#2526)
• Add __str__ for samplers (#2539)
• Add n_min_trials argument for PercentilePruner and MedianPruner (#2556)
• Copy study (#2607)
• Allow None as study_name when there is only a single study in load_study (#2608)
• Add MaxTrialsCallback class to enable stopping after fixed number of trials (#2612)
• Implement PatientPruner (#2636)
• Support multi-objective optimization in CLI (optuna create-study) (#2640)
• Constant liar for TPESampler (#2664)
• Add automatic retry callback (#2694)
• Sorts categorical values on axis that contains only numerical values in visualization.matplotlib.plot_slice (#2709, thanks @Muktan!)

Enhancements

• PyTorchLightningPruningCallback to warn when an evaluation metric does not exist (#2157, thanks @bigbird555!)
• Pareto front visualization to visualize study progress with color scales (#2563)
• Sort categorical values on axis that contains only numerical values in visualization.plot_contour (#2569)
• Improve param_importances (#2576)
• Sort categorical values on axis that contains only numerical values in visualization.matplotlib.plot_contour (#2593)
• Show legend of optuna.visualization.matplotlib.plot_edf (#2603)
• Show legend of optuna.visualization.matplotlib.plot_intermediate_values (#2606)
• Make MOTPEMultiObjectiveSampler a thin wrapper for MOTPESampler (#2615)
• Do not wait for next heartbeat on study completion (#2686, thanks @Turakar!)
• Change colour scale of contour plot by matplotlib for consistency with plotly results (#2711, thanks @01-vyom!)

Bug Fixes

• Add type conversion for reference point and solution set (#2584)
• Fix contour plot with multi-objective study and target being specified (#2589)
• Fix distribution's _contains (#2652)
• Read environment variables in dump_best_config (#2681)
• Update version info entry on RDB storage upgrade (#2687)
• Fix results not reproducible when running AllenNLPExecutor multiple t… (Backport of #2717) (#2728)

Installation

• Replace sklearn constraint (#2634)
• Add constraint of Sphinx version (#2657)
• Add click==7.1.2 to GitHub workflows to solve AllenNLP import error (#2665)
• Avoid tensorflow 2.5.0 (#2674)
• Remove example from setup.py (#2676)

Documentation

• Add example to optuna.logging.disable_propagation (#2477, thanks @jeromepatel!)
• Add documentation for hyperparameter importance target parameter (#2551)
• Remove the news section in README.md (#2586)
• Documentation updates to CmaEsSampler (#2591, thanks @turian!)
• Rename ray-joblib.py to snakecase with underscores (#2594)
• Replace If with if in a sentence (#2602)
• Use CmaEsSampler instead of TPESampler in the batch optimization example (#2610)
• README fixes (#2617, thanks @Scitator!)
• Remove wrong returns description in docstring (#2619)
• Improve document on BoTorchSampler page (#2631)
• Add the missing colon (#2661)
• Add missing parameter WAITING details in docstring (#2683, thanks @jeromepatel!)
• Update URLs to optuna-examples (#2684)
• Fix indents in the ask-and-tell tutorial (#2690)
• Join sampler examples in README.md (#2692)
• Fix typo in the tutorial (#2704)
• Update command for installing auto-formatters (#2710, thanks @01-vyom!)
• Some edits for CONTRIBUTING.md (#2719)

Examples

• Split GitHub Actions workflows (https://github.com/optuna/optuna-examples/pull/1)
• Cherry pick #2611 of optuna/optuna (https://github.com/optuna/optuna-examples/pull/2)
• Add checks workflow (https://github.com/optuna/optuna-examples/pull/5)
• Add MaxTrialsCallback class to enable stopping after fixed number of trials (https://github.com/optuna/optuna-examples/pull/9)
• Update README.md (https://github.com/optuna/optuna-examples/pull/10)
• Add an example of warm starting CMA-ES (https://github.com/optuna/optuna-examples/pull/11, thanks @nmasahiro!)
• Replace old links to example files (https://github.com/optuna/optuna-examples/pull/12)
• Avoid tensorflow 2.5.0 (https://github.com/optuna/optuna-examples/pull/13)
• Avoid tensorflow 2.5 (https://github.com/optuna/optuna-examples/pull/15)
• Test multi_objective in CI (https://github.com/optuna/optuna-examples/pull/16)
• Use only one GPU for PyTorch Lightning example by default (https://github.com/optuna/optuna-examples/pull/17)
• Remove example of CatBoost in pruning section (https://github.com/optuna/optuna-examples/pull/18, #2702)
• Add issues and pull request templates (https://github.com/optuna/optuna-examples/pull/20)
• Add CONTRIBUTING.md file ((https://github.com/optuna/optuna-examples/pull/21)
• Change PR approvers from two to one (https://github.com/optuna/optuna-examples/pull/22)
• Improved search space XGBoost (#2346, thanks @jeromepatel!)
• Remove n_jobs for study.optimize in examples/ (#2588, thanks @jeromepatel!)
• Using the "log" key is deprecated in pytorch_lightning (#2611, thanks @sushi30!)
• Move examples to a new repository (#2655)
• Remove remaining examples (#2675)
• optuna-examples (https://github.com/optuna/optuna-examples/pull/11 follow up (#2689)

Tests

• Remove assertions for supported dimensions from test_plot_pareto_front_unsupported_dimensions (#2578)
• Update a test function of matplotliv.plot_pareto_front for consistency (#2583)
• Add deterministic parameter to make LightGBM training reproducible (#2623)
• Add force_col_wise parameter of LightGBM in test cases of LightGBMTuner and LightGBMTunerCV (#2630, thanks @tetsuoh0103!)
• Remove CudaCallback from the fastai test (#2641)
• Add test cases in optuna/visualization/matplotlib/edf.py (#2642)
• Refactor a unittest in test_median.py (#2644)
• Refactor pruners_test (#2691, thanks @tsumli!)

Code Fixes

• Remove redundant lines in CI settings of examples (#2554)
• Remove the unused argument of functions in matplotlib.contour (#2571)
• Fix axis labels of optuna.visualization.matplotlib.plot_pareto_front when axis_order is specified (#2577)
• Remove list casts (#2601)
• Remove _get_distribution from visualization/matplotlib/_param_importances.py (#2604)
• Fix grammatical error in failure message (#2609, thanks @agarwalrounak!)
• Separate MOTPESampler from TPESampler (#2616)
• Simplify add_distributions in _SearchSpaceGroup (#2651)
• Replace old example URLs in optuna.integrations (#2700)

Continuous Integration

• Supporting Python 3.9 with integration modules and optional dependencies (#2530, thanks @0x41head!)
• Upgrade pip in PyPI and Test PyPI workflow (#2598)
• Fix PyPI publish workflow (#2624)
• Introduce speed benchmarks using asv (#2673)

Other

• Bump master version to 2.8.0dev (#2562)
• Upload to TestPyPI at the time of release as well (#2573)
• Install blackdoc in formats.sh (#2637)
• Use command to check the existence of the libraries to avoid partially matching (#2653)
• Add an example section to the README (#2667)
• Fix formatting in contribution guidelines (#2668)
• Update CONTRIBUTING.md with optuna-examples (#2669)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@toshihikoyanase, @himkt, @Scitator, @tohmae, @crcrpar, @c-bata, @01-vyom, @sushi30, @tsumli, @not522, @tetsuoh0103, @jeromepatel, @bigbird555, @hvy, @g-votte, @nzw0301, @turian, @Crissman, @sile, @agarwalrounak, @Muktan, @Turakar, @HideakiImamura, @keisuke-umezawa, @0x41head, @toshihikoyanase, @himkt, @Scitator, @tohmae, @crcrpar, @c-bata, @01-vyom, @sushi30, @tsumli, @not522, @tetsuoh0103, @jeromepatel, @bigbird555, @hvy, @g-votte, @nzw0301, @turian, @nmasahiro, @Crissman, @sile, @agarwalrounak, @Muktan, @Turakar, @HideakiImamura, @keisuke-umezawa, @0x41head

- Python
Published by hvy about 5 years ago