orion - New Multi-Task Warm-starting and Dashboard Prototype

🚀 Features

• Add experiment progress bar to dashboard @notoraptor (#1080)
• Add benchmarks endpoints to Orion Web API @notoraptor (#996)
• Knowledge Base + Multi-Task Warm-starting @lebrice (#933)
• Display trial status in trials table on dashboard @notoraptor (#1071)

🏗 Enhancements

• Make ExperimentClient.observe support single value @bouthilx (#1063)
• Bump json5 from 1.0.1 to 1.0.2 in /dashboard/src @dependabot (#1057)
• Improve experiment stats @notoraptor (#1038)
• Remove references to get_storage @Delaunay (#1042)
• Add assert_contains to print a more precise error message @Delaunay (#1013)
• Do not close the executor after workon #863 @Delaunay (#1041)
• Update yarn.lock @notoraptor (#1032)
• Add Python 3.10 @Delaunay (#690)
• reservation_timeout removed @Simnol22 (#1011)
• Enhance table @notoraptor (#982)
• Add poolexecutor to the executor entry point @Delaunay (#1047)
• remove python 3.6 special handling @Delaunay (#1045)
• Only Protect gather from interruptions #789 @Delaunay (#1048)
• Changing experiment.algorithms to experiment.algorithm @Simnol22 (#1014)
• Reduce Warnings @Delaunay (#1072)

🐛 Bug Fixes

• Limit nevergrad version to fix long algo tests in CI @notoraptor (#1086)
• Fix detection of dashboard build files in installed packages. @notoraptor (#1084)
• CI - fix pre-commit and backward compatibility @notoraptor (#1079)
• Xfail HEBO and add warning in docs @bouthilx (#1051)
• Fix algos for nested spaces @bouthilx (#1064)
• Remove pandas version cap @bouthilx (#1006)
• Fix uniform real dimension interval mapping to ConfigSpace @Delaunay (#1034)
• Downgrade pandas in conda build too @bouthilx (#1001)
• Downgrade pandas temporarily @bouthilx (#998)
• Fix a few bugs in the Profet tasks @lebrice (#1030)

📜 Documentation

• Update Storage doc config example #1055 @Delaunay (#1056)
• Add credit to Guillaume Alain for logos @bouthilx (#1043)
• SpeechBrain Tutorial @Simnol22 (#1004)
• Add logos for Oríon @bouthilx (#941)
• Fixing some details for the tutorials to run. @Simnol22 (#997)
• Adding HuggingFace Tutorial @Simnol22 (#1040)

🧰 Maintenance

• [ci] Install Orion from local cloned repository @notoraptor (#1068)
• Dashboard - new testing framework: playwright @notoraptor (#1078)
• Move numpy version cap to HEBO optional deps @bouthilx (#1065)
• Use list instead of array @bouthilx (#1050)
• Add matplotlib dependency to pb2 @Delaunay (#1020)
• Upgrade MongoDB to 6.0 @Delaunay (#1036)
• Reactivate dashboard CI @notoraptor (#952)
• Update pre-commit to use the right flake8 repo @Delaunay (#1028)

- Python
Published by release-drafter[bot] over 3 years ago

orion - Bugfix for user script stdout/stderr

Changes

🐛 Bug Fixes

• Revert Subprocess pipe @Delaunay (#985)

- Python
Published by release-drafter[bot] almost 4 years ago

orion - Bugfixes for PBT

🏗 Enhancements

• Add configspace conversion @Delaunay (#832)
• no singleton test @Delaunay (#942)
• Simplify tree.py @lebrice (#966)
• Minor refactor: Make creation of Experiment objects more explicit @lebrice (#968)
• [dashboard/experiments/database] make columns sortable, except non-singleton array columns @notoraptor (#969)
• [dashboard/DB page] Display only 7 first characters of trial ID in ID column. @notoraptor (#960)
• [dashboard] Display trials for selected experiment in Database page @notoraptor (#944)
• Add a dashboard launcher @notoraptor (#943)
• [dashboard/experiments/database page] New table framework @notoraptor (#975)
• [dashboard/experiments/navbar] Remove (x) icon on experiment menu and rely on highlight only @notoraptor (#970)
• Small fixes @abergeron (#964)
• [dashboard/db page] DIsplay parameters in separated flattened columns after ID column @notoraptor (#961)
• Add repr to dbs to help debug with orion db test @bouthilx (#958)
• Remove experiment from Trial hash @bouthilx (#950)

🐛 Bug Fixes

• Fix bug with algo param_names @lebrice (#947)
• Fix PBT issues with working dir and promotion of max fidelity trials @bouthilx (#903)

📜 Documentation

• Fix some documentation formatting @abergeron (#946)
• Add more info on how trial config is passed to scripts @bouthilx (#957)

🧰 Maintenance

• Fix pre-commit issues present in develop @lebrice (#967)
• Replace (part of) pretest of CI with Pre-commit equivalent @lebrice (#918)
• Fix ConfigSpace dependency issue @bouthilx (#955)
• Ignore .post in version for backward comp tests. @bouthilx (#949)
• Add mongodb & test_no_extras to pypi test deps @bouthilx (#940)

- Python
Published by github-actions[bot] almost 4 years ago

orion - Algorithm Integrations and Improvements

🚀 Features

• Add Ax Optimizer @satyaog (#866)
• Migrate Orion dashboard into Orion repository @notoraptor (#920)
• Add MOFA @wagnersj (#864)
• Add DEHB @bouthilx (#849)
• Add Hebo @lebrice (#915)
• Add PB2 @bouthilx (#831)
• Add BOHB @bouthilx (#848)
• Move nevergrad to orion repo @Delaunay (#906)
• new parallel advantage assessment support @donglinjy (#685)
• Add tasks from Profet @lebrice (#632)

🏗 Enhancements

• Fix dependabot issues @notoraptor (#926)
• Dashboard: updates dependencies to try resolve dependabot issues @notoraptor (#923)
• Add number of suggest() attempts in Algo wrapper @lebrice (#883)
• Reduce verbosity of the Runner @bouthilx (#879)
• Detect _param_names from __init__ signature @lebrice (#867)
• Add Registry, rework how Space is passed to algorithms @lebrice (#833)
• Upgrade default python to 3.9 in build.yaml @lebrice (#855)

🐛 Bug Fixes

• Fix IndexError for freshly branched exp @vmichals (#898)
• Handle all tests using Dask @bouthilx (#828)

📜 Documentation

• Add video links to last presentations @bouthilx (#928)
• Add CITATION.bib to make it easier to cite Orion @lebrice (#896)

🧰 Maintenance

• Add Github Actions to automatically compile Orion dashboard @notoraptor (#930)
• [dashboard] Increase Axios timeout @notoraptor (#927)
• Add .pre-commit-config.yaml for local setup @lebrice (#895)
• Hotfix for nevergrad tests @lebrice (#913)
• Simplify tests suite, remove spy_phase and related functions @lebrice (#886)
• Refactor tox to run for any algo @Delaunay (#907)
• Fix Pypi, add CITATION to manifest @Delaunay (#899)
• Remove remaining mongodb configs @bouthilx (#868)
• Simplify setting test phases in Test Classes @lebrice (#874)
• Update black @Delaunay (#861)
• Drop support for python 3.6 @lebrice (#853)
• Test algo can handle broken trials @bouthilx (#813)
• Tests run with no extras and all extras @lebrice (#810)
• Don't build wheels in tox build step by @lebrice (#938)

Full Changelog: https://github.com/Epistimio/orion/compare/v0.2.3...v0.2.4.post1

- Python
Published by release-drafter[bot] about 4 years ago

orion - Bug fix to adress severe I/O load during optimization

🏗 Enhancements

• Add config option frontends_uri to specify allowed frontend addresses for Orion Web API @notoraptor (#811)

🐛 Bug Fixes

• Make sure subprocess/process/thread/dask can create a runner instance @Delaunay (#816)
• Reduce DB I/O during optimization @bouthilx (#823)

- Python
Published by github-actions[bot] over 4 years ago

orion - Improved parallelization and PBT algorithm

Parallelization is improved leveraging new Runner (#684) and a shared algorithm state (#786).

🚀 Features

• PBT implementation @bouthilx (#705)

🏗 Enhancements

• Shared algorithm status @bouthilx (#786)
• Fix get_prior_string [#774] @Delaunay (#776)
• Replace np.object by object; np.object is deprecated @Delaunay (#773)
• Update backend_plotly.py @Delaunay (#772)
• Rework workon @Delaunay (#684)
• Print version on debug verbosity level @nurbal (#765)
• Remove ParallelStrategies from Producer and make them reusable modules for algorithms instead. @bouthilx (#760)

🐛 Bug Fixes

• (re) Fix parsing of hps with "." in name from commandline @lebrice (#785)
• Fix multiprocess del guard @Delaunay (#784)
• Replace exp's stat dict by a dataclass @bouthilx (#771)
• replace count by count_documents #706 @Delaunay (#707)

📜 Documentation

• Escape the escape @Delaunay (#783)

🧰 Maintenance

• Fix Track requirement URL for github @bouthilx (#761)

- Python
Published by release-drafter[bot] over 4 years ago

orion - Minor release: Bug fix for ASHA when some trials have identical objectives

🐛 Bug Fixes

• Fix ASHA for identical objectives @bouthilx (#699)

- Python
Published by release-drafter[bot] over 4 years ago

orion - New interface for algorithm's `suggest()` and `observe()` methods.

Algorithm interface for suggest() and observe() is changed to use trials instead of list of points. This will provide more flexibility when designing new algorithms. There is no impact for users, except for algorithm plugins that will not be backward compatible with previous versions of Oríon.

The factory abstract class used to create database, storage, algorithm, adapters and parallel strategy has been redesigned to better support external implementations and be easier to debug. This redesign should also have no direct impact for users, except for backward compatibility issues with algorithm plugins for previous versions of Oríon.

🏗 Enhancements

• Improve resiliency of trial reservation @bouthilx (#693)
• Support batched promotions in ASHA @bouthilx (#689)
• Refactor algorithms observe/suggest to work on trial objects directly @bouthilx (#686)
• Rework factory @bouthilx (#681)
• Improve Getting started example command @Delaunay (#666)

🐛 Bug Fixes

• Avoid inf-loop in TPE.suggest @bouthilx (#680)

- Python
Published by release-drafter[bot] over 4 years ago

orion - Revert to default inf max trials and pool-size of 1

🏗 Enhancements

• Revert to default inf max_trials and pool-size of 1 @bouthilx (#659)
• Support to configure executor for benchmark @donglinjy (#634)

🐛 Bug Fixes

• Fix grouping of plots in legend @lebrice (#662)
• Support linearize log integer properly @bouthilx (#658)
• Fix shape argument in categorical dims @bouthilx (#654)
• Use user_script_config from parser in EVC @bouthilx (#655)
• Fix TPE sampling in narrow spaces @bouthilx (#650)
• Handle repo with invalid HEAD state @bouthilx (#645)
• List EVC tree once only when querying with name @bouthilx (#642)
• Handle properly all types in config during branch @bouthilx (#641)

📜 Documentation

• Clear error message for dup branching error @bouthilx (#652)
• Add missing documentation for EVC enable option @bouthilx (#651)

- Python
Published by release-drafter[bot] almost 5 years ago

orion - Disabled Experiment Version Control

Experiment Version Control (EVC) will now be disabled by default. When the EVC is disabled, any changes in the experiment will be saved to the DB, overriding the previous version. See configuration doc to enable the EVC: https://orion.readthedocs.io/en/stable/user/config.html#experiment-version-control.

🏗 Enhancements

• Feature/filter duplicate in evc @bouthilx (#630)
• Add enable option for EVC, with default = false @bouthilx (#626)

🐛 Bug Fixes

• Warn only if diffs exists during exp build @bouthilx (#638)
• Fix lost trials of parents @bouthilx (#637)
• Compute cardinality for loguniform with precision @bouthilx (#635)
• Duplicate pending trials from parent/child for exc @bouthilx (#631)
• Remove space upgrade from DB upgrade @bouthilx (#627)
• Allow branching when parent script isn't available @bouthilx (#625)
• fix benchmark ranking and regret visualization @donglinjy (#620)
• fix tpe space cardinality @donglinjy (#619)

📜 Documentation

• Change algo documentation @bouthilx (#623)

- Python
Published by release-drafter[bot] almost 5 years ago

orion - New Executor Backend for Parallelism with Joblib and Dask

🚀 Features

• Add Executor class and CLI arguments for parallelism @bouthilx (#608)
• Orion python api in parallel @donglinjy (#601)

🏗 Enhancements

• Add has_suggested/observed, new test-suite and re-implementation of ASHA @bouthilx (#604)
• Refactor hunt command to reuse ExperimentClient.workon @bouthilx (#605)

🐛 Bug Fixes

• Test different type of categorical dims @bouthilx (#609)
• Add has_suggested/observed, new test-suite and re-implementation of ASHA @bouthilx (#604)

📜 Documentation

• Documentation for the visualizations @bouthilx (#557)
• Add has_suggested/observed, new test-suite and re-implementation of ASHA @bouthilx (#604)
• Add presentations section in doc @bouthilx (#597)

- Python
Published by release-drafter[bot] about 5 years ago

orion - PickledDB is now the default storage backend

🚀 Features

• Add plot commandline plot @gyom (#593)

:building_construction: Enhancements

• Add where parameter to Storage.fetch_trials @breuleux (#584)
• Make command orion db setup ask for right arguments based on storage backend. @notoraptor (#586)
• Change the default database to PickledDB @abergeron (#585)
• Deterministic algorithm benchmark @donglinjy (#567)

🐛 Bug Fixes

• Add context-manager-enabled Trial subclass. @obilaniu (#587)
• Handle cases when trial.status is not reserved anymore locally. @bouthilx (#591)

🧰 Maintenance

• Ignore all test files in coverage @bouthilx (#592)
• Fix for coverage issue @abergeron (#581)
• Remove MongoDB dependency from core tests @breuleux (#582)
• Write generic tests for DB @satyaog (#590)

- Python
Published by release-drafter[bot] over 5 years ago

orion - Benchmark, Partial dependency plot and Grid Search

🚀 Features

• Add hover to partial deps scatter @bouthilx (#565)
• Make plots use trials in EVC by default @bouthilx (#564)
• Add error bar to LPI @bouthilx (#563)
• Add benchmark support into orion @donglinjy (#513)
• Add partial dependency plot @bouthilx (#544)
• Add grid search algorithm @bouthilx (#539)

🐛 Bug Fixes

• Set ignore_code_changes in Consumer too @bouthilx (#541)

📜 Documentation

• Major rehaul of documentation @bouthilx (#549)
• Fix readthedocs build @bouthilx (#542)

- Python
Published by release-drafter[bot] over 5 years ago

orion - Minor release: Read/Write/Execution modes for Experiments, branching on orion version, bug fixes

🚀 Features

• Add Read/Write/Execution modes for Experiment @bouthilx (#536)
• Branch if Oríon version changes @bouthilx (#530)

🐛 Bug Fixes

• Detect code change when no CMDLINE passed @bouthilx (#529)
• Support EVC in all algos @bouthilx (#523)
• Test that precision isn't messed up by linearization @bouthilx (#522)
• Ignore non-completed trials when computing LPI @bouthilx (#521)
• Add debugging verbosity @bouthilx (#520, #528)

📜 Documentation

• Add missing plotting doc in API section @bouthilx (#532)

🧰 Maintenance

• Automatically update backward test versions @bouthilx (#512)
• Use pypi Token to release @bouthilx (#511)

- Python
Published by release-drafter[bot] over 5 years ago

orion - Minor release: Support for python 3.9, bug fixes, improved CI

Changes

• Sync back with master @bouthilx (#496)

🚀 Features

• Add support for python 3.9 @bouthilx (#497)

🐛 Bug Fixes

• Use SoftFileLock if FS doesn't support global lock @bouthilx (#506)
• Fix algorithm requirements backward @bouthilx (#498)

🧰 Maintenance

• Add isort @bouthilx (#505)
• Add black checks and remove flake8 @bouthilx (#504)
• Fix drafter config @bouthilx (#502)
• Test release drafter @bouthilx (#501)
• Add release drafter @bouthilx (#499)

- Python
Published by release-drafter[bot] over 5 years ago

orion - Parallel Coordinates & Local Parameter Importance Plots

Detailed list of changes

New features

• Add max_broken in exp configuration (#473)
• Add Parallel Coordinates Plot (#480)
• Add ReshapedSpace and Linearize transformer (#482)
• Add LPI plot (#483)

Bug Fixes

• Add extensive functional testing for configuration (#474)
• Handle rm/add properly in EVC (#475)
• Override fields with set, not update (#479)
• Fix cardinality and integer cast (#481)

Other improvements

• Improve CI (#464, #472)

- Python
Published by bouthilx over 5 years ago

orion - Plotting API and Database commands

Important changes

New plotting API

Plotting capability is being added to experiment clients. You can now plot the regret (curve of best objective found during optimization) with simply experiment.plot.regret(). You can find an example here.

Web API (REST server)

A web API was added for v0.1.9 in order to support the visualization dashboard that is currently under work. See full documentation here.

Database commands rm & set

Command line helpers have been added to simplify the process of deleting experiments and trials as well as modifying trials in the database. See full documentation for both commands here.

Detailed list of changes

New features

• New plotting API (#400, #426)
• REST server (#437, #438, #441, #446)
• New command orion db rm (#425)
• New command orion db set (#428)

Breaking changes

• Drop support for orion < 0.1.6 (#420)
• Deprecate init_only and add hunt --init-only (#436)

Bug Fixes

• Fix backward compatibility tests in master - Drop support for orion < 0.1.6 (#420)
• Fix test using 'in' to assert equality (#427)
• Avoid side effects in setup_storage (#444)
• Return onehot interval instead of categorical (#447)
• Adjust OneHotEncoding interval shape (#449)
• Support non existing user script conf files (#452)
• Support load yaml config for dict items (#453)
• Fix fetchlosttrials (Causing random FailedUpdate when using many workers) (#445)

Other improvements

• Improve CI - Convert functional algo tests to python API (#418)
• Monitor code change between trial execution (#429)
• Add info about working_dir when using orion info CLI (#430)
• Enable is_broken for ExperimentView (#432)
• Specialized properties for Trials (#433)
• Add completed field to info cmd (#435)
• Add orion.client.get_experiment() (#439)
• Sort modules and extensions alphabetically (#443)
• Print user friendly err msg if script inexecutable (#445)
• Migrate from Travis to Github-actions (#458, #459, #461)

Documentation improvements

• Add a code of conduct (#353)
• Refresh database documentation (#413)
• Merging Sequential and Service API (#416)
• CLI documentation improvements (#440)
• Remove outdated branch_from parameter documentation in experiment_builder.build() (#442)
• Changed variable name in docs to reflect the correct name (#448)
• Fix config of EvolutionaryES (#451)

- Python
Published by bouthilx over 5 years ago

orion - Python API and New Algorithms

Important changes

The python API is finally ready for release v0.1.8! :tada:

Python API

An API is now available to run experiments directly from python instead of using the commandline.

```python from orion.client import create_experiment

experiment = create_experiment( name='foo', space=dict(x='uniform(-50,50)'))

trial = experiment.suggest()

Do something using trial.params['x']

results = [dict( name='dummyobjective', type='objective', value=dummyobjective)]

experiment.observe(trial, results) ```

Current API provides a simple function workon for cheap experiments that can be executed by a single worker, and a generic ExperimentClient (see example above) object for optimization with multiple workers.

See documentation for more details.

New Algorithms

Hyperband

Hyperband extends the Successive Halving algorithm by providing a way to exploit a fixed budget with different number of configurations for SuccessiveHalving algorithm to evaluate. It is especially useful when the trials are expensive to run and cheap noisy evaluations are possible. Think of it as using early evaluation during training to filter out bad candidates.

For more information on the algorithm, see original paper.

TPE

Tree-structured Parzen Estimator (TPE) algorithm is one of Sequential Model-Based Global Optimization (SMBO) algorithms, which will build models to propose new points based on the historical observed trials.

Instead of modeling p(y|x) like other SMBO algorithms, TPE models p(x|y) and p(y), and p(x|y) is modeled by transforming that generative process, replacing the distributions of the configuration prior with non-parametric densities.

TPE has the advantage of scaling particularly well compared to most Model-Based algorithm which are typically sequential. It does not model however dependencies between hyper-parameters, they are assumed independent.

For more information on the algorithm, see original papers at: - Algorithms for Hyper-Parameter Optimization - Making a Science of Model Search: Hyperparameter Optimizationin Hundreds of Dimensions for Vision Architectures

Storage

To support integration with other tools and services such as MLFlow or Weight & Biases we wrapped our previous database backend with a storage backend. The database backends are now available within the Legagy storage backend. In addition, we now have a backend for Track. The latter is planned to serve as a bridge between Oríon and other experiment management platforms or services. Track package development is on the ice for now, but contributions are very much welcomed. :)

Drop python 3.5, support 3.8

Although Oríon may still be compatible with python 3.5 we do not maintain it's support anymore. Python 3.8 is now officially supported.

Precision of real dimensions

By default Oríon now rounds hyperparameters to 4 decimals (ex 0.00041239123 would become 0.0004124). The rational is that little variations on continuous hyperparameters typically leads to little variations in the in objective. When sharing hyperparameters (ex: in publications), one can now share the rounded values with the exact corresponding objectives instead of rounding the hyperparameters after the execution and risk sharing unreproducible results.

Rework of documentation

The documentation has been through a major rework. - The introduction has been updated to better reflect current features. - A section Getting Started and an Overview was added. - The plugins documentation was updated with a new cookiecutter template to create algorithm plugins. - A minimalist example of scikit-learn was added to serve as the simplest tutorial for Oríon. - Extensive documentation on contributions has been added.

Detailed list of changes

New features

• Storage Backend with Track and Legacy (#289, #302, #316, #318)
• Python API (#297, #299, #300, #305, #307, #311, #312, #355, #399, #401)
• Add option to ignore some commandline options of user's script (#308)
• Add option to ignore code changes in user's script (#310)
• Add trial hash id that does not include fidelifty (useful to resume across fidelities)(#313, #322)
• Customize precision (#331)
• Support commandline call where first argument is not user's script (#333, #338)
• Add support for non-standard -args (#341)
• Add more helper functions for cmdline client (#343)
• Hyperband implementation (#354, #363, #383)
• TPE implementation (#381, #387, #389)

Breaking changes

• Drop python 3.5 support in favor of 3.8 (#303)
• Remove deprecated score_handle (#315)
• Make global, local and cmdline args coherent (#349, #404)

Bug Fixes

• Fix EphemeralDB index update after document deletion (#306)
• Support small fidelity scales for ASHA (#314)
• Algo redefinition should not drop algo config (#319)
• Random algorithm will hang if search space is smaller than specified trials number (#336)
• Inconsistent output in orion status and orion list (#342)
• Use EVC tree trials in producer (#347)
• Update fidelity sample method to honor required n_samples (#351)
• Add maxtrials to algorithm.isdone (#360)
• Handle trials with corrupted status (#372)
• Making upper bound inclusive (#373)
• Fix --debug (#374)
• Round ASHA budget properly (#402)
• Update Plugins documentation section (#406)
• Print help when calling orion alone (#408)

Other improvements

• improve error message for ASHA (#317)
• Print important information at end of worker (#345)
• Add comprehensive error message for branching err (#346)
• Make global, local and cmdline args coherent (#349, #404)
• Improve tox configuration (#350, #398)
• Generic cardinality check in base algorithm (#352)
• Avoid duplicates in RandomSearch (#361)
• Handle timeout in DBs (PickledDB in particular) (#362)
• Add stress test (#364)
• Remove useless deprecation warnings (#376)
• Ignore test files for code coverage (#377)
• Show proper yaml syntax in resolve_config deprecation warnings containing None (#391)
• Raise exception when no prior is provided (#410)

Documentation improvements

• Add citation section (#301, #379)
• Fix doc for bayesopt's alpha (#309)
• Formatted commands and added a command description (#327)
• Clone via HTTPS instead of SSH (#328)
• Minimalist example with scikit-learn (#339)
• Full configuration documentation (#349)
• Contribution guidelines (#356)
• Update developer documentation (#365, #378)
• Release & Packaging (#371, #411)
• Update the documentation structure (#375)
• Refreshed introduction of Oríon's features (#380, #384)
• Overhaul of the documentation for new comers (#385, #403)
• Add Issue Template (#386)
• Add PR Template (#388)
• Add doc about label categories (#397)
• Update Plugins documentation section (#405)
• Document Windows compatibility (#409)
• General documentation updates (#332, #340, #344, #366, #367, #370)

- Python
Published by bouthilx about 6 years ago

orion - Bug fix with database upgrade

Important changes

Database upgrade

There was an incompatibility introduced in v0.1.6 that would break pickleddb created with previous versions. This minor release introduces a new command orion db upgrade to upgrade the database scheme so that databases created with orion<v0.1.6 can still be used in new versions.

Detailed list of changes

New feature

• Add orion db upgrade command (#293)

Breaking changes

• Turn test-db into db test (#291)

Bug fixes

• Handle non-existing fields in EphemeralDB (#284)
• Fix deepcopy of Enumerate transform (#290)
• Instantiate adapter even if empty (#295)

Others improvements

• Add functional tests for backward-compatibility (#285)
• Save parser state in DB (#288)

Documentation improvements

• Add mention of windows in install doc (#287)
• Update skopt doc and prepare v0.1.7 (#292)

- Python
Published by bouthilx almost 7 years ago

orion - Auto-resolution and experiment versions

Important changes

Migrated to orion on PyPi

We finally have orion namespace: pypa/warehouse#4189!!! :tada: :confetti_ball:

Auto-resolution and experiment versions

The modification of experiments (code change, search space modification, etc) will now be automatically resolved and will no longer lead to the (confusing) branch resolver prompt. When there is any modification that leads to a branching, the version of the experiment will be incremented (starting at 1). The unique index of the experiments are now (name, version) instead of (name, user).

No more user index

The user name is no longer part of the experiment's index. This means that someone with a username A in one environment may retrieve the same experiments with a username B in another environment without any problem. Previously this would only be possible using the option --user B to override the default system's username. This was due to the fact that queries on the database was done using the username and it is no longer the case.

Improved ASHA

Interface

The new prior fidelity(low, high, base) now makes it more convenient to define the fidelity dimension. The different budget for each rungs can now be scaled between low and high according to a base logarithm.

Efficiency

The algorithm will now stop registering trials in the low rungs once the higher rungs are filled. This reduces the waste of resources spend on trials in low rungs that cannot be promoted to higher rungs since the latter will be completed shortly.

Detailed list of changes

New features

• Auto-resolution (#244)
• Add versioning to experiments (#240, #242, #243, #245, #246, #247, #248, #259, #268, #272, #279, #281 )
• Add to algo the ability to opt out (#266)
• Add env-vars to consumer (#280)

Breaking changes

• Remove user as an index for Experiment (#264, #273 )
• Add low, high, base to Fidelity (#269)

Bug Fixes

• Fix Trial ID mismatch with higher dimension (#237)
• Make Space Sorted instead of Ordered (#257, #263 )
• Fix bug in conda test (#258)
• Handle status --name when empty (#270)
• Fix ASHA termination condition and branching with fidelity() (#274 )
• Avoid branching in info (#277)

Other improvements

• Add a global configuration object orion.core.config (#239, #251, #265)
• Remove direct access to database backend in favor of storage backend (#241)
• Remove generic queries in favor of specific queries in storage backend (#250)
• Add best trial id to info command (#260)
• Make ASHA opt out when last rungs are filled (#267)
• Migrate to orion on PyPi (#271 )

Documentation improvements

• Fix invalid section names (#252)
• Make tutorial a bit cleaner (#262)
• Improve documentation of pool-size and fix experiment.API (#278)
• Document script integration with templates (#280 & #282)

- Python
Published by bouthilx almost 7 years ago

orion - New commands `list`, `status` and `info` and support interruption/resumption of trials

New commands

New commands list, status and info now enable simple monitoring and inspection of experiments. The command list prints out all the experiments' name so you can easily keep track of what is available in your database. The command status gives a summary of experiments' status, with number of trials in different status (completed, reserved, etc) and best objective found so far. Finally the command info prints out all details of a given experiment, command line used, search space, algorithm configuration and current statistics.

Trial interruption

So far interrupted trials would keep the status reserved and wouldn't be available for selection
again, so they would be lost somehow. They will now be set to interrupted and will be available for resumption. If user's script does not support checkpointing internally, the process will simply be re-executed from scratch.

New features

• New command list (#208, #216 )
• New command status (#222, #229)
• New command info (#211)
• New StorageProtocol (#221)
• Handling of interrupted trials (#197, #202, #203, #235)
• Limit number of broken trials before experiment stop execution (#204)
• Add working_dir attribute to Trial (#205) ## Breaking Changes
• Deprecate pack/unpack in orion.algo.space (#236) ## Bug Fixes
• Fix code conflict detection when not using code versioning (#207)
• Add $ne and handle non-supported operations in EphemeralDB (#214)
• Fix ScriptConfigConflict for data with non-string values (#215)
• Fix race conditions during algo updates (#217)
• Fix experiment views (#218, #223, #234)
• Make sure higher shape points are not ndarray objects (#224, #236) ## Other Improvements
• Mute (intense) logging of filelock (#201)
• Functional tests for user script config (#212)
• Use yaml.safe_load instead of yaml (#219)
• Remove upsert from write (#220) ## Documentation Improvements
• Add documentation for new commands list, info and status (#226, #228)
• Add documentation on Search Space (#227)

- Python
Published by bouthilx almost 7 years ago

orion - Minor Release: OSX support, New Roadmap and flake8 fix

Bug Fixes

• Fix flake8 dependency because of pydocstyle's new release #198 ## Other Improvements
• Add continuous integration for OSX #196 ## Documentation Improvements
• Add RoadMap #195

- Python
Published by bouthilx about 7 years ago

orion - ASHA, simpler DB setup and new DB backend for filesystem

New Features

• Add ASHA :tada: #187
• Add command orion setup to create a database configuration file #192
• Add command orion test-db to test database setup #188
• Add PickledDB to support FileSystem backend #190
• Add port option for db backend #174
• Add multi-fidelity dimensions #177
• Add working-dir argument #184 ## Bug Fixes
• Handle bad socket.hostname (OSX) #179
• Handle bool arg in EVC Conflict #180
• Fix null comparison in EphemeralDB #182 ## Other Improvements
• Introduce OrionCmdlineParser, a specialized type of CmdlineParser for Oríon specific parsing needs like priors' expression and configuration files. #172
• Add error message for missing configuration #178
• Handle bad results in trials #181
• Add timeout for connection to Database #183
• Add statedict/setstate to algos to serialize or synchronize states #185 ## Documentation Improvements
• Adapt documentation to new algos and database commands #193

- Python
Published by bouthilx about 7 years ago

orion - Relax git constraint, make user name customizable and fix some bugs

Breaking Changes

• Relax git commit constraint for version control #157
• Set default pool-size value to 1 #170 ## Bug Fixes
• Properly set RNG state in algo #167
• Make worker count executed trials properly #171 ## Other Improvements
• Make user name in experiment customizable with --user #169
  ## Documentation Improvements
• Fix some slips #158 #160 #163

- Python
Published by bouthilx over 7 years ago

orion - Git commit detection, Strategies and Trials history

Breaking Changes

• Add git commit detection for version control #115
• Add Strategies with Trials history #152 ## Additional New Features
• Add hunt --worker-trials argument #127
• Add node property to experiment for the EVC system. #146
• Add a generic commandline parser #155 ## Bug Fixes
• Set verbosity >= 2 to DEBUG #123
• Fix github links #126 ## Deprecation
• Drop support for py34 and add py37 #140
• Fix mongodb deprecation #151 ## Other Improvements
• Add recipe for conda packages #141 ## Documentation Improvements
• Updated instructions for Atlas MongoDB. Added instructions for non-root user #131
• Clarify doc section name #133
• Add RL example #136
• Fix README utf-8 encoding #117
• Changed URLs from mila-udem to epistimio #139
• Add simple tutorial #153

- Python
Published by bouthilx over 7 years ago

orion - Initial release

Initial Release :confetti_ball: :tada:

Includes

1. MongoDB backend
2. Commandline and configuration file agnostic support
3. Experiment Version Control
4. Manual insertion of new points in search space
5. Algorithms: Random Search, Wrapper of Scikit-optimize bayesian optimizer (plugin orion.algo.skopt)
6. Transformers of dimensions

Working on and soon available

1. Detection of code changes for scripts inside git repositories
2. Dynamic algorithms and schedulers
3. Visualizations
4. Parallel strategies
5. Full reproducibility with log of algorithm history and synchronized Random Number Generators

- Python
Published by bouthilx about 8 years ago