Recent Releases of kubeflow-katib

kubeflow-katib - Katib v0.18.0

This is the Katib v0.18.0 release.

The key highlights:

KEP-2339: Hyperparameter optimization for LLMs fine-tuning.
KEP-2340: Support for push-based metrics collection.
KEP-2374: Advanced parameter distribution, such as uniform, log-uniform, normal, and log-normal.

Breaking Changes

Move Katib manifest image references to ghcr (#2535 by @saileshd1402)
Migrate docker images to ghcr (#2531 by @mahdikhashan)
Upgrade Kubernetes to v1.31.3 (#2478 by @Electronic-Waste)
Upgrade Kubernetes to v1.30.7 (#2463 by @Electronic-Waste)
Drop Python 3.7 and Support Python 3.11 in the SDK (#2337 by @tenzen-y)

New Features

Hyperparameter Optimization for LLMs

[DOCS] move llm hyperparameter optimisation design image to the proposal directory and rename it (#2472 by @mahdikhashan)
[GSoC] Update tune API for LLM hyperparameters optimization (#2393 by @helenxie-bit)
[GSoC] Create LLM Hyperparameters Optimization API Proposal (#2333 by @helenxie-bit)

Support for Advanced Distributions for HPO

[GSOC] optuna suggestion service logic update (#2446 by @shashank-iitbhu)
[GSOC] hyperopt suggestion service logic update (#2412 by @shashank-iitbhu)
[GSOC] Add validator for feasible space distribution (#2404 by @shashank-iitbhu)
[GSOC] added Unknown distribution and convertDistribution in suggestion client (#2403 by @shashank-iitbhu)
[GSOC] Support for various Parameter distributions in Katib (#2334 by @shashank-iitbhu)
[GSoC] Added DistributionType to Experiment API (#2377 by @shashank-iitbhu)

Push-based Metrics Collector

[GSoC] Provide a PyTorch MNIST Example for Push-based Metrics Collection (#2437 by @Electronic-Waste)
[GSoC] Compatibility Changes in Trial Controller (#2394 by @Electronic-Waste)
[GSoC] New Interface report_metrics in Python SDK (#2371 by @Electronic-Waste)
[GSoC] KEP for Project 6: Push-based Metrics Collection for Katib (#2328 by @Electronic-Waste)
[GSoC] Add New Parameter in tune (#2369 by @Electronic-Waste)

SDK Updates

[SDK] Support PyTorchJob as a Trial Worker (#2512 by @andreyvelich)
[SDK] test: Add e2e test for tune function. (#2399 by @Electronic-Waste)
[SDK] improve PVC creation name error (#2496 by @mahdikhashan)
[SDK] Fix empty list for env variables and numpy version (#2360 by @andreyvelich)
[SDK] Explain Python version support cycle (#2354 by @andreyvelich)

Bug Fixes

fix(webhook): fix validation message in experiment webhook (#2507 by @Electronic-Waste)
Install typing-extensions v4.10.0 to fix Python test error (#2504 by @helenxie-bit)
[SDK] Update tune API (#2497 by @helenxie-bit)
fix(api): resolve all api voilation exceptions in katib api (#2482 by @truc0)
fix(trial): use propagated gomega to improve debuggability. (#2432 by @Electronic-Waste)
fix(ui): update None Collector with Push Collector. (#2418 by @Electronic-Waste)
fix: Resolve errors in e2e tests for cypress in Katib UI (#2384 by @tariq-hasan)
doc(example): fix the broken link. (#2433 by @Electronic-Waste)
fix: remove remaining MXNet dependency. (#2456 by @Electronic-Waste)
Remove Dropout layer from ENAS Trial container to fix E2E tests (#2455 by @andreyvelich)
[SDK] fix grpc related bugs in Python SDK (#2398 by @Electronic-Waste)
[SDK] Fix types error (#2424 by @helenxie-bit)
fix: remove the dependency of protocmp in google.golang.org/protobuf/testing/protocmp. (#2391 by @Electronic-Waste)
Fix TestReconcileBatchJob (#2350 by @forsaken628)
Fix apple silicon rosetta error when building images from the source code (#2342 by @helenxie-bit)
fix katib use crds token pipeline trail template guide (#2330 by @Jerry-yz)
Fix Scikit-Learn Version for Skopt Tests (#2336 by @andreyvelich)

Misc

Support old-style TensorFlow events (tensorboard) (#2517 by @garymm)
Set experiment names at a max of 40 characters. (#2468 by @AydanPirani)
[CI] optimize katib ui dockerfile (#2505 by @mahdikhashan)
Sort experiments by descending creation date by default in katib-ui (#2498 by @Doris-xm)
[GSoC] Add unit tests for tune API (#2423 by @helenxie-bit)
Update MutatingWebhookConfiguration: Switch from objectSelector to AdmissionWebhookMatchConditions (#2241 by @lianghao208)
chore: supporting the listen-address parameter on db-manager (#2465 by @caiofralmeida)
Upgrade klog to v2 (#2470 by @Doris-xm)
Ignore cache exporting errors in the image building workflows (#2487 by @Doris-xm)
Upgrade grpcio version to v1.64.1 (#2483 by @Electronic-Waste)
docs: remove katib workflow (#2443 by @gonmmarques)
Migrate KatibCertGenerator to OPA CertController (#2345 by @forsaken628)
Promote @Electronic-Waste and @helenxie-bit as Katib reviewers (#2439 by @andreyvelich)
Update README and out-of-date docs (#2438 by @andreyvelich)
Changes isort profile to black, to be fully compatible and adds 'pkg' dir for black and flake8 (#2413 by @Ygnas)
Introduced error constants and replaced reflect with cmp (#2289 by @tariq-hasan)
[Test] Refactor inject_webhook_test.go according to the Developer Guide (#2401 by @Electronic-Waste)
Enhance pre-commit hooks with flake8 and black (#2407 by @Ygnas)
added Distribution field to feasibleSpace in api.proto (#2397 by @shashank-iitbhu)
Begin enabling pre-commit hooks (#2242 by @droctothorpe)
Update Instructions for Argo Workflows (#2382 by @jaffe-fly)
docs: update suggestion.md (#2387 by @eltociear)
Add command to re-run GitHub Actions tests (#2385 by @andreyvelich)
Bump Katib Python SDK to 0.17.0 version (#2379 by @andreyvelich)
Add Changelog for Katib v0.17.0 (#2380 by @andreyvelich)
Replaced hpcloud with nxadm for tail package in Go (#2375 by @tariq-hasan)
Use ErrorList for experiment validator (#2329 by @ckcd)
Add Changelog for Katib v0.17.0-rc.1 (#2370 by @andreyvelich)
Remove default caBundle value (#2368 by @vihangm)
Bump Katib Python SDK to 0.17.0rc1 version (#2365 by @andreyvelich)
Add unit test for create_experiment in the katib_client module (#2325 by @tariq-hasan)
Remove code generation from release script (#2363 by @andreyvelich)
Upgrade the protobuf version to >=4.21.12,<5 (#2358 by @tenzen-y)
Replace gRPC code generation tool from Znly/protoc to Buf (#2344 by @forsaken628)
Replace already closed github.com/golang/mock with go.uber.org/mock (#2357 by @forsaken628)
Use cache-dependency-path in actions/setup-go for CI workflow (#2355 by @forsaken628)
Update Slack Invitation (#2349 by @andreyvelich)
Update GitHub template to better triage Issues (#2335 by @andreyvelich)
Add Changelog for Katib v0.17.0-rc.0 (#2319 by @andreyvelich)
Update outdated actions (#2324 by @Mersho)
Make test fields private in Go unit tests (#2316 by @tariq-hasan)
Bump Katib Python SDK to 0.17.0rc0 Version (#2318 by @andreyvelich)

New Contributors

@saileshd1402 made their first contribution in #2535
@mahdikhashan made their first contribution in #2496
@helenxie-bit made their first contribution in #2333
@shashank-iitbhu made their first contribution in #2334
@truc0 made their first contribution in #2482
@tariq-hasan made their first contribution in #2289
@Ygnas made their first contribution in #2407
@jaffe-fly made their first contribution in #2382
@eltociear made their first contribution in #2387
@vihangm made their first contribution in #2368

- Python
Published by andreyvelich about 1 year ago

kubeflow-katib - Katib v0.18.0-rc.0

This is the Katib v0.18.0-rc.0 release.

The key highlights: - KEP-2339: Hyperparameter optimization for LLMs fine-tuning. - KEP-2340: Support for push-based metrics collection. - KEP-2374: Advanced parameter distribution, such as uniform, log-uniform, normal, and log-normal.

Breaking Changes

Upgrade Kubernetes to v1.31.3 (#2478 by @Electronic-Waste)
Upgrade Kubernetes to v1.30.7 (#2463 by @Electronic-Waste)
Drop Python 3.7 and Support Python 3.11 in the SDK (#2337 by @tenzen-y)

New Features

Hyperparameter Optimization for LLMs

[DOCS] move llm hyperparameter optimisation design image to the proposal directory and rename it (#2472 by @mahdikhashan)
[GSoC] Update tune API for LLM hyperparameters optimization (#2393 by @helenxie-bit)
[GSoC] Create LLM Hyperparameters Optimization API Proposal (#2333 by @helenxie-bit)

Push-based metrics collection

[GSoC] Provide a PyTorch MNIST Example for Push-based Metrics Collection (#2437 by @Electronic-Waste)
[GSoC] Compatibility Changes in Trial Controller (#2394 by @Electronic-Waste)
[GSoC] New Interface report_metrics in Python SDK (#2371 by @Electronic-Waste)
[GSoC] KEP for Project 6: Push-based Metrics Collection for Katib (#2328 by @Electronic-Waste)
[GSoC] Add New Parameter in tune (#2369 by @Electronic-Waste)

Advanced distributions for HPO

[GSOC] optuna suggestion service logic update (#2446 by @shashank-iitbhu)
[GSOC] hyperopt suggestion service logic update (#2412 by @shashank-iitbhu)
[GSOC] Add validator for feasible space distribution (#2404 by @shashank-iitbhu)
[GSOC] added Unknown distribution and convertDistribution in suggestion client (#2403 by @shashank-iitbhu)
[GSOC] Support for various Parameter distributions in Katib (#2334 by @shashank-iitbhu)
[GSoC] Added DistributionType to Experiment API (#2377 by @shashank-iitbhu)

SDK Updates

[SDK] Support PyTorchJob as a Trial Worker (#2512 by @andreyvelich)
[SDK] test: Add e2e test for tune function. (#2399 by @Electronic-Waste)
[SDK] improve PVC creation name error (#2496 by @mahdikhashan)
[SDK] Fix empty list for env variables and numpy version (#2360 by @andreyvelich)
[SDK] Explain Python version support cycle (#2354 by @andreyvelich)

Bug Fixes

fix(webhook): fix validation message in experiment webhook (#2507 by @Electronic-Waste)
Install typing-extensions v4.10.0 to fix Python test error (#2504 by @helenxie-bit)
[SDK] Update tune API (#2497 by @helenxie-bit)
fix(api): resolve all api voilation exceptions in katib api (#2482 by @truc0)
fix(trial): use propagated gomega to improve debuggability. (#2432 by @Electronic-Waste)
fix(ui): update None Collector with Push Collector. (#2418 by @Electronic-Waste)
fix: Resolve errors in e2e tests for cypress in Katib UI (#2384 by @tariq-hasan)
doc(example): fix the broken link. (#2433 by @Electronic-Waste)
fix: remove remaining MXNet dependency. (#2456 by @Electronic-Waste)
Remove Dropout layer from ENAS Trial container to fix E2E tests (#2455 by @andreyvelich)
[SDK] fix grpc related bugs in Python SDK (#2398 by @Electronic-Waste)
[SDK] Fix types error (#2424 by @helenxie-bit)
fix: remove the dependency of protocmp in google.golang.org/protobuf/testing/protocmp. (#2391 by @Electronic-Waste)
Fix TestReconcileBatchJob (#2350 by @forsaken628)
Fix apple silicon rosetta error when building images from the source code (#2342 by @helenxie-bit)
fix katib use crds token pipeline trail template guide (#2330 by @Jerry-yz)
Fix Scikit-Learn Version for Skopt Tests (#2336 by @andreyvelich)

Misc

Set experiment names at a max of 40 characters. (#2468 by @AydanPirani)
[CI] optimize katib ui dockerfile (#2505 by @mahdikhashan)
Sort experiments by descending creation date by default in katib-ui (#2498 by @Doris-xm)
[GSoC] Add unit tests for tune API (#2423 by @helenxie-bit)
Update MutatingWebhookConfiguration: Switch from objectSelector to AdmissionWebhookMatchConditions (#2241 by @lianghao208)
chore: supporting the listen-address parameter on db-manager (#2465 by @caiofralmeida)
Upgrade klog to v2 (#2470 by @Doris-xm)
Ignore cache exporting errors in the image building workflows (#2487 by @Doris-xm)
Upgrade grpcio version to v1.64.1 (#2483 by @Electronic-Waste)
docs: remove katib workflow (#2443 by @gonmmarques)
Migrate KatibCertGenerator to OPA CertController (#2345 by @forsaken628)
Promote @Electronic-Waste and @helenxie-bit as Katib reviewers (#2439 by @andreyvelich)
Update README and out-of-date docs (#2438 by @andreyvelich)
Changes isort profile to black, to be fully compatible and adds 'pkg' dir for black and flake8 (#2413 by @Ygnas)
Introduced error constants and replaced reflect with cmp (#2289 by @tariq-hasan)
[Test] Refactor inject_webhook_test.go according to the Developer Guide (#2401 by @Electronic-Waste)
Enhance pre-commit hooks with flake8 and black (#2407 by @Ygnas)
added Distribution field to feasibleSpace in api.proto (#2397 by @shashank-iitbhu)
Begin enabling pre-commit hooks (#2242 by @droctothorpe)
Update Instructions for Argo Workflows (#2382 by @jaffe-fly)
docs: update suggestion.md (#2387 by @eltociear)
Add command to re-run GitHub Actions tests (#2385 by @andreyvelich)
Bump Katib Python SDK to 0.17.0 version (#2379 by @andreyvelich)
Add Changelog for Katib v0.17.0 (#2380 by @andreyvelich)
Replaced hpcloud with nxadm for tail package in Go (#2375 by @tariq-hasan)
Use ErrorList for experiment validator (#2329 by @ckcd)
Add Changelog for Katib v0.17.0-rc.1 (#2370 by @andreyvelich)
Remove default caBundle value (#2368 by @vihangm)
Bump Katib Python SDK to 0.17.0rc1 version (#2365 by @andreyvelich)
Add unit test for create_experiment in the katib_client module (#2325 by @tariq-hasan)
Remove code generation from release script (#2363 by @andreyvelich)
Upgrade the protobuf version to >=4.21.12,<5 (#2358 by @tenzen-y)
Replace gRPC code generation tool from Znly/protoc to Buf (#2344 by @forsaken628)
Replace already closed github.com/golang/mock with go.uber.org/mock (#2357 by @forsaken628)
Use cache-dependency-path in actions/setup-go for CI workflow (#2355 by @forsaken628)
Update Slack Invitation (#2349 by @andreyvelich)
Update GitHub template to better triage Issues (#2335 by @andreyvelich)
Add Changelog for Katib v0.17.0-rc.0 (#2319 by @andreyvelich)
Update outdated actions (#2324 by @Mersho)
Make test fields private in Go unit tests (#2316 by @tariq-hasan)
Bump Katib Python SDK to 0.17.0rc0 Version (#2318 by @andreyvelich)

- Python
Published by andreyvelich over 1 year ago

kubeflow-katib - Katib v0.17.0

This is the Katib v0.17.0 release.

Breaking Changes

[SDK] Drop Python 3.7 and Support Python 3.11 (#2337 by @tenzen-y)
[SDK] Upgrade the protobuf version to >=4.21.12,<5 (#2358 by @tenzen-y)
Drop Kubernetes v1.26, and support Kubernetes v1.29 (#2308 by @tenzen-y)
Drop Kubernetes v1.25, and Support Kubernetes v1.28 (#2303 by @tenzen-y)
Remove MXNet examples (#2267 by @tenzen-y)

New Features

Core Features

Replace gRPC code generation tool from Znly/protoc to Buf (#2344 by @forsaken628)
Support ARM64 arch for release images (#2315 by @andreyvelich)
DB: Add environment variable option to skip DB table creation (#2245 by @lkaybob)
Add environment variable option to set postgres ssl mode (#2266 by @ckcd)
Upgrade TensorFlow version to v2.16.1 (#2282 by @tenzen-y)
Upgrade PyTorch version to v2.2.1 (#2279 by @tenzen-y)

SDK Features

[SDK] Generate Name functionality for creating experiments. (#2272 by @bharathk005)
[SDK] Add env & env_from in client tune (#2235 by @shipengcheng1230)
[SDK] Add 'algorithm_settings' in client tune (#2227 by @shipengcheng1230)
[SDK] Raise more human-readable name conflict exception (#2199 by @droctothorpe)

Bug Fixes

Remove code generation from release script (#2364 by @andreyvelich)
[SDK] Fix empty list for env variables and numpy version (#2360 by @andreyvelich)
Use cache-dependency-path in actions/setup-go for CI workflow (#2355 by @forsaken628)
Fix TestReconcileBatchJob (#2350 by @forsaken628)
Fix Scikit-Learn Version for Skopt Tests (#2336 by @andreyvelich)
[SDK] Fix env per Trial parameter in tune API (#2304 by @andreyvelich)
Fix: clean up UTs for file metrics collector (#2285 by @Electronic-Waste)
Fix tensor devices for DARTS Trial (#2273 by @sifa1024)
Typo fix stale.yaml (#2257 by @tarilabs)
Fix Optuna Validation for CMA-ES (#2240 by @andreyvelich)

Misc

Replace already closed github.com/golang/mock with go.uber.org/mock (#2357 by @forsaken628)
Update outdated actions (#2324 by @Mersho)
Upgrade Go version to v1.22 (#2309 by @tenzen-y)
CI: Enable parallel mode for the coveralls (#2297 by @tenzen-y)
Upgrade Python version to 3.11 (#2278 by @tenzen-y)
chore: add unit testcases for files in Text format. (#2274 by @Electronic-Waste)
Upgrade google/go-containerregistry/pkg/authn/k8schain (#2252 by @tenzen-y)
Add Technical and style guide to the contribution guide (#2250 by @tenzen-y)
Install typing-extensions v4.6.3 for Optuna (#2251 by @tenzen-y)
Remove legacy BO code (#2246 by @andreyvelich)
Add Changelog for Katib v0.16.0 (#2239 by @andreyvelich)
Add Katib ROADMAP 2022/2023 (#2153 by @andreyvelich)
Update Ubuntu to 22.04 for E2E Tests (#2222 by @andreyvelich)
Run Stale Action Every 5th Hour (#2221 by @andreyvelich)
Add Stale GitHub Action (#2220 by @andreyvelich)
Add Changelog for Katib v0.16.0-rc.1 (#2218 by @andreyvelich)
Add Changelog for Katib v0.16.0-rc.0 (#2204 by @andreyvelich)
Use the controller-runtime logger in the cert-generator (#2219 by @tenzen-y)

New Contributors

@forsaken628 made their first contribution in #2344
@lkaybob made their first contribution in #2245
@ckcd made their first contribution in #2266
@bharathk005 made their first contribution in #2272
@shipengcheng1230 made their first contribution in #2227
@Electronic-Waste made their first contribution in #2274
@sifa1024 made their first contribution in #2273
@tarilabs made their first contribution in #2257
@Mersho made their first contribution in #2324

- Python
Published by andreyvelich almost 2 years ago

kubeflow-katib - Katib v0.16.0

This is the Katib v0.16.0 release.

Breaking Changes

Implement KatibConfig API (#2176 by @tenzen-y)
Drop Kubernetes v1.24 and support Kubernetes v1.27 (#2182 by @tenzen-y)
Drop Kubernetes v1.23 and support Kubernetes v1.26 (#2177 by @tenzen-y)
Change failurePolicy to Fail for Katib Webhooks (#2018 by @andreyvelich)

New Features

Core Features

Consolidate the Katib Cert Generator to the Katib Controller (#2185 by @tenzen-y)
Containerize tests for Katib Conformance (#2146 by @nagar-ajay)

UI Improvements

[UI] Default Resume Policy to never from UI (#2195 by @mChowdhury-91)
[UI] Remove Deprecated Katib UI (#2179 by @andreyvelich)
[UI] Fix Trial Logs when Kubernetes Job Fails (#2164 by @andreyvelich)
kwa(front): Support all namespaces (#2119 by @elenzio9)
kwa(front): Update the use of SnackBarService (#2113 by @orfeas-k)
UI: Remove an unsed import, EventV1beta1Api (#2116 by @tenzen-y)

SDK Improvements

[SDK] Enable resource specification for trial containers (#2192 by @droctothorpe)
[SDK] Add namespace parameter to KatibClient (#2183 by @droctothorpe)
[SDK] Import all Kubernetes Models (#2148 by @andreyvelich)

Bug fixes

Bug: Wait for the certs to be mounted inside the container (#2213 by @tenzen-y)
Start waiting for certs to be ready before sending data to the channel (#2215 by @tenzen-y)
E2E: Add additional checks to verify if the components are ready (#2212 by @tenzen-y)
Remove a katib-webhook-cert Secret from components (#2214 by @tenzen-y)
Skip to inject the metrics-collector pods to the Katib controller (#2211 by @tenzen-y)
Sending an empty data to the certsReady channel (#2196 by @tenzen-y)
Fix conformance docker image (#2147 by @nagar-ajay)

Documentation

Add PITS Global Data Recovery Services to the adopters list (#2160 by @ghost)
Add SDK Breaking Change to Changelog (#2133 by @andreyvelich)
Add Changelog for Katib v0.15.0 (#2129 by @andreyvelich)
Add Changelog for Katib v0.15.0-rc.1 (#2123 by @andreyvelich)
Add Changelog for Katib v0.15.0-rc.0 (#2106 by @andreyvelich)

Misc

Upgrade Tensorflow version to v2.13.0 (#2216 by @tenzen-y)
Upgrade Go version to v1.20 (#2190 by @tenzen-y)
Replace grpchealthprobe with the built-in gRPC container probe feature (#2189 by @tenzen-y)
Allow install binaries for the arm64 in the envtest (#2188 by @tenzen-y)
Replace action to setup minikube with medyagh/setup-minikube (#2178 by @tenzen-y)
Remove Charmed Operators for Katib (#2161 by @ca-scribner)
Namespace and trial pod annotations as CLI argument (#2138 by @nagar-ajay)
Relax dependencies restriction for the gRPC libraries (#2140 by @tenzen-y)
Add SDK Breaking Change to Changelog (#2133 by @andreyvelich)
Increase the free spaces in CI (#2131 by @tenzen-y)
Reformat katib-operators (#2114 by @tenzen-y)

- Python
Published by andreyvelich over 2 years ago

kubeflow-katib - Katib v0.15.0

This is the Katib v0.15.0 release.

Breaking Changes

Use Never Resume Policy as Default (#2102 by @andreyvelich)
Chocolate Suggestion Service is removed (#2071 by @tenzen-y)
request_number is removed from the GRPC APIs (#1994 by @johnugeorge)
Enabling Authorization in Katib UI (#1983 and #2041 by @apo-ger)
The new improved and refactored Katib SDK is not backward compatible (#2075 by @andreyvelich)

New Features

Major Features

Narrow down Katib RBAC rules (#2091 by @johnugeorge)
Support Postgres as a Katib DB (#1921 by @anencore94)
More Suggestion container fields in Katib Config (#2000 by @fischor)
Katib UI: Create the LOGS tab of Trial's details page (#2117 by @elenzio9)
Katib UI: Enable pagination/sorting/filtering (#2017 and #2040 by @elenzio9)
[SDK] Create Tune API in the Katib SDK (#1951 by @andreyvelich)
[SDK] Get Trial Metrics from Katib DB (#2050 by @andreyvelich)

Core Features

Add Conformance Program Doc for AutoML and Training WG (#2048 by @andreyvelich)
Support for grid search algorithm in Optuna Suggestion Service (#2060 by @tenzen-y)
Add Trial Labels During Pod Mutation (#2047 by @andreyvelich)
Support for k8s v1.25 in CI (#1997 by @johnugeorge)
Add the CI to build multi-platform container images (#1956 by @tenzen-y)
Drop Kubernetes v1.21 and introduce Kubernetes v1.24 (#1953 by @tenzen-y)
Add --connect-timeout flag to katib-db-manager (#1937 by @tenzen-y)
Implement validations for DARTS suggestion service (#1926 by @tenzen-y)
Implement validation for Optuna suggestion service (#1924 by @tenzen-y)

UI Improvements

Make links in KWA's tables actual links (#2090 by @elenzio9)
frontend: Rework the trial graph using ECharts in KWA (#2089 by @elenzio9)
kwa(front): Add UI tests with Cypress (#2088 by @orfeas-k)
frontend: Enable actions in experiment graph (#2065 by @elenzio9)
frontend: Show message in case of uncompleted trial instead of the graph (#2063 by @elenzio9)
frontend: Add source maps in the browser (#2043 by @elenzio9)
Backend for getting logs of a trial (#2039 by @d-gol)
frontend: Show the successful trials in the experiment graph (#2013) (#2033 by @elenzio9)
frontend: Migrate from tslint to eslint in KWA (#2042 by @elenzio9)
Dedicated yaml tab for Trials (#2034 by @elenzio9)
KWA: Use new Editor component (Monaco) (#2023 by @orfeas-k)
kwa(build): Introduce COMMIT file for building KWA (#2014 by @orfeas-k)
frontend: Fix 500 error after detail page refresh (#1967) (#2001 by @elenzio9)
Introduce KWA's frontend component for kfp links (#1991 by @elenzio9)
UI: Rename and right align the age column (#1989 by @elenzio9)
Show the trials table's status column first (#1990 by @elenzio9)
UI: Make KWA's main table responsive and add toolbar (#1982 by @elenzio9)
UI: Fix unit tests (#1977 by @elenzio9)
UI: Format code (#1979 by @orfeas-k)
Recreate the Experiments Parallel Coordinates Graph (#1974 by @elenzio9)
Improve UI API/controller logging to ease troubleshooting (#1966 by @lukeogg)

SDK Improvements

[SDK] Use Katib SDK for E2E Tests (#2075 by @andreyvelich)
[SDK] Use Katib Client without Kube Config (#2098 by @andreyvelich)
[SDK] Fix namespace parameter in tune API (#1981 by @andreyvelich)
[SDK] Remove Final Keyword from constants (#1980 by @andreyvelich)

Bug fixes

Fix Release Script for Updating SDK Version (#2104 by @andreyvelich)
[Fix] add early stopped trials in converter (#2004 by @shaowei-su)
[bugfix] Fix value passing bug in New Experiment form (#2027 by @orfeas-k)
Fix main process retrieve logic for early stopping (#1988 by @shaowei-su)
[hotfix]: filter by name of experiment (#1920 by @anencore94)
Fix push script to include new images (#1911 by @johnugeorge)
fix: only validate Kubernetes Job (#2025 by @zhixian82)
Upgrade grpc-health-probe version to fix some security issues (#2093 by @tenzen-y)
Format Katib Charm Operator (#2115 by @tenzen-y)

Documentation

Add CERN to adopters (#2010 by @d-gol)
Add More Katib Presentations 2022 (#2009 by @andreyvelich)
Add the documentation for simple-pbt (#1978 by @tenzen-y)
Add the license to pbt (#1958 by @tenzen-y)
Update the Katib version in docs (#1950 by @tenzen-y)
Update CHANGELOG for v0.14.0 release (#1932 by @johnugeorge)

Misc

Update Training operator Image in CI (#2103 by @johnugeorge)
Upgrade Go libraries to resolve security issues (#2094 by @tenzen-y)
Run e2e with various Python versions to verify Python SDK (#2092 by @tenzen-y)
Add a --prefer-binary flag to 'pip install' command (#2096 by @tenzen-y)
Upgrade PyTorch version to v1.13.0 (#2082 by @tenzen-y)
Upgrade Tensorflow version (#2079 by @tenzen-y)
Upgrade Python version to 3.10 (#2057 by @tenzen-y)
Pin the NumPy version with v1.23.5 in some images (#2070 by @tenzen-y)
Upgrade the actions-setup-minikube version to v2.7.2 (#2064 by @tenzen-y)
Remove Certificate Chain from Cert Generator (#2045 by @andreyvelich)
Add resources to earlystopping container (#2038 by @zhixian82)
Add scripts to verify generated codes and Go Modules (#1999 by @tenzen-y)
[Test] Reduce Katib GitHub Action Runs (#2036 by @andreyvelich)
gh-actions: Extend action to run Frontend Unit tests (#1998 by @orfeas-k)
[chore] Upgrade docker/metadata-action, actions/checkout, and actions/setup-python version (#1996 by @tenzen-y)
[chore] Upgrade Go version to v1.19 (#1995 by @tenzen-y)
Support for arm64 in simple-pbt image (#1948 by @tenzen-y)
Support arm64 in darts-cnn-cifar10 image (#1947 by @tenzen-y)
Support for arm64 in enas-cnn-cifar10 image (#1944 by @tenzen-y)
Support for arm64 in pytorch-mnist image (#1943 by @tenzen-y)
Support for arm64 in mxnet-mnist image (#1940 by @tenzen-y)
Use the katib-new-ui for Charmed gh-actions (#1987 by @tenzen-y)
[feat] health check for katib-controller (#1934 by @anencore94)
Upgrade Optuna from v2.x.x to v3.0.0 (#1942 by @keisuke-umezawa)
Add validation webhooks for maxFailedTrialCount and parallelTrialCount (#1936 by @tenzen-y)
Introduce Automatic platform ARGs (#1935 by @tenzen-y)
Update training operator image in CI (#1933 by @johnugeorge)
Update Katib SDK version (#1931 by @johnugeorge)
[chore] Upgrade Go version to v1.18 (#1925 by @tenzen-y)
Add the pytorch-mnist with GPU support container image (#1916 by @tenzen-y)

Changelog

Full Changelog

- Python
Published by andreyvelich about 3 years ago

kubeflow-katib - Katib v0.14.0

This is the Katib v0.14.0 release.

New Features

Core Features

Population based training (#1833 by @a9p)
Support JSON format logs in file-metrics-collector (#1765 by @tenzen-y)
Include MetricsUnavailable condition to Complete in Trial (#1877 by @tenzen-y)
Allow running examples on Apple Silicon M1 and fix image build errors for arm64 (#1898 by @tenzen-y)
Configurable job name and service name for cert generator (#1889 by @shaowei-su)

UI Features and Enhancements

Add PBT to experiment creation form (#1909 by @a9p)
Distinct page for each Trial in the UI (#1783 by @d-gol)

Bug fixes

Add the pytorch-mnist with GPU support container image (#1917 by @tenzen-y)
Fix push script to include new images (#1912 by @johnugeorge)
Fixes lint warnings in YAML files (#1902 by @Rishit-dagli)
Fix errors when running the test on Apple Silicon M1 (#1886 by @tenzen-y)
Reconcile trial assignments by comparing suggestion and trials being executed (#1831 by @henrysecond1)
Increate the probes seconds in manifests (#1845 by @haoxins)
Set upper constraint for Optuna (#1852 by @himkt)
Don't check if trial's metadata is in spec.parameters (#1848 by @alexeygorobets)

Documentation

Fix the FPGA examples documentation (#1841 by @eliaskoromilas)
Add CyberAgent to adopters (#1894 by @tenzen-y)

Misc

Updating the training operator image in CI (#1910 by @johnugeorge)
Upgrade Python and Pytorch versions for some examples (#1906 by @tenzen-y)
Linting for K8s YAML files (#1901 by @Rishit-dagli)
Change integration test sysytem from KinD Cluster to Minikube Cluster (#1899 by @tenzen-y)
Upgrade mysql version to v8.0.29 (#1897 by @tenzen-y)
Upgrade tensorflow-aarch64 version to v2.9.1 (#1891 by @tenzen-y)
chore: Upgrade Go libraries to resolve some security issues in the katib-controller (#1888 by @tenzen-y)
Migrate kubeflow-katib-presubmit to GitHub Actions (#1882 by @tenzen-y)
Add semicolon when using command command in Makefile (#1885 by @tenzen-y)
Fix HAS_SHELLCHECK and HAS_SETUP_ENVTEST in Makefile (#1884 by @tenzen-y)
Remove presubmit tests depending on optional-test-infra (#1871 by @aws-kf-ci-bot)
Upgrade the Tensorflow version to address some security issues (#1870 by @tenzen-y)
Upgrade the grpchealthprobe version to v0.4.11 to resolve security vulnerability CVE-2022-27191 (#1875 by @tenzen-y)
additional metric names should not include objective metric name (#1874 by @henrysecond1)
Upgrade the Kubernetes Python client to 22.6.0 (#1869 by @tenzen-y)
Upgrade the kubebuilder to v3.2.0 and Kubernetes Go libraries to v1.22.2 (#1861 by @tenzen-y)
Update FPGA XGBoost example (#1865 by @eliaskoromilas)
Fix kubeflowkatib/mxnet-mnist image (#1866 by @tenzen-y)
pins pip and setuptools versions operators to avoid installation issues (#1867 by @DnPlas)
Add shellcheck (#1857 by @tenzen-y)
Bump kubeflow-katib and kfp version in notebook examples (#1849 by @tenzen-y)
Add prometheus scraping and grafana support to charmed katib-controller operator (#1839 by @jardon)
Upgrade Black to fix linting (#1842 by @jardon)

Change Log

Check the Full Change Log.

- Python
Published by johnugeorge almost 4 years ago

kubeflow-katib - Katib v0.14.0-rc.0

- Python
Published by johnugeorge almost 4 years ago

kubeflow-katib - Katib v0.13.0

This is the Katib v0.13.0 release.

Breaking changes:

Namespace label for Metrics collector enabled Katib namespaces is changed to katib.kubeflow.org/metrics-collector-injection=enabled #1740
Current request number field in gRPC API is renamed to current_request_number #1728
training.kubeflow.org prefix is added to the default primary pod labelsjob-role and replica-type of the Training Operators #1813

New Features

### Algorithms and Components

Implement validation for Early Stopping (#1709 by @tenzen-y)
Change namespace label for Metrics Collector injection (#1740 by @andreyvelich)
Modify gRPC API with Current Request Number (#1728 by @andreyvelich)
Allow to remove each resource in Katib config (#1729 by @andreyvelich)
Support leader election for Katib Controller (#1713 by @tenzen-y)
Change default Metrics Collect format (#1707 by @anencore94)
Bump Python version to 3.9 (#1731 by @tenzen-y)
Update Go version to 1.17 (#1683 by @andreyvelich)
Create Python script to run e2e Argo Workflow (#1674 by @andreyvelich)
Reimplement Katib Cert Generator in Go (#1662 by @tenzen-y)
SDK: change list apis to return objects as default (#1630 by @anencore94)

### UI Features

Enhance Katib UI feasible space (#1721 by @seong7)
Handle missing TrialTemplates in Katib UI (#1652 by @kimwnasptd)
Add Prettier devDependency in Katib UI (#1629 by @seong7)

## Documentation

Fix a link for GRPC API documentation (#1786 by @tenzen-y)
Add my presentations that include Katib (#1753 by @terrytangyuan)
Add Akuity to list of adopters (#1749 by @terrytangyuan)
Change Argo -> Argo Workflows (#1741 by @terrytangyuan)
Update Algorithm Service Doc for the new CI script (#1724 by @andreyvelich)
Update link to Training Operator (#1699 by @terrytangyuan)
Refactor examples folder structure (#1691 by @andreyvelich)
Fix README in examples directory (#1687 by @tenzen-y)
Add Kubeflow MXJob example (#1688 by @andreyvelich)
Update FPGA examples (#1685 by @eliaskoromilas)
Refactor README (#1667 by @andreyvelich)
Change the minimal Kustomize version in the developer guide (#1675 by @tenzen-y)
Add Katib release process guide (#1641 by @andreyvelich)

## Bug Fixes

Remove unrecognized keys from metadata.yaml in Charmed operators (#1759 by @DnPlas)
Fix the default Metrics Collector regex (#1755 by @andreyvelich)
Fix Status Handling in Charmed Operators (#1743 by @DomFleischmann)
Fix bug on list type HP in Katib UI (#1704 by @seong7)
Fix Range for Int and Double values in Grid search (#1732 by @andreyvelich)
Check if parameter references exist in Experiment parameters (#1726 by @henrysecond1)
Fix same set for HyperParameters in Bayesian Optimization algorithm (#1701 by @fabianvdW)
Close MySQL statement and rows resources when SQL exec ends (#1720 by @chenwenjun-github)
Fix Cluster Role of Katib Controller to access image pull secrets (#1725 by @henrysecond1)
Emit events when fails to reconcile all Trials (#1706 by @henrysecond1)
Missing metrics port annotation (#1715 by @alexeykaplin)
Fix absolute value in Katib UI (#1676 by @anencore94)
Add missing omitempty parameter to APIs (#1645 by @andreyvelich)
Reconcile semantics for Suggestion Algorithms (#1633 by @johnugeorge)
Fix default label for Training Operators (#1813 by @andreyvelich)
Update supported Python version for Katib SDK (#1798 by @tenzen-y)

## Misc

Use release tags for Trial images (#1757 by @andreyvelich)
Upgrade cert-manager API from v1alpha2 to v1 (#1752 by @haoxins)
Add Workflow to Publish Katib Images (#1746 by @andreyvelich)
Update Charmed Katib Operators + CI to 0.12 (#1717 by @knkski)
Updating Katib CI to use Training Operator (#1710 by @midhun1998)
Update OWNERS for Charmed operators (#1718 by @ca-scribner)
Implement some unit tests for the Katib Config package (#1690 by @tenzen-y)
Add GitHub Actions for Python unit tests (#1677 by @andreyvelich)
Add OWNERS file for the Katib new UI (#1681 by @kimwnasptd)
Add envtest to check reconcileRBAC (#1678 by @tenzen-y)
Use golangci-lint as linter for Go (#1671 by @tenzen-y)

- Python
Published by johnugeorge about 4 years ago

kubeflow-katib - Katib v0.12.0

This is the Katib v0.12.0 release.

The major advantages: - Optuna Suggestion service with the new algorithms, big thanks to @g-votte and @c-bata. - Sobol's Quasirandom Sequence algorithm and IPOP-CMA-ES or BIPOP-CMA-ES restart strategies, big thanks to @c-bata. - Katib can perform Argo Workflows, big thanks to @andreyvelich.

New Features

Algorithms and Components

Add Optuna based suggestion service (#1613 by @g-votte)
Support Sobol's Quasirandom Sequence using Goptuna. (#1523 by @c-bata)
Bump the Goptuna version up to v0.8.0 with IPOP-CMA-ES and BIPOP-CMA-ES support. (#1519 by @c-bata)
Validate possible operations for Grid suggestion (#1205 by @andreyvelich)
Validate for Bayesian Optimization algorithm settings (#1600 by @anencore94)
Add Support for Argo Workflows (#1605 by @andreyvelich)
Add Support for XGBoost Operator with LightGBM example (#1603 by @andreyvelich)
Allow empty resources for CPU and Memory in Katib config (#1564 by @andreyvelich)
Add kustomization overlay: katib-openshift (#1513 by @maanur)
Switch to SDI in Katib Charm (#1555 by @knkski)

UI Features

Add Multivariate TPE to Katib UI (#1625 by @andreyvelich)
Update Katib UI with Optuna Algorithm Settings (#1626 by @andreyvelich)
Change the default image for the new Katib UI (#1608 by @andreyvelich)

Documentation

Add Katib 2021 ROADMAP (#1524 by @andreyvelich)
Add AutoML and Training WG Summit July 2021 (#1615 by @andreyvelich)
Add the new Katib presentations 2021 (#1539 by @andreyvelich)
Add Doc checklist to PR template (#1568 by @andreyvelich)
Fix typo in operators/README (#1557 by @evilnick)
Adds docs on how to use Katib Charm within KF (#1556 by @RFMVasconcelos)
Fix a link to Kustomize manifest for new Katib UI (#1521 by @c-bata)

Bug Fixes

Fix UI for handling missing params (#1657 by @kimwnasptd)
Reconcile semantics for Suggestion Algorithms (#1644 by @johnugeorge)
Fix Metrics Collector error in case of non-existing Process (#1614 by @andreyvelich)
Fix mysql version in docker image (#1594 by @munagekar)
Fix grep in Tekton Experiment Doc (#1578 by @andreyvelich)
Error messages corrected (#1522 by @himanshu007-creator)
Install charmcraft 1.0.0 (#1593 by @DomFleischmann)

Misc

Modify XGBoostJob example for the new Controller (#1623 by @andreyvelich)
Modify Labels for controller resources (#1621 by @andreyvelich)
Modify Labels for Katib Components (#1611 by @andreyvelich)
Upgrade CRDs to apiextensions.k8s.io/v1 (#1610 by @andreyvelich)
Update Katib SDK with OpenAPI generator (#1572 by @andreyvelich)
Disable default PV for Experiment with resume from volume (#1552 by @andreyvelich)
Remove PV from MySQL component (#1527 by @andreyvelich)
feat: add naming regex check on validating webhook (#1541 by @anencore94)

Change Log

Check the Full Change Log.

- Python
Published by andreyvelich over 4 years ago

kubeflow-katib - Katib v0.11.1

This is the Katib v0.11.1 release.

Bug fixes

Fix Katib manifest for Kubeflow 1.3 (https://github.com/kubeflow/katib/pull/1503 by @yanniszark)
Fix Katib release script (https://github.com/kubeflow/katib/pull/1510 by @andreyvelich)

Enhancements

Remove Application CR (https://github.com/kubeflow/katib/pull/1509 by @yanniszark)
Modify Katib manifest to support newer Kustomize version (https://github.com/kubeflow/katib/pull/1515 by @DavidSpek and @andreyvelich)

Check the Full Change Log.

- Python
Published by andreyvelich almost 5 years ago

kubeflow-katib - Katib v0.11.0

This is the Katib v0.11.0 release. The major advantages: - Katib is now supporting Kubernetes >= 1.18 - Possibility to deploy a new Katib UI, big thanks to @kimwnasptd! - Juju operator support, big thanks to @DomFleischmann, @knkski and @RFMVasconcelos!

New Features

Core Features

Disable dynamic Webhook creation (https://github.com/kubeflow/katib/pull/1450 by @andreyvelich and @tenzen-y)
Add the waitAllProcesses flag to the Katib config (https://github.com/kubeflow/katib/pull/1394 by @robbertvdg)
Migrate Katib to Go modules (https://github.com/kubeflow/katib/pull/1438 by @andreyvelich)
Update Katib SDK with the get_success_trial_details API (https://github.com/kubeflow/katib/pull/1442 by @Adarsh2910)
Add release process script (https://github.com/kubeflow/katib/pull/1473 by @andreyvelich)
Refactor the Katib installation using Kustomize (https://github.com/kubeflow/katib/pull/1464 by @andreyvelich)

UI Features and Enhancements

First step for the Katib new UI implementation (https://github.com/kubeflow/katib/pull/1427 by @kimwnasptd)
Add missing fields to the Katib new UI (https://github.com/kubeflow/katib/pull/1463 by @kimwnasptd)
Add instructions to install the new Katib UI (https://github.com/kubeflow/katib/pull/1476 by @kimwnasptd)

Katib Juju operator

Add Juju operator support for Katib (https://github.com/kubeflow/katib/pull/1403 by @knkski and @RFMVasconcelos)
Add GitHub Actions for the Juju operator (https://github.com/kubeflow/katib/pull/1407 by @knkski)
Add install docs for the Juju operator (https://github.com/kubeflow/katib/pull/1411 by @RFMVasconcelos)
Modify ClusterRoles for the Juju operator (https://github.com/kubeflow/katib/pull/1426 by @DomFleischmann)
Update the Juju operator with the new Katib Webhooks (https://github.com/kubeflow/katib/pull/1465 by @knkski)

Bug fixes

Fix compare step for Early Stopping (https://github.com/kubeflow/katib/pull/1386 by @andreyvelich)
Fix Early Stopping in the Goptuna Suggestion (https://github.com/kubeflow/katib/pull/1404 by @andreyvelich)
Fix SDK examples to work with the Katib 0.10 (https://github.com/kubeflow/katib/pull/1402 by @andreyvelich)
Fix links in the TFEvent Metrics Collector (https://github.com/kubeflow/katib/pull/1417 by @zuston)
Fix the gRPC build script (https://github.com/kubeflow/katib/pull/1492 by @andreyvelich)

Documentation

Modify docs for the Katib 0.10 (https://github.com/kubeflow/katib/pull/1392 by @andreyvelich)
Add Katib presentation list (https://github.com/kubeflow/katib/pull/1446 by @andreyvelich)
Add Canonical to the Katib Adopters (https://github.com/kubeflow/katib/pull/1401 by @RFMVasconcelos)
Update developer guide with the Katib controller flags (https://github.com/kubeflow/katib/pull/1449 by @annajung)
Add Fuzhi to the Katib Adopters (https://github.com/kubeflow/katib/pull/1451 by @Planck0591)
Fix Katib broken links to the Kubeflow guides (https://github.com/kubeflow/katib/pull/1477 by @theofpa)
Add the Katib Webhook docs (https://github.com/kubeflow/katib/pull/1486 by @andreyvelich)

Misc

Add recreate strategy for the MySQL deployment (https://github.com/kubeflow/katib/pull/1393 by @andreyvelich)
Modify worker image for the Katib AWS CI/CD (https://github.com/kubeflow/katib/pull/1423 by @PatrickXYS)
Add the SVG logo for Katib (https://github.com/kubeflow/katib/pull/1414 by @knkski)
Verify empty Objective in the Experiment defaults (https://github.com/kubeflow/katib/pull/1445 by @andreyvelich)
Move the Katib manifests upstream (https://github.com/kubeflow/katib/pull/1432 by @yanniszark)
Build the Trial images in the Katib CI (https://github.com/kubeflow/katib/pull/1457 by @andreyvelich)
Add script to update the boilerplates (https://github.com/kubeflow/katib/pull/1491 by @andreyvelich)

Change Log

Check the Full Change Log.

- Python
Published by andreyvelich about 5 years ago

kubeflow-katib - v0.10.1

Check the Full Change Log.

- Python
Published by andreyvelich about 5 years ago

kubeflow-katib - v0.10.0

This is the Katib 0.10 release for the Kubeflow 1.2. The new Katib v1beta1 API version has been released.

New Features

Core Features

The new Trial template design (https://github.com/kubeflow/katib/issues/1208)
Support custom Kubernetes CRD in the Trial template (https://github.com/kubeflow/katib/issues/1214)
- Add example for the Tekton Pipeline (https://github.com/kubeflow/katib/pull/1339)
- Add example for the Kubeflow MPIJob (https://github.com/kubeflow/katib/pull/1342)
Support early stopping with the Median Stopping Rule (https://github.com/kubeflow/katib/pull/1344)
Resume Experiment from the volume (https://github.com/kubeflow/katib/pull/1275)
- Support volume settings in the Katib config (https://github.com/kubeflow/katib/pull/1291)
Extract the Experiment metrics in multiple ways (https://github.com/kubeflow/katib/pull/1140)
Update the Python SDK for the v1beta1 version (https://github.com/kubeflow/katib/pull/1252)

UI Features and Enhancements

Show the Trial parameters on the submit Experiment page (https://github.com/kubeflow/katib/pull/1224)
Enable to set the Trial template YAML from the submit Experiment page (https://github.com/kubeflow/katib/pull/1363)
Optimise the Katib UI image (https://github.com/kubeflow/katib/pull/1232)
Enable sorting in the Trial list table (https://github.com/kubeflow/katib/pull/1251)
Add pages to the Trial list table (https://github.com/kubeflow/katib/pull/1262)
Use the V4 version for the Material UI (https://github.com/kubeflow/katib/pull/1254)
Automatically delete an empty ConfigMap with Trial templates (https://github.com/kubeflow/katib/pull/1260)
Create a ConfigMap with Trial templates (https://github.com/kubeflow/katib/pull/1265)
Support metrics strategies on the submit Experiment page (https://github.com/kubeflow/katib/pull/1364)
Add the resume policy to the submit Experiment page (https://github.com/kubeflow/katib/pull/1362)
Enable to create an early stopping Experiment from the submit Experiment page (https://github.com/kubeflow/katib/pull/1373)

Bug fixes

Check the Trials count before deleting it (https://github.com/kubeflow/katib/pull/1223)
Check that Trials are deleted (https://github.com/kubeflow/katib/pull/1288)
Fix the out of range error in the Hyperopt suggestion (https://github.com/kubeflow/katib/pull/1315)
Fix the pod ownership to inject the metrics collector (https://github.com/kubeflow/katib/pull/1303)

Misc

Switch the test infra to the AWS (https://github.com/kubeflow/katib/pull/1356)
Use the docker.io/kubeflowkatib registry to release images (https://github.com/kubeflow/katib/pull/1372)