What's Changed

docs: Update diagram image by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5431
chore(deps): bump actions/checkout from 4 to 5 by @dependabot[bot] in https://github.com/bentoml/BentoML/pull/5432
fix: Correct argument merging logic in set_arguments function by @frostming in https://github.com/bentoml/BentoML/pull/5433
feat: Add new GPU type 'amd-mi300x' to GpuLiteralType by @frostming in https://github.com/bentoml/BentoML/pull/5435
feat: allow to get command later for service by @frostming in https://github.com/bentoml/BentoML/pull/5437
fix: Add error handling for failed file downloads in JSONSerde and MultipartSerde by @frostming in https://github.com/bentoml/BentoML/pull/5436
docs: Add examples by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5439
fix: respect the metrics namespace passed from config by @frostming in https://github.com/bentoml/BentoML/pull/5440

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.21...v1.4.22

- Python
Published by frostming 9 months ago

What's Changed

fix: Move wheels directory if src/wheels exists by @frostming in https://github.com/bentoml/BentoML/pull/5430

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.20...v1.4.21

- Python
Published by frostming 10 months ago

What's Changed

docs: Add rollback section by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5423
feat: Replace usages of fs library with fsspec and pathlib by @frostming in https://github.com/bentoml/BentoML/pull/5424
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci[bot] in https://github.com/bentoml/BentoML/pull/5425
fix: Add env var to disable local bentoml URL in requirements by @frostming in https://github.com/bentoml/BentoML/pull/5427
chore(deps): bump actions/download-artifact from 4 to 5 by @dependabot[bot] in https://github.com/bentoml/BentoML/pull/5428

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.19...v1.4.20

- Python
Published by frostming 10 months ago

What's Changed

fix: run post setup commands for codespace by @frostming in https://github.com/bentoml/BentoML/pull/5414
fix: Ignore python.packages when requirements_txt is provided by @qimcis in https://github.com/bentoml/BentoML/pull/5421
chore(deps): bump the pip group across 2 directories with 1 update by @dependabot[bot] in https://github.com/bentoml/BentoML/pull/5413

New Contributors

@qimcis made their first contribution in https://github.com/bentoml/BentoML/pull/5421

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.18...v1.4.19

- Python
Published by frostming 10 months ago

What's Changed

docs: Update contact us link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5398
docs: Update canary deployment by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5397
docs: Add built-in status endpoints by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5400
build: Use python3.10 package that exists on Ubuntu22.04 by @byronclark in https://github.com/bentoml/BentoML/pull/5403
feat: better way to retrieve api token from CLI by @frostming in https://github.com/bentoml/BentoML/pull/5404
docs: Add sandbox observability images by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5401
fix: honor the default buckets in request duration metrics config by @frostming in https://github.com/bentoml/BentoML/pull/5405
doc: sandboxes by @ssheng in https://github.com/bentoml/BentoML/pull/5411
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci[bot] in https://github.com/bentoml/BentoML/pull/5410
docs: Move sandboxes doc to BentoCloud by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5412
feat: add lastrequesttimestamp metric by @xianml in https://github.com/bentoml/BentoML/pull/5416
docs: Update runtime doc for private pypi by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5415
fix: missing secrets and canarys param in cli by @xianml in https://github.com/bentoml/BentoML/pull/5417
fix: update assertions in IOMixin to enforce BaseModel subclassing by @frostming in https://github.com/bentoml/BentoML/pull/5419
fix: description/readme too large by @xianml in https://github.com/bentoml/BentoML/pull/5420

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.17...v1.4.18

- Python
Published by frostming 10 months ago

What's Changed

chore(config): export accelerator literal type by @aarnphm in https://github.com/bentoml/BentoML/pull/5384
fix: accept bento type as the bento argument for deployment APIs by @frostming in https://github.com/bentoml/BentoML/pull/5385
chore: return early python_packages by @aarnphm in https://github.com/bentoml/BentoML/pull/5387
docs: Update adaptive batching example by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5388
feat: reading bento args from a YAML file by @frostming in https://github.com/bentoml/BentoML/pull/5389
docs: Add --arg-file flag by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5391
fix: Adjust keras version comparison by @rmarquis in https://github.com/bentoml/BentoML/pull/5392
chore: update AWS cloudformation template by @sauyon in https://github.com/bentoml/BentoML/pull/5394
docs: Add canary deployment by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5390
docs: Add BentoML Sandboxes by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5396
feat: support canary for bentoml deployment apply commands by @xianml in https://github.com/bentoml/BentoML/pull/5395

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.16...v1.4.17

- Python
Published by frostming 11 months ago

What's Changed

docs: Add codespaces video by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5370
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5378
feat: read dependencies from uv lock by @frostming in https://github.com/bentoml/BentoML/pull/5379
refactor: move legacy APIs to a separate module by @frostming in https://github.com/bentoml/BentoML/pull/5381
feat: support custom service start command by @frostming in https://github.com/bentoml/BentoML/pull/5382
fix: better way to set service name by @frostming in https://github.com/bentoml/BentoML/pull/5383

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.15...v1.4.16

- Python
Published by frostming 12 months ago

What's Changed

docs: Add CI/CD pipeline doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5366
fix: disable GPU allocation if CUDAVISIBLEDEVICES is set by @frostming in https://github.com/bentoml/BentoML/pull/5368
fix: race condition issue initializing result db by @frostming in https://github.com/bentoml/BentoML/pull/5369

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.14...v1.4.15

- Python
Published by frostming about 1 year ago

What's Changed

chore: fix link in readme by @imba-tjd in https://github.com/bentoml/BentoML/pull/5353
docs: correct description of maxlatencyms by @jianshen92 in https://github.com/bentoml/BentoML/pull/5348
feat: support --arg for bentoml deployment create and update by @frostming in https://github.com/bentoml/BentoML/pull/5354
docs: reword some docs and a handful of typos by @b8zhong in https://github.com/bentoml/BentoML/pull/5356
fix: align the task status with bentocloud by @frostming in https://github.com/bentoml/BentoML/pull/5359
fix: compatible implementation for evaltypelenient by @frostming in https://github.com/bentoml/BentoML/pull/5358
docs: Add triton example by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5357
docs: stabilization window by @jianshen92 in https://github.com/bentoml/BentoML/pull/5360
docs: replace outdated deployment parameter by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5361
fix: apply stabilization window config by @jianshen92 in https://github.com/bentoml/BentoML/pull/5362
fix: allow route name without leading slash by @jianshen92 in https://github.com/bentoml/BentoML/pull/5363
fix(deployment): remove duplicated --config-file by @b8zhong in https://github.com/bentoml/BentoML/pull/5364

New Contributors

@imba-tjd made their first contribution in https://github.com/bentoml/BentoML/pull/5353
@b8zhong made their first contribution in https://github.com/bentoml/BentoML/pull/5356

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.13...v1.4.14

- Python
Published by frostming about 1 year ago

What's Changed

docs: Fix BentoCloud CLI link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5340
chore(deps): bump astral-sh/setup-uv from 5 to 6 by @dependabot in https://github.com/bentoml/BentoML/pull/5341
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5342
fix: subclass checking for client classes by @frostming in https://github.com/bentoml/BentoML/pull/5345
fix: insert install command even if the base image is specified by @frostming in https://github.com/bentoml/BentoML/pull/5344
fix: do not allow separate python requirements calls by @frostming in https://github.com/bentoml/BentoML/pull/5347
fix: support pre-built wheels in bento spec v2 by @frostming in https://github.com/bentoml/BentoML/pull/5349
feat: add paths to schemas to support openai chat endpoints by @aarnphm in https://github.com/bentoml/BentoML/pull/5346
fix: delete preheatpippackages by @bojiang in https://github.com/bentoml/BentoML/pull/5350
docs: Add docs for wheels and run_script() by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5351
chore: minor fix, wrong message by @frostming in https://github.com/bentoml/BentoML/pull/5352

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.12...v1.4.13

- Python
Published by frostming about 1 year ago

What's Changed

docs: Use command to list GPU types by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5330
chore: disable progress bar on cloud build by @aarnphm in https://github.com/bentoml/BentoML/pull/5331
fix: handling equal sign in value of secret by @larme in https://github.com/bentoml/BentoML/pull/5332
fix: make sure to inherit from base image ENV by @aarnphm in https://github.com/bentoml/BentoML/pull/5334
fix: don't add duplicate global options to requirements.txt by @frostming in https://github.com/bentoml/BentoML/pull/5336
fix: don't restore cwd when importing service in worker by @frostming in https://github.com/bentoml/BentoML/pull/5339
docs: Unify image styles with BentoML theme by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5338
feat: custom readiness hook method by @frostming in https://github.com/bentoml/BentoML/pull/5337

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.10...v1.4.12

- Python
Published by aarnphm about 1 year ago

What's Changed

doc: Update saml.rst by @bojiang in https://github.com/bentoml/BentoML/pull/5309
docs: Add template arg doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5311
chore: update gpu_type for supported GPU on bentocloud by @aarnphm in https://github.com/bentoml/BentoML/pull/5312
doc: rename PythonImage to Image by @frostming in https://github.com/bentoml/BentoML/pull/5313
docs: add monitoring API tokens section to manage-api-tokens.rst by @Haivilo in https://github.com/bentoml/BentoML/pull/5316
feat: show fastapi routes in playground ui by @frostming in https://github.com/bentoml/BentoML/pull/5314
fix: add /app/.venv/bin to PATH to support Kubernetes exec-style entrypoints by @restato in https://github.com/bentoml/BentoML/pull/5315
fix: Merge command line args with the ones in the config file by @frostming in https://github.com/bentoml/BentoML/pull/5317
doc: add examples of bentoml deploy for bento args by @frostming in https://github.com/bentoml/BentoML/pull/5318
fix: usage tracking for bentoml build by @frostming in https://github.com/bentoml/BentoML/pull/5321
revert: "feat: show fastapi routes in playground ui" by @frostming in https://github.com/bentoml/BentoML/pull/5323
fix: only regard the positional arg as build ctx if it is a local dir by @frostming in https://github.com/bentoml/BentoML/pull/5322
fix: make sure to use self.args in with_defaults by @aarnphm in https://github.com/bentoml/BentoML/pull/5324

New Contributors

@restato made their first contribution in https://github.com/bentoml/BentoML/pull/5315

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.8...v1.4.10

- Python
Published by frostming about 1 year ago

What's Changed

feat: make it possible to override values of Image from bentofile.yaml by @frostming in https://github.com/bentoml/BentoML/pull/5298
feat: implement bento arguments by @frostming in https://github.com/bentoml/BentoML/pull/5299
fix: add end of line if the requirments file doesn't have by @frostming in https://github.com/bentoml/BentoML/pull/5302
docs(security): update security advisory email by @ssheng in https://github.com/bentoml/BentoML/pull/5304
fix: remove start commands from CLI by @frostming in https://github.com/bentoml/BentoML/pull/5303
chore: update security notes by @frostming in https://github.com/bentoml/BentoML/pull/5305
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5306
chore: locking transformers <4.51 for quickstart tests by @aarnphm in https://github.com/bentoml/BentoML/pull/5307

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.7...v1.4.8

- Python
Published by frostming about 1 year ago

What's Changed

docs(saml): add sso saml doc by @ssheng in https://github.com/bentoml/BentoML/pull/5292
docs: Add some diagrams for get started section by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5285
fix: do not start dependency if it's from a URL or deployment by @frostming in https://github.com/bentoml/BentoML/pull/5293
feat: support packgitpackages and fix add group for alpine by @frostming in https://github.com/bentoml/BentoML/pull/5295
fix: initialize async http client with async method by @frostming in https://github.com/bentoml/BentoML/pull/5296
feat: support --name option for bentoml build and bentoml code by @frostming in https://github.com/bentoml/BentoML/pull/5297

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.6...v1.4.7

- Python
Published by aarnphm about 1 year ago

What's Changed

docs: Add org token doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5277
docs: remove openllm from examples overview by @ssheng in https://github.com/bentoml/BentoML/pull/5278
fix: format substitution by @byronclark in https://github.com/bentoml/BentoML/pull/5279
fix: add missing .route of __endpoint by @reobroqn in https://github.com/bentoml/BentoML/pull/5281
fix: non-standard ssh git url by @frostming in https://github.com/bentoml/BentoML/pull/5282
docs: Update adaptive batching doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5283
docs: Add BentoOCR to example list by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5284
docs: Update private repo notes by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5262
fix: replace namespace import to python_multipart by @yxtay in https://github.com/bentoml/BentoML/pull/5280
docs: Fix format by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5287
chore: better visibility by @bojiang in https://github.com/bentoml/BentoML/pull/5288
fix: on load bento for dependent services by @frostming in https://github.com/bentoml/BentoML/pull/5286
fix: visibility by @bojiang in https://github.com/bentoml/BentoML/pull/5289
fix: wrong error context by @frostming in https://github.com/bentoml/BentoML/pull/5291

New Contributors

@byronclark made their first contribution in https://github.com/bentoml/BentoML/pull/5279
@reobroqn made their first contribution in https://github.com/bentoml/BentoML/pull/5281

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.5...v1.4.6

- Python
Published by frostming about 1 year ago

What's Changed

fix: loading service issue in worker by @frostming in https://github.com/bentoml/BentoML/pull/5276

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.4...v1.4.5

- Python
Published by frostming about 1 year ago

What's Changed

feat: update readme by @ssheng in https://github.com/bentoml/BentoML/pull/5259
docs: Add update command to cloud deployment doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5260
fix: fallback to v1 if pack git packages is set by @frostming in https://github.com/bentoml/BentoML/pull/5261
build: Smaller docker images by @jeffmarshall in https://github.com/bentoml/BentoML/pull/5264
feat(serve): add --env and inject service-defined envs to serving environment by @aarnphm in https://github.com/bentoml/BentoML/pull/5263
docs(tasks): update async tasks docs by @ssheng in https://github.com/bentoml/BentoML/pull/5266
feat(client): add task apis to client methods by @frostming in https://github.com/bentoml/BentoML/pull/5265
feat: default min replicas is None by @yetone in https://github.com/bentoml/BentoML/pull/5268
tests: banning pickle requests by @frostming in https://github.com/bentoml/BentoML/pull/5269
feat: Fail fast when running inside container by @frostming in https://github.com/bentoml/BentoML/pull/5270
docs(byoc): update aws byoc role template link by @ssheng in https://github.com/bentoml/BentoML/pull/5272
docs: Update langgraph example by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5273
fix(supervisor): quit the arbiter correctly by @frostming in https://github.com/bentoml/BentoML/pull/5274
fix: preserve global options in requirements.txt by @frostming in https://github.com/bentoml/BentoML/pull/5275

New Contributors

@jeffmarshall made their first contribution in https://github.com/bentoml/BentoML/pull/5264

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.3...v1.4.4

- Python
Published by frostming about 1 year ago

What's Changed

docs: Update examples to use new APIs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5252
Add alt text to all images in documentation by @devin-ai-integration in https://github.com/bentoml/BentoML/pull/5253
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5254
docs: update deprecated links by @aarnphm in https://github.com/bentoml/BentoML/pull/5256
docs: Add root input usage by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5257
fix: reformat the code by @frostming in https://github.com/bentoml/BentoML/pull/5258
fix: forbid requests with pickle encoding at the entry service

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.2...v1.4.3

- Python
Published by frostming about 1 year ago

What's Changed

Fix RunPod unhashable model error by converting svc.models to hashable ids by @rivaon in https://github.com/bentoml/BentoML/pull/5244
fix(io): OpenAPI schema for multipart form request body by @frostming in https://github.com/bentoml/BentoML/pull/5249
fix: drop uv as a hard dependency by @frostming in https://github.com/bentoml/BentoML/pull/5238
docs: Update examples to use new HF API by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5242
refactor: drop deepmerge dependency by @frostming in https://github.com/bentoml/BentoML/pull/5250
refactor: unify logic of loading service by @frostming in https://github.com/bentoml/BentoML/pull/5232
fix: collect requirements from image spec for codespaces by @frostming in https://github.com/bentoml/BentoML/pull/5251

New Contributors

@rivaon made their first contribution in https://github.com/bentoml/BentoML/pull/5244

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.1...v1.4.2

- Python
Published by frostming over 1 year ago

What's Changed

docs: use diff for runtime image by @parano in https://github.com/bentoml/BentoML/pull/5236
docs: Update runtime explanations by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5240
fix(regression): call error when context parameter is present by @frostming in https://github.com/bentoml/BentoML/pull/5247

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.0...v1.4.1

- Python
Published by frostming over 1 year ago

bentoml - v1.4.0

We are thrilled to announce the release of BentoML 1.4! This version introduces several new features and improvements to accelerate your iteration cycle and enhance the overall developer experience.

Below are the key highlights of 1.4, and you can find more details in the release blog post.

🚀 20x faster iteration with Codespaces

Introduced BentoML Codespaces, a development platform built on BentoCloud
Added the bentoml code command for creating a Codespace
Auto-sync of local changes to the cloud environment
Access to a variety of powerful cloud GPUs
Real-time logs and debugging through the cloud dashboard
Eliminate dependency headaches and ensure consistency between dev and prod environments

🐍 New Python SDK for runtime configurations

Added bentoml.images.PythonImage for defining the Bento runtime environment in Python instead of using bentofile.yaml or pyproject.toml
Support customizing runtime configurations (e.g., Python version, system packages, and dependencies) directly in the service.py file
Introduced context-sensitive run() method for running custom build commands
Backward compatible with existing bentofile.yaml and pyproject.toml configurations

⚡ Accelerated model loading with safetensors

Implemented build-time model downloads and parallel loading of model weights using safetensors to reduce cold start time and improve scaling performance. See the documentation to learn more.
Added bentoml.models.HuggingFaceModel for loading models from HF. It supports private model repositories and custom endpoints
Added bentoml.models.BentoModel for loading models from BentoCloud and the Model Store

🌍 External deployment dependencies

Extended bentoml.depends() to support external deployments
Added support for calling BentoCloud Deployments via name or URL
Added support for calling self-hosted HTTP AI services outside BentoCloud

⚠️ Legacy Service API deprecation

The legacy bentoml.Service API (with runners) is now officially deprecated and is scheduled for removal in a future release. We recommend you use the @bentoml.service decorator.

Note that:

1.4 remains fully compatible with Bentos created by 1.3.
The BentoML documentation has been updated with examples and guides for 1.4.

🙏 As always, we appreciate your continued support!

What's Changed

feat: support bentoml serve without service name by @frostming in https://github.com/bentoml/BentoML/pull/5208
feat(service): expose service-level labels definition by @aarnphm in https://github.com/bentoml/BentoML/pull/5211
fix: restore path after import by @frostming in https://github.com/bentoml/BentoML/pull/5214
fix: compile bytecode when installing python packages by @frostming in https://github.com/bentoml/BentoML/pull/5212
fix: IO descriptor honor validators by @frostming in https://github.com/bentoml/BentoML/pull/5213
feat(image): add support for chaining .pyproject.toml by @aarnphm in https://github.com/bentoml/BentoML/pull/5218
feat: support root input spec using positonal-only argument by @frostming in https://github.com/bentoml/BentoML/pull/5217
fix: gradio error when uploading file by @frostming in https://github.com/bentoml/BentoML/pull/5220
fix: input data validation for root input by @frostming in https://github.com/bentoml/BentoML/pull/5221
fix: don't restore model store after importing service by @frostming in https://github.com/bentoml/BentoML/pull/5223
feat(metrics): extend histogram buckets to support LLM latencies by @devin-ai-integration in https://github.com/bentoml/BentoML/pull/5222
fix: always add bentoml req unless it is specified as a url dependency by @frostming in https://github.com/bentoml/BentoML/pull/5225
docs: update links to examples by @aarnphm in https://github.com/bentoml/BentoML/pull/5224
docs: add environment variable authentication documentation by @devin-ai-integration in https://github.com/bentoml/BentoML/pull/5231
docs: Update docs to use new runtime API by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5177
fix: add files under env/docker by @frostming in https://github.com/bentoml/BentoML/pull/5234

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.22...v1.4.0

- Python
Published by frostming over 1 year ago

What's Changed

fix: restore path after import by @frostming in https://github.com/bentoml/BentoML/pull/5214

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.4.0a1...v1.4.0a2

- Python
Published by frostming over 1 year ago

What's Changed

feat: support bentoml serve without service name by @frostming in https://github.com/bentoml/BentoML/pull/5208
feat(service): expose service-level labels definition by @aarnphm in https://github.com/bentoml/BentoML/pull/5211

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.22...v1.4.0a1

- Python
Published by frostming over 1 year ago

What's Changed

fix: add line endings for python_packages(...) by @frostming in https://github.com/bentoml/BentoML/pull/5203
fix: correct the log for locking platform by @frostming in https://github.com/bentoml/BentoML/pull/5205
fix: uv installation in container by @frostming in https://github.com/bentoml/BentoML/pull/5206
fix: add cluster option for secrets command by @xianml in https://github.com/bentoml/BentoML/pull/5209
fix(build): setattr frozen sets by @aarnphm in https://github.com/bentoml/BentoML/pull/5207
fix:correct context path when serving from bento by @frostming in https://github.com/bentoml/BentoML/pull/5210

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.21...v1.3.22

- Python
Published by frostming over 1 year ago

What's Changed

docs: correct types for example by @NitzanShwartz in https://github.com/bentoml/BentoML/pull/5188
fix: make all top-level attributes lazy by @frostming in https://github.com/bentoml/BentoML/pull/5184
fix: mounted apps don't eagerly match path prefix by @frostming in https://github.com/bentoml/BentoML/pull/5190
chore: add spec into bento manifest schema by @xianml in https://github.com/bentoml/BentoML/pull/5192
fix: get commit id from requested revision when it is missing by @frostming in https://github.com/bentoml/BentoML/pull/5193
feat: add @bentoml.on_startup decorator by @devin-ai-integration in https://github.com/bentoml/BentoML/pull/5194
fix: conditionally include endpoint_urls in deployment dictionary by @Haivilo in https://github.com/bentoml/BentoML/pull/5195
fix: enable image spec by default by @frostming in https://github.com/bentoml/BentoML/pull/5197
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5200
docs: update AWS byoc cloudformation template by @sauyon in https://github.com/bentoml/BentoML/pull/5201
feat: support running scripts in new image spec by @frostming in https://github.com/bentoml/BentoML/pull/5196

New Contributors

@NitzanShwartz made their first contribution in https://github.com/bentoml/BentoML/pull/5188

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.20...v1.3.21

- Python
Published by frostming over 1 year ago

What's Changed

Add logging for service cleanup hooks in new-style BentoML services by @devin-ai-integration in https://github.com/bentoml/BentoML/pull/5171
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5170
fix: set the arbiter thread to be daemon thread by @frostming in https://github.com/bentoml/BentoML/pull/5172
chore(analytics): use utils is_jupyter to determine notebook state by @aarnphm in https://github.com/bentoml/BentoML/pull/5173
docs: Add comfyUI example doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5148
fix: allow building bento without build config file by @frostming in https://github.com/bentoml/BentoML/pull/5175
fix: Remove distutils dependency by @jaume-ferrarons in https://github.com/bentoml/BentoML/pull/5179
docs: Add streaming and websocket docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5180
docs: Update ASGI decorator by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5182
fix: allow usage like bentoml deploy service:MyService by @frostming in https://github.com/bentoml/BentoML/pull/5181
docs: Update Gradio example link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5183
fix: use a specific manylinux version as build platform for better compatibility by @frostming in https://github.com/bentoml/BentoML/pull/5185
feat(metrics): init zero for request count of apis by @Haivilo in https://github.com/bentoml/BentoML/pull/5186

New Contributors

@devin-ai-integration made their first contribution in https://github.com/bentoml/BentoML/pull/5171
@jaume-ferrarons made their first contribution in https://github.com/bentoml/BentoML/pull/5179

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.19...v1.3.20

- Python
Published by frostming over 1 year ago

What's Changed

fix: turn off build log stream in jupyter notebook by @frostming in https://github.com/bentoml/BentoML/pull/5165
docs: Add notes for loading HF private models by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5166
fix: patch annotations inside a container type by @frostming in https://github.com/bentoml/BentoML/pull/5167

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.18...v1.3.19

- Python
Published by frostming over 1 year ago

What's Changed

docs: Simplify model composition doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5147
docs: Restructure image folder and add alt for SEO by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5150
docs: Fix links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5153
docs: Merge examples to io type doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5152
fix: Permission issue of uv python installation directory by @frostming in https://github.com/bentoml/BentoML/pull/5151
fix: lazy safetensor patch by @frostming in https://github.com/bentoml/BentoML/pull/5149
docs: Clean up examples folder by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5154
fix: use kantoku instead of circus by @bojiang in https://github.com/bentoml/BentoML/pull/5155
fix: Update lock file by @frostming in https://github.com/bentoml/BentoML/pull/5156
fix: use emoji code instead of emoji char by @frostming in https://github.com/bentoml/BentoML/pull/5157
fix: set model revision correctly by @frostming in https://github.com/bentoml/BentoML/pull/5158
fix: add missing secrets params in bentoml sdk by @xianml in https://github.com/bentoml/BentoML/pull/5159
fix: expose cloud api to get a single model by @frostming in https://github.com/bentoml/BentoML/pull/5160
fix: use the same spinner for bento and model apis by @frostming in https://github.com/bentoml/BentoML/pull/5161
fix: closed file descriptior issue by @frostming in https://github.com/bentoml/BentoML/pull/5162

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.17...v1.3.18

- Python
Published by frostming over 1 year ago

What's Changed

docs: Simplify packaging docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5116
docs: Fix links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5124
docs: Update API tokens doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5125
fix: image builder log stream doesn't work by @frostming in https://github.com/bentoml/BentoML/pull/5126
docs: Add GPU access messaging by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5127
fix: disable markup and highlight for image log stream by @frostming in https://github.com/bentoml/BentoML/pull/5129
fix: don't override built-in function by @frostming in https://github.com/bentoml/BentoML/pull/5131
fix: add default install command for image spec by @frostming in https://github.com/bentoml/BentoML/pull/5130
docs: Update examples by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5128
fix: loosen the constraints of opentelemetry dependencies by @frostming in https://github.com/bentoml/BentoML/pull/5134
fix: only honor ctx or context parameter with correct annotation by @frostming in https://github.com/bentoml/BentoML/pull/5135
fix: temp file is deleted unexpected on Windows by @frostming in https://github.com/bentoml/BentoML/pull/5133
docs: Update old comfyui doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5137
feat: install packages into venv and chown by @frostming in https://github.com/bentoml/BentoML/pull/5136
docs: Update onboarding docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5138
fix: raise error when certfile is not given but ssl enabled and fix local dep by @frostming in https://github.com/bentoml/BentoML/pull/5139
fix: bad file descirptor issue when deploying bento by @frostming in https://github.com/bentoml/BentoML/pull/5142
feat: select a subset of files for huggingface models by @frostming in https://github.com/bentoml/BentoML/pull/5144
docs: Update example overview by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5143
docs: Fix images by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5145
fix: use content hash as the default model version by @frostming in https://github.com/bentoml/BentoML/pull/5146

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.16...v1.3.17

- Python
Published by frostming over 1 year ago

What's Changed

ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5108
docs: Simplify Adaptive batching docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5107
docs: Update homepage messaging by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5105
fix: add basic metrics and access logs for websocket by @frostming in https://github.com/bentoml/BentoML/pull/5109
fix: ensure all models to exists on deployment by @frostming in https://github.com/bentoml/BentoML/pull/5110
fix: enable image spec when image is explictly given by @frostming in https://github.com/bentoml/BentoML/pull/5114
fix: drop the redundant parameter for runner_service by @frostming in https://github.com/bentoml/BentoML/pull/5115
docs: Fix links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5111
feat: support post commands in image spec by @frostming in https://github.com/bentoml/BentoML/pull/5118
feat: add --with-models option to bentoml pull command by @frostming in https://github.com/bentoml/BentoML/pull/5117
docs: Update cloud deployment doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5119
docs: Fix typo by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5121
docs: Fix links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5122
fix: calculate bentofile path against cwd if given explicitly by @frostming in https://github.com/bentoml/BentoML/pull/5123

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.15...v1.3.16

- Python
Published by frostming over 1 year ago

What's Changed

docs: doc refactoring by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5089
docs: Fix invalid icon images by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5096
docs: Update readme examples and add distributed service doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5097
fix: protect model repository creation with a lock to avoid a race condition by @frostming in https://github.com/bentoml/BentoML/pull/5095
fix: generate schema compatible with pydantic>=2.10 by @frostming in https://github.com/bentoml/BentoML/pull/5100
docs: Modify 'metric' menu link on 'observability index' page by @jin-daram in https://github.com/bentoml/BentoML/pull/5101
fix: fix bug of serialization error and runner can't be shutdown by @frostming in https://github.com/bentoml/BentoML/pull/5103
feat(cli): Support passing key value pairs from reading a dotenv file by @frostming in https://github.com/bentoml/BentoML/pull/5104
fix: calculate readme file path from build ctx rather than cwd by @frostming in https://github.com/bentoml/BentoML/pull/5106

New Contributors

@jin-daram made their first contribution in https://github.com/bentoml/BentoML/pull/5101

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.14...v1.3.15

- Python
Published by frostming over 1 year ago

What's Changed

docs: update buf.yaml reference by @emmanuel-ferdman in https://github.com/bentoml/BentoML/pull/5090
fix: read stream before accessing response.text by @frostming in https://github.com/bentoml/BentoML/pull/5091
fix: set manifest.dev for bare bento upload by @frostming in https://github.com/bentoml/BentoML/pull/5092
fix: pin pydantic to < 2.10 and restore cwd when building bento by @frostming in https://github.com/bentoml/BentoML/pull/5093

New Contributors

@emmanuel-ferdman made their first contribution in https://github.com/bentoml/BentoML/pull/5090

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.13...v1.3.14

- Python
Published by bojiang over 1 year ago

What's Changed

fix: build git repo with uv by @frostming in https://github.com/bentoml/BentoML/pull/5085
fix: use the model_id in bento by @frostming in https://github.com/bentoml/BentoML/pull/5087
fix: create dir before move by @frostming in https://github.com/bentoml/BentoML/pull/5088

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.12...v1.3.13

- Python
Published by frostming over 1 year ago

What's Changed

fix: correct file response for internal service calling by @frostming in https://github.com/bentoml/BentoML/pull/5071
docs: Add explanations for model loading acceleration by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5066
docs: Refactor examples by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5065
docs: Update codespaces desc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5072
fix: don't ignore empty dirs when unpacking model and bento by @frostming in https://github.com/bentoml/BentoML/pull/5073
docs: Remove dead link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5075
docs: Add doc canonical link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5076
doc: add comfyui example by @ssheng in https://github.com/bentoml/BentoML/pull/5077
doc: update comfyui to include export instructions by @ssheng in https://github.com/bentoml/BentoML/pull/5078
doc: update diffusion models to include comfyui by @ssheng in https://github.com/bentoml/BentoML/pull/5079
docs: Update examples index by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5080
fix: move monitoring project to tests folder by @frostming in https://github.com/bentoml/BentoML/pull/5081
docs: Update example list by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5083
fix: check dependency close method (#5082) by @yxtay in https://github.com/bentoml/BentoML/pull/5084
fix: empty doc for bare built bento by @xianml in https://github.com/bentoml/BentoML/pull/5086

New Contributors

@yxtay made their first contribution in https://github.com/bentoml/BentoML/pull/5084

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.11...v1.3.12

- Python
Published by frostming over 1 year ago

What's Changed

fix: spark usage following API change by @frostming in https://github.com/bentoml/BentoML/pull/5052
docs: Add model loading doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5049
doc: suggest to use pip for CUDA first by @bojiang in https://github.com/bentoml/BentoML/pull/5054
docs: Update build-options.rst by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5055
fix: use the model revision in bento info for huggingface models by @frostming in https://github.com/bentoml/BentoML/pull/5053
feat: Adding SDK support for deployment labels by @jianshen92 in https://github.com/bentoml/BentoML/pull/5057
fix: postpone the model id normalization for huggingface models by @frostming in https://github.com/bentoml/BentoML/pull/5060
chore: upload descripiton to remote bento store by @xianml in https://github.com/bentoml/BentoML/pull/5047
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5061
feat: Add task_id to request inference context by @Leon-Sander in https://github.com/bentoml/BentoML/pull/5059
chore: copy existing README.md instead of creating a new one by @xianml in https://github.com/bentoml/BentoML/pull/5063
feat: allow extending bentoml with extra commands by @frostming in https://github.com/bentoml/BentoML/pull/5064
fix: remove the version limit of pynvml by @frostming in https://github.com/bentoml/BentoML/pull/5068

New Contributors

@Leon-Sander made their first contribution in https://github.com/bentoml/BentoML/pull/5059

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.10...v1.3.11

- Python
Published by frostming over 1 year ago

What's Changed

fix: [bug bash] prefetch url by @xianml in https://github.com/bentoml/BentoML/pull/5028
fix: trigger reset when system package changes; find dependent by @frostming in https://github.com/bentoml/BentoML/pull/5027
fix(decorator): reject wrapper as dunder methods by @aarnphm in https://github.com/bentoml/BentoML/pull/5030
fix: upload a flag file after project ready by @frostming in https://github.com/bentoml/BentoML/pull/5029
fix: update spinner with the service readyz status by @frostming in https://github.com/bentoml/BentoML/pull/5031
docs: Add IO tables by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5032
docs: Create secrets for deployment by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5034
doc: api docs for build and build_bentofile by @frostming in https://github.com/bentoml/BentoML/pull/5035
feat: validate required envs when creating deployment by @frostming in https://github.com/bentoml/BentoML/pull/5033
docs: fix links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5036
docs: Add Gradio integration by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5040
fix: remove duplicate logs by @frostming in https://github.com/bentoml/BentoML/pull/5038
docs: Add byoc request quota by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5037
feat: Opt-in bento image support by @frostming in https://github.com/bentoml/BentoML/pull/5039
feat: support reading build config and dependencies from pyproject.toml by @frostming in https://github.com/bentoml/BentoML/pull/5042
docs: Add doc for external deployment dependency by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5043
docs: Add Base Hyperlint Config by @bllchmbrs in https://github.com/bentoml/BentoML/pull/5041
fix: reconstruct build config for getting requirement by @frostming in https://github.com/bentoml/BentoML/pull/5045
docs: Add LangGraph example doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5046
docs: Add codespaces doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5044

New Contributors

@bllchmbrs made their first contribution in https://github.com/bentoml/BentoML/pull/5041

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.9...v1.3.10

- Python
Published by frostming over 1 year ago

What's Changed

fix: add a flag to mark dev bento by @frostming in https://github.com/bentoml/BentoML/pull/5021
docs: Fix typo by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5022
fix: circus graceful shutdown by @bojiang in https://github.com/bentoml/BentoML/pull/5024
fix: emit deprecation warings for legacy APIs by @frostming in https://github.com/bentoml/BentoML/pull/5023
fix: skip safetensor patching if not installed by @frostming in https://github.com/bentoml/BentoML/pull/5025
doc: update aws byoc by @ssheng in https://github.com/bentoml/BentoML/pull/5026

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.8...v1.3.9

- Python
Published by frostming over 1 year ago

What's Changed

doc: Update shieldgemma doc name by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4999
fix: tolerate build error when watching for changes by @frostming in https://github.com/bentoml/BentoML/pull/5000
doc: correct the requirements by @frostming in https://github.com/bentoml/BentoML/pull/5001
fix: expose errors occurring before the first yield of stream response by @frostming in https://github.com/bentoml/BentoML/pull/5002
doc: fix endpoint url related typos by @ssheng in https://github.com/bentoml/BentoML/pull/5007
docs: Add CUDA version notes by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5004
docs: Add new project examples by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5010
docs: Clean up examples by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5011
docs: Update adaptive batching doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5012
docs: Refactor more examples section by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5014
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/5009
feat: upload setup.sh to the remote dev by @frostming in https://github.com/bentoml/BentoML/pull/5016
feat: always make an HTTP call when calling self API methods by @frostming in https://github.com/bentoml/BentoML/pull/5017
feat: gradio integration by @parano in https://github.com/bentoml/BentoML/pull/5008
Fix a typo by @snowsky in https://github.com/bentoml/BentoML/pull/5018
docs: Add quotation marks to python versions by @Sherlock113 in https://github.com/bentoml/BentoML/pull/5019

New Contributors

@snowsky made their first contribution in https://github.com/bentoml/BentoML/pull/5018

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.7...v1.3.8

- Python
Published by frostming over 1 year ago

What's Changed

fix: do not initialize cloud client when building bento by @frostming in https://github.com/bentoml/BentoML/pull/4998

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.6...v1.3.7

- Python
Published by frostming over 1 year ago

What's Changed

doc: include cli commands in secret doc by @ssheng in https://github.com/bentoml/BentoML/pull/4968
fix: switch back to post-release version scheme by @frostming in https://github.com/bentoml/BentoML/pull/4969
doc: Add documentation about platform option of bentoml build by @frostming in https://github.com/bentoml/BentoML/pull/4966
fix: only add bentoml requirement if absent by @frostming in https://github.com/bentoml/BentoML/pull/4973
fix: null filename causes parsing failure for image input by @frostming in https://github.com/bentoml/BentoML/pull/4976
doc: add instruction for bentoml.IODescriptor by @frostming in https://github.com/bentoml/BentoML/pull/4975
chore: update login error message by @xianml in https://github.com/bentoml/BentoML/pull/4952
chore: remove grpc client by @aarnphm in https://github.com/bentoml/BentoML/pull/4983
fix: persist trace across internal requests by @frostming in https://github.com/bentoml/BentoML/pull/4982
feat: expose request id and trace id in request scope by @frostming in https://github.com/bentoml/BentoML/pull/4977
fix: add secrets in update/apply commands by @xianml in https://github.com/bentoml/BentoML/pull/4985
feat: development of release 1.4 by @frostming in https://github.com/bentoml/BentoML/pull/4984
doc: update quick start to use bentoml importing context manager by @frostming in https://github.com/bentoml/BentoML/pull/4986
fix: make spinner optional for backward compatibility by @frostming in https://github.com/bentoml/BentoML/pull/4987
fix: don't initialize cloud client until needed by @frostming in https://github.com/bentoml/BentoML/pull/4989
doc: fix typo in model composition by @ssheng in https://github.com/bentoml/BentoML/pull/4991
refactor: unify same schema by @frostming in https://github.com/bentoml/BentoML/pull/4988
fix(framework): export unsloth lazily by @aarnphm in https://github.com/bentoml/BentoML/pull/4992
fix: compare bento manifest and dependency client by @frostming in https://github.com/bentoml/BentoML/pull/4993
fix: structure bento manifest by @frostming in https://github.com/bentoml/BentoML/pull/4994
fix: allow setting timeout for cloud client; hf api; manifest eq by @frostming in https://github.com/bentoml/BentoML/pull/4995
fix: with_defaults for docker option and show diff report for bento manifest comparison by @frostming in https://github.com/bentoml/BentoML/pull/4996
docs: Add ShieldGemma example doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4997

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.5...v1.3.6

- Python
Published by frostming over 1 year ago

What's Changed

chore(ci): release workflow fix by @frostming in https://github.com/bentoml/BentoML/pull/4962
fix: ondeployment failed to run for servegrpc by @frostming in https://github.com/bentoml/BentoML/pull/4963

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.4...v1.3.5

- Python
Published by frostming over 1 year ago

What's Changed

chore(ci): release workflow fix by @frostming in https://github.com/bentoml/BentoML/pull/4962

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.4...v1.3.4post1

- Python
Published by frostming over 1 year ago

What's Changed

fix: don't use temporary directory for temp bento copying by @sauyon in https://github.com/bentoml/BentoML/pull/4938
refactor: path spec ignore pattern checks children files only by @frostming in https://github.com/bentoml/BentoML/pull/4939
fix: improve exception handling for pynvml by @rlleshi in https://github.com/bentoml/BentoML/pull/4943
doc: update cloud deploy with yaml guide by @ssheng in https://github.com/bentoml/BentoML/pull/4944
doc: fix instructions of http testing by @frostming in https://github.com/bentoml/BentoML/pull/4945
chore(deps): bump protobufjs from 7.2.4 to 7.2.6 in /grpc-client/node by @dependabot in https://github.com/bentoml/BentoML/pull/4647
chore(deps): bump transformers from 4.30.0 to 4.38.0 in /examples/inference-graph by @dependabot in https://github.com/bentoml/BentoML/pull/4695
chore(deps): bump h2 from 0.3.24 to 0.3.26 in /grpc-client/rust by @dependabot in https://github.com/bentoml/BentoML/pull/4635
chore(deps): bump tar from 6.1.11 to 6.2.1 in /grpc-client/node by @dependabot in https://github.com/bentoml/BentoML/pull/4650
chore(deps): bump mio from 0.8.8 to 0.8.11 in /grpc-client/rust by @dependabot in https://github.com/bentoml/BentoML/pull/4550
fix: bentoml.serve to support serving new SDK services from a bento by @frostming in https://github.com/bentoml/BentoML/pull/4946
docs: Let search engine ignore 1.1 doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4950
docs: Update readme example links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4951
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4953
fix: don't pull models if already exists when puling bento by @frostming in https://github.com/bentoml/BentoML/pull/4954
feat: lock packages for linux by default by @frostming in https://github.com/bentoml/BentoML/pull/4955
chore(deps): bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows in the github_actions group across 1 directory by @dependabot in https://github.com/bentoml/BentoML/pull/4957
docs: Add function calling example doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4956
docs: Add a placeholder RAG doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4959
fix: support pydantic 2.9 by @frostming in https://github.com/bentoml/BentoML/pull/4960
fix: retry for model and bento upload failure by @frostming in https://github.com/bentoml/BentoML/pull/4961
fix: reload plugin missing file changes by @frostming in https://github.com/bentoml/BentoML/pull/4958

New Contributors

@rlleshi made their first contribution in https://github.com/bentoml/BentoML/pull/4943

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.3...v1.3.4

- Python
Published by frostming over 1 year ago

What's Changed

fix: restore keras.save_model compatibility with Keras 3 by @rmarquis in https://github.com/bentoml/BentoML/pull/4922
fix(cli): do not hide output behind a spinner for building bento by @frostming in https://github.com/bentoml/BentoML/pull/4923
feat: expose client url and sync client by @frostming in https://github.com/bentoml/BentoML/pull/4925
fix: always preserve str and bytes when serializing by @frostming in https://github.com/bentoml/BentoML/pull/4929
fix(cloud-client): raise error if multipart push fails by @Haivilo in https://github.com/bentoml/BentoML/pull/4932
feat: Pass torch kwargs to bentoml.pytorch.load_model by @TokeReines in https://github.com/bentoml/BentoML/pull/4930
docs: Add function calling example link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4934
fix: pydantic/numpy patching causes Cannot interpret 'numpy.float16 | numpy.float32' as a data type by @frostming in https://github.com/bentoml/BentoML/pull/4933
fix: uploading large files saving to disk instead of memory by @frostming in https://github.com/bentoml/BentoML/pull/4935

New Contributors

@rmarquis made their first contribution in https://github.com/bentoml/BentoML/pull/4922
@TokeReines made their first contribution in https://github.com/bentoml/BentoML/pull/4930

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.2...v1.3.3

- Python
Published by frostming almost 2 years ago

What's Changed

fix: press text by @xianml in https://github.com/bentoml/BentoML/pull/4898
ci: pre-commit autoupdate by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4897
docs: Update doc images by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4901
fix: fix resource type by @FogDong in https://github.com/bentoml/BentoML/pull/4904
fix: direct return response if tracer id is null by @FogDong in https://github.com/bentoml/BentoML/pull/4899
docs: update azure byoc instructions by @ssheng in https://github.com/bentoml/BentoML/pull/4905
fix: metrics duration defined with min,max,factor not taking effect by @frostming in https://github.com/bentoml/BentoML/pull/4903
refactor: store the cloud context into the container instead of passing it around by @frostming in https://github.com/bentoml/BentoML/pull/4907
docs: Update README.md by @parano in https://github.com/bentoml/BentoML/pull/4906
docs: Update messages by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4910
docs: Update index.rst by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4913
doc: update the HTTP behavior test instruction by @frostming in https://github.com/bentoml/BentoML/pull/4915
docs: Add metrics doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4912
feat(cli): reload option for start-http-server by @frostming in https://github.com/bentoml/BentoML/pull/4916
chore: update pdm.lock file witht the lock targets by @frostming in https://github.com/bentoml/BentoML/pull/4911
fix(build): preserve the index url options in requirements.txt file by @frostming in https://github.com/bentoml/BentoML/pull/4914
fix: try to fix the timeout test cases by @frostming in https://github.com/bentoml/BentoML/pull/4917
doc: update CI status badge by @frostming in https://github.com/bentoml/BentoML/pull/4918

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.1...v1.3.2

- Python
Published by frostming almost 2 years ago

What's Changed

docs: Add 2 examples by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4872
docs: update theme to match branding by @ssheng in https://github.com/bentoml/BentoML/pull/4874
docs: Add task doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4873
fix: set prometheus env before importing service by @frostming in https://github.com/bentoml/BentoML/pull/4875
docs: Add API docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4877
fix: inject prometheus env from arbiter by @frostming in https://github.com/bentoml/BentoML/pull/4878
fix: Don't run bentoml.build_bentofile in subprocess by @frostming in https://github.com/bentoml/BentoML/pull/4876
docs: update azure byoc instructions by @ssheng in https://github.com/bentoml/BentoML/pull/4881
docs(azure): update azure byoc instructions by @ssheng in https://github.com/bentoml/BentoML/pull/4882
fix(framework): transformers vs 1.3.0 "TypeError: init() got an unexpected keyword argument 'task'" by @frostming in https://github.com/bentoml/BentoML/pull/4880
fix: silence git output by @frostming in https://github.com/bentoml/BentoML/pull/4883
fix: surface config error on bento build by @frostming in https://github.com/bentoml/BentoML/pull/4888
docs: Refactor tasks doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4890
Docs: Sync the BYOC doc with blog and update BentoCloud guides index page by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4891
fix: move user setup to base template to make custom base image work by @frostming in https://github.com/bentoml/BentoML/pull/4893
fix: Cannot define custom duration histogram buckets via @bentoml.service(metrics=...) by @frostming in https://github.com/bentoml/BentoML/pull/4895

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.0...v1.3.1

- Python
Published by frostming almost 2 years ago

We are excited to announce the release of BentoML 1.3! Following the feedback received since the launch of 1.2 earlier this year, we are introducing a host of new features and enhancements in 1.3. Below are the key highlights of 1.3 and stay tuned for an upcoming blog post, where we'll provide a detailed exploration of the new features and the driving forces behind the development.

Here are some of the important points to note about 1.3:

1.3 ensures full backward compatibility, meaning that Bentos built with 1.2 will continue to work seamlessly with this release.
We remain committed to supporting 1.2. Critical bug fixes and security updates will be backported to the 1.2 branch.
The BentoML documentation has been updated with examples and guides for 1.3. More guides will be added in the coming weeks.
BentoCloud supports Bento Deployments from both 1.2 and 1.3 releases of BentoML.

Now, let’s take a look at the major features and enhancements:

🕙 Implemented BentoML task execution

Introduced the @bentoml.task decorator to set a task endpoint for executing a long-running workload (such as batch processing or video generation).
Added the .submit() method to both the sync and async clients, which can submit task inputs via the task endpoint and dedicated worker processes constantly monitor task queues for new work to perform.
Full compatibility with BentoCloud to run Bentos defined with task endpoints.
See the Services and Clients doc with examples of a Service API by initializing a long running task in the Service constructor, creating clients to call the endpoint, and retrieving task status.

🚀 Optimized the build cache to accelerate the build process

Enhanced build speed for bentoml build & containerize through pre-installed large packages like torch
Switch to uv as the installer and resolver, replacing pip

🔨 Supported concurrency-based autoscaling on BentoCloud

Added the concurrency configuration to the @bentoml.service decorator to set the ideal number of simultaneous requests a Service is designed to handle.
Added the external_queue configuration to the @bentoml.service decorator to queue excess requests until they can be processed within the defined concurrency limits.
See the documentation to configure concurrency and external queue.

🔒 Secure data handling with secrets in BentoCloud:

You can now create and manage credentials, such as HuggingFace tokens and AWS secrets, securely on BentoCloud and easily apply them across multiple Deployments.
Added secret subcommands to the BentoML CLI for secret management. Run bentoml secret -h to learn more.

🗒️ Added streamed logs for Bento image deployment.

Easier to troubleshoot build issues and enable faster development iterations

🙏 Thank you for your continued support!

What's Changed

fix: change forbid extra keys to false for bentocloud by @FogDong in https://github.com/bentoml/BentoML/pull/4866
feat(dev): 1.3 by @frostming in https://github.com/bentoml/BentoML/pull/4849
fix: delete cluster and ns if it is first cluster by @FogDong in https://github.com/bentoml/BentoML/pull/4869
fix: auto login confirm ask logic by @xianml in https://github.com/bentoml/BentoML/pull/4864
fix: secret default value by @xianml in https://github.com/bentoml/BentoML/pull/4870
fix: fix typo in error msg by @FogDong in https://github.com/bentoml/BentoML/pull/4871

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.20...v1.3.0

- Python
Published by frostming almost 2 years ago

bentoml - v1.3.0a3

What's Changed

fix: don't fail when image pod is not found by @frostming in https://github.com/bentoml/BentoML/pull/4862
fix: async task response deserialization crash by @frostming in https://github.com/bentoml/BentoML/pull/4865

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.0a2...v1.3.0a3

- Python
Published by frostming almost 2 years ago

bentoml - v1.3.0a2

What's Changed

fix: options are not required by @xianml in https://github.com/bentoml/BentoML/pull/4861

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.3.0a1...v1.3.0a2

- Python
Published by xianml almost 2 years ago

bentoml - v1.3.0a1

What's Changed

feat: get_service api for selected frameworks by @frostming in https://github.com/bentoml/BentoML/pull/4782
feat: cli auto login by @xianml in https://github.com/bentoml/BentoML/pull/4817
feat: use rich.print to replace click.echo by @frostming in https://github.com/bentoml/BentoML/pull/4827
chore: update print and fix cors error by @xianml in https://github.com/bentoml/BentoML/pull/4829
feat: login to current context instead of default context if --context is not passed by @frostming in https://github.com/bentoml/BentoML/pull/4828
feat: switch to uv as the resolver and installer by @frostming in https://github.com/bentoml/BentoML/pull/4822
feat: Officially support Python 3.12 by @frostming in https://github.com/bentoml/BentoML/pull/4830
Docs: Add auto cli login docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4844
feat: retry and cancel task by @frostming in https://github.com/bentoml/BentoML/pull/4852

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.20...v1.3.0a1

- Python
Published by frostming almost 2 years ago

bentoml - v1.2.20

What's Changed

docs: Add logging and tracing docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4725
feat: --label option for bentoml build by @frostming in https://github.com/bentoml/BentoML/pull/4832
docs: Update the concurrency doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4834
docs: Update services.rst by @vikramjit-sidhu in https://github.com/bentoml/BentoML/pull/4837
docs: Update concurrency links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4840
docs: Fix broken link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4841
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4838
fix: stop spinner before starting to stream logs by @frostming in https://github.com/bentoml/BentoML/pull/4846
docs: Add azure byoc setup doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4847
docs: Update the BYOC architecture image by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4848
fix: fix error message with trace id by @FogDong in https://github.com/bentoml/BentoML/pull/4853
doc: locking by @bojiang in https://github.com/bentoml/BentoML/pull/4850
fix(containerize): command to create group and user in alpine base image by @frostming in https://github.com/bentoml/BentoML/pull/4855
fix: encode special charaters in path by @frostming in https://github.com/bentoml/BentoML/pull/4854
doc: locking for platform by @frostming in https://github.com/bentoml/BentoML/pull/4858
fix: correct the params passed to keras.save_model by @frostming in https://github.com/bentoml/BentoML/pull/4857

New Contributors

@vikramjit-sidhu made their first contribution in https://github.com/bentoml/BentoML/pull/4837

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.19...v1.2.20

- Python
Published by frostming almost 2 years ago

bentoml - v1.2.19

What's Changed

docs: Update diffusion model repos to the new one by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4797
feat(monitoring): support GRPC LogExporter by @judahrand in https://github.com/bentoml/BentoML/pull/4808
docs: Add Docker deployment for Bentos by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4812
docs: update examples service decorators with new features by @ssheng in https://github.com/bentoml/BentoML/pull/4798
doc: Custom Error Handling by @jianshen92 in https://github.com/bentoml/BentoML/pull/4811
doc: Wording change in error handling doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4813
fix: ASCII animation was not working properly in the image builder logs by @yetone in https://github.com/bentoml/BentoML/pull/4810
fix: avoid editing list in during iteration by @judahrand in https://github.com/bentoml/BentoML/pull/4809
docs: Add secrets doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4814
fix: improve the code of spinner and log tailer by @frostming in https://github.com/bentoml/BentoML/pull/4815
doc: Update the docs about multipart output by @frostming in https://github.com/bentoml/BentoML/pull/4818
docs: Add batch inference jobs doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4816
doc: Fix typo and format by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4821
fix(cli): hanging issue when deployment ready by @frostming in https://github.com/bentoml/BentoML/pull/4825
fix(deployment-client): add build process and extend url by @Haivilo in https://github.com/bentoml/BentoML/pull/4790
doc: Update vllm.rst by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4824
docs: Add custom models use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4819
docs: Reorg blip doc to multimodality section by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4826
fix: inject prometheus multiproc_dir into env var when worker starts by @frostming in https://github.com/bentoml/BentoML/pull/4820

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.18...v1.2.19

- Python
Published by frostming almost 2 years ago

bentoml - v1.2.18

What's Changed

docs: scale from zero by @ssheng in https://github.com/bentoml/BentoML/pull/4769
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4772
docs: Update installation.rst by @parano in https://github.com/bentoml/BentoML/pull/4763
docs: Add chatTTS to example project list by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4777
fix: bad performance when walking the file tree and check includes by @frostming in https://github.com/bentoml/BentoML/pull/4778
update readme to include chattts by @ssheng in https://github.com/bentoml/BentoML/pull/4786
docs: Update README.md by @parano in https://github.com/bentoml/BentoML/pull/4785
Fixed FileExistsError: [Errno 17] File exists: '/root/bentoml' by @tripathiarpan20 in https://github.com/bentoml/BentoML/pull/4781
fix: set env to null if it is empty for merge override by @FogDong in https://github.com/bentoml/BentoML/pull/4783
fix: add none check for env for deploy by @FogDong in https://github.com/bentoml/BentoML/pull/4792
fix(pandas): use correct mime type for Parquet by @judahrand in https://github.com/bentoml/BentoML/pull/4780
fix: add memoization to pkg_version_info to reduce cpu overhead by @adrianlyjak in https://github.com/bentoml/BentoML/pull/4794
fix: bug: memory leak when using bentoml>=1.2 by @frostming in https://github.com/bentoml/BentoML/pull/4775
fix: honor host and port config in service decorator by @frostming in https://github.com/bentoml/BentoML/pull/4776
fix(pandas): Annotated[pd.DataFrame, DataframeSchema(columns=["a"])] output spec by @judahrand in https://github.com/bentoml/BentoML/pull/4799
fix: ensure relative path works within bento project by @aarnphm in https://github.com/bentoml/BentoML/pull/4803
fix: map error responses from HTTP client to bentoml exceptions by @frostming in https://github.com/bentoml/BentoML/pull/4804
fix(numpy): numpy annotation processing by @judahrand in https://github.com/bentoml/BentoML/pull/4795
fix: populate bento name and version of server context by @frostming in https://github.com/bentoml/BentoML/pull/4787
fix(cloud-client): consider httpcore timeout when getting status by @Haivilo in https://github.com/bentoml/BentoML/pull/4740
fix: cd into src instead of bento parent by @frostming in https://github.com/bentoml/BentoML/pull/4805
feat: show image builder logs by @yetone in https://github.com/bentoml/BentoML/pull/4796

New Contributors

@tripathiarpan20 made their first contribution in https://github.com/bentoml/BentoML/pull/4781
@adrianlyjak made their first contribution in https://github.com/bentoml/BentoML/pull/4794

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.17...v1.2.18

- Python
Published by frostming almost 2 years ago

bentoml - v1.2.17

What's Changed

docs: update model store by @ssheng in https://github.com/bentoml/BentoML/pull/4741
docs: Add env var for mistral 7b docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4747
refactor(tests): testing.util.async_request to client.AsyncHTTPClient by @sanspareilsmyn in https://github.com/bentoml/BentoML/pull/4746
docs: Update GitHub readme by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4749
chore(build): update build_config.py by @eltociear in https://github.com/bentoml/BentoML/pull/4751
docs: Add git package packing doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4750
fix: configure server logging before importing service by @frostming in https://github.com/bentoml/BentoML/pull/4744
docs: Add user roles docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4724
feat(batching): break batch into smaller partitions if it's larger than max batch size by @frostming in https://github.com/bentoml/BentoML/pull/4752
docs: Update the examples folder readme by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4748
docs: Add byoc setup docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4757
fix: new sdk e2e test by @xianml in https://github.com/bentoml/BentoML/pull/4758
fix: metrics did not include 4xx by @xianml in https://github.com/bentoml/BentoML/pull/4756
fix: should enable existing loggers in server logging config by @rickzx in https://github.com/bentoml/BentoML/pull/4759
fix: client response deserialization when the output is string by @frostming in https://github.com/bentoml/BentoML/pull/4762
fix: middleware reorder by @xianml in https://github.com/bentoml/BentoML/pull/4761
fix: log errors inside a streaming response and detect service name conflicts by @frostming in https://github.com/bentoml/BentoML/pull/4767
fix: don't try to validate non-bytes as Path by @frostming in https://github.com/bentoml/BentoML/pull/4765

New Contributors

@sanspareilsmyn made their first contribution in https://github.com/bentoml/BentoML/pull/4746
@eltociear made their first contribution in https://github.com/bentoml/BentoML/pull/4751
@rickzx made their first contribution in https://github.com/bentoml/BentoML/pull/4759

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.16...v1.2.17

- Python
Published by frostming almost 2 years ago

bentoml - v1.2.16

What's Changed

fix(build): make sure to parse pip_args in correct order by @aarnphm in https://github.com/bentoml/BentoML/pull/4722
docs: Add compound data types doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4721
docs: Add docs for installing GitHub packages for Bento by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4720
docs: Update deployment details explanations by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4726
fix: load model aliases before loading new SDK service by @frostming in https://github.com/bentoml/BentoML/pull/4727
docs: Fix cpu as strings by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4735
docs: Update the services doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4734
feat: add trace id to monitoring module by @jianshen92 in https://github.com/bentoml/BentoML/pull/4732
feat: add a switch to opt-out packing git packages by @frostming in https://github.com/bentoml/BentoML/pull/4729
fix: missing components from output schema if it is a root model by @frostming in https://github.com/bentoml/BentoML/pull/4733
feat: expose the service url as a property by @frostming in https://github.com/bentoml/BentoML/pull/4730
fix(container): preheat packages support markers by @bojiang in https://github.com/bentoml/BentoML/pull/4737

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.15...v1.2.16

- Python
Published by frostming about 2 years ago

bentoml - v1.2.15

What's Changed

fix: handle subdirectory in git ssh url by @frostming in https://github.com/bentoml/BentoML/pull/4719

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.14...v1.2.15

- Python
Published by ssheng about 2 years ago

bentoml - v1.2.14

What's Changed

fix: add scaled to zero in client by @FogDong in https://github.com/bentoml/BentoML/pull/4712
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4713
fix: serailization error when request with an invalid json input by @frostming in https://github.com/bentoml/BentoML/pull/4714
fix: config pollution from other services by @frostming in https://github.com/bentoml/BentoML/pull/4715
fix: handle ssh git url without ref part by @frostming in https://github.com/bentoml/BentoML/pull/4716
fix: correct the default value of timeout-keep-alive by @frostming in https://github.com/bentoml/BentoML/pull/4717

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.13...v1.2.14

- Python
Published by frostming about 2 years ago

bentoml - v1.2.13

What's Changed

docs: fix indentation in build option docs by @jianshen92 in https://github.com/bentoml/BentoML/pull/4688
fix: bug: module 'bentoml' has no attribute 'build' by @frostming in https://github.com/bentoml/BentoML/pull/4689
docs: update autoscaling by @ssheng in https://github.com/bentoml/BentoML/pull/4691
docs: init io-descriptors example by @lycheel1 in https://github.com/bentoml/BentoML/pull/4646
fix: uncaught error missing trace headers by @frostming in https://github.com/bentoml/BentoML/pull/4692
chore: remove outdated examples by @larme in https://github.com/bentoml/BentoML/pull/4694
fix: bypass /livez in traffic middleware by @frostming in https://github.com/bentoml/BentoML/pull/4696
docs:Update credits by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4700
docs: Add model store reference in the Services doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4706
docs: Update example dir readme by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4707
fix: fix table overflow with fold by @FogDong in https://github.com/bentoml/BentoML/pull/4677
fix: server socket binding on Windows by @frostming in https://github.com/bentoml/BentoML/pull/4685
fix: correct the logic of bypassing infra endpoints in traffic middleware by @frostming in https://github.com/bentoml/BentoML/pull/4704
fix: add scaled to zero to ready status by @FogDong in https://github.com/bentoml/BentoML/pull/4708
chore(container): cache large pypi packages in layers by @bojiang in https://github.com/bentoml/BentoML/pull/4690
feat: add uvicorn timeouts options by @gusghrlrl101 in https://github.com/bentoml/BentoML/pull/4682
fix: include models from config for new SDK bentos by @frostming in https://github.com/bentoml/BentoML/pull/4705
fix: annotation preparation list iteration bug by @adamliter in https://github.com/bentoml/BentoML/pull/4701
fix: tuple schema and websocket connection by @frostming in https://github.com/bentoml/BentoML/pull/4709
fix: show warning instead of hard failure when model is missing by @frostming in https://github.com/bentoml/BentoML/pull/4693

New Contributors

@gusghrlrl101 made their first contribution in https://github.com/bentoml/BentoML/pull/4682
@adamliter made their first contribution in https://github.com/bentoml/BentoML/pull/4701

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.12...v1.2.13

- Python
Published by frostming about 2 years ago

bentoml - v1.2.12

What's Changed

docs: Add GPU inference doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4654
chore: update quickstart by @ssheng in https://github.com/bentoml/BentoML/pull/4655
docs: Add JSON output for bentovllm by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4657
chore: cleanup quickstart by @ssheng in https://github.com/bentoml/BentoML/pull/4658
docs: Update help info by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4664
fix: remove the uvicorn server header by @frostming in https://github.com/bentoml/BentoML/pull/4665
docs: Fix format by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4666
docs: Add model composition doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4668
docs: Update example project list by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4673
docs: Add the monitoring and data collection doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4662
docs: Add addasgimiddleware doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4672
fix: delete useless enum and fix enum value by @FogDong in https://github.com/bentoml/BentoML/pull/4674
docs: Add RAG tutorial by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4675
docs: Update the clients doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4676
docs: Add some explanations for bentoml.models.get by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4660
docs: Add e2e test doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4679
fix(cloud client): various type error by @bojiang in https://github.com/bentoml/BentoML/pull/4680
fix(cli): bentoml cli verbosity not passed to the subprocess correctly by @frostming in https://github.com/bentoml/BentoML/pull/4661

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.11...v1.2.12

- Python
Published by bojiang about 2 years ago

bentoml - v1.2.11

What's Changed

docs: added multi model load and store example by @lycheel1 in https://github.com/bentoml/BentoML/pull/4575
docs: inference graph by @lycheel1 in https://github.com/bentoml/BentoML/pull/4609
docs: Update high-level BentoML messaging by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4644
fix: delete trace error for timeout when deploying by @FogDong in https://github.com/bentoml/BentoML/pull/4649
fix(cli): correct the comparison of upload status by @frostming in https://github.com/bentoml/BentoML/pull/4643

New Contributors

@lycheel1 made their first contribution in https://github.com/bentoml/BentoML/pull/4575

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.10...v1.2.11

- Python
Published by frostming about 2 years ago

bentoml - v1.2.10

What's Changed

docs: Remove old example links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4610
feat(build): use jinja template for install script by @aarnphm in https://github.com/bentoml/BentoML/pull/4607
feat: download git+ssh dependencies to wheels/ folder by @frostming in https://github.com/bentoml/BentoML/pull/4612
docs: Update the ASGI doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4614
fix: non batched request executed in non FIFO manner by @frostming in https://github.com/bentoml/BentoML/pull/4619
feat: light-weighted dev sdist builder by @frostming in https://github.com/bentoml/BentoML/pull/4615
fix: add response headers for error responses by @frostming in https://github.com/bentoml/BentoML/pull/4622
docs: Refactor the autoscaling doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4620
docs: Add simple verification test for Services by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4629
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4630
docs: Add workers doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4628
docs: Add worker doc links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4636
docs: Add docs for bentoml.runner_service by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4637
docs: Add sync to async doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4639
fix: handle call on clients by @frostming in https://github.com/bentoml/BentoML/pull/4618
fix(cli): bentoml get output in YAML format by @frostming in https://github.com/bentoml/BentoML/pull/4638
fix: issue overriding default service config from config file by @frostming in https://github.com/bentoml/BentoML/pull/4627

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.9...v1.2.10

- Python
Published by frostming about 2 years ago

bentoml - v1.2.9

What's Changed

docs: add autoscaling docs by @ssheng in https://github.com/bentoml/BentoML/pull/4601
docs: Fix old configurations doc link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4600
docs: Update the concurrency doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4602
fix(reload): make sure to watch for all src for editable by @aarnphm in https://github.com/bentoml/BentoML/pull/4604
fix(openapi): remove invalid fields for operations parameter by @aarnphm in https://github.com/bentoml/BentoML/pull/4605
chore: update swagger UI generated to 5.12 [skip ci] by @aarnphm in https://github.com/bentoml/BentoML/pull/4606
fix(cloud cli): workaround env update issue by @bojiang in https://github.com/bentoml/BentoML/pull/4608

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.8...v1.2.9

- Python
Published by bojiang about 2 years ago

bentoml - v1.2.8

What's Changed

fix(stub): add envs to __init__ construction by @aarnphm in https://github.com/bentoml/BentoML/pull/4576
docs: Add testing doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4578
chore(type): use TypedDict for inferred stubs by @aarnphm in https://github.com/bentoml/BentoML/pull/4577
docs: Add lifecycle hooks doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4579
fix(build): remove deprecated warnings by @aarnphm in https://github.com/bentoml/BentoML/pull/4580
fix: improve the bentoml build progress by @frostming in https://github.com/bentoml/BentoML/pull/4582
fix: allow API endpoint to return response directly by @frostming in https://github.com/bentoml/BentoML/pull/4581
fix: raise the priority of the routes from mount apps by @frostming in https://github.com/bentoml/BentoML/pull/4584
fix(sdk): python build missing envs argument by @jianshen92 in https://github.com/bentoml/BentoML/pull/4585
fix: build config env default to None by @jianshen92 in https://github.com/bentoml/BentoML/pull/4589
fix: config from multiple services override with each other by @frostming in https://github.com/bentoml/BentoML/pull/4591
feat: add concurrency config by @xianml in https://github.com/bentoml/BentoML/pull/4560
docs: Add OpenAI compatible endpoints client doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4593
fix: ignoreextrakeys for v2 configuration schema by @xianml in https://github.com/bentoml/BentoML/pull/4598
fix: emit warnings when copying large file by @frostming in https://github.com/bentoml/BentoML/pull/4597
feat: service.to_async property by @frostming in https://github.com/bentoml/BentoML/pull/4595
feat: picklev5 serialization by @frostming in https://github.com/bentoml/BentoML/pull/4594

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.7...v1.2.8

- Python
Published by frostming about 2 years ago

bentoml - v1.2.7

What's Changed

docs: Add doc for serverreadytimeout by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4563
fix: depracate all enum in schema by @FogDong in https://github.com/bentoml/BentoML/pull/4565
feat: threaded serving by @frostming in https://github.com/bentoml/BentoML/pull/4562
docs: Adjust existing Bento deployment wording by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4567
fix: fix list instance type by @FogDong in https://github.com/bentoml/BentoML/pull/4566
docs: Add doc for using pydantic model at the top level by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4568
fix: delete access in admin console by @FogDong in https://github.com/bentoml/BentoML/pull/4569
fix(client): set upload timeouts to 10 hours by @sauyon in https://github.com/bentoml/BentoML/pull/4564
fix: respect the content type config for file output by @frostming in https://github.com/bentoml/BentoML/pull/4570
docs: Update the adaptive batching doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4571
feat: Service.to_asgi() method by @frostming in https://github.com/bentoml/BentoML/pull/4572
docs: Add asgi doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4574
fix: allow legacy runnable methods when calling API by @frostming in https://github.com/bentoml/BentoML/pull/4573

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.6...v1.2.7

- Python
Published by frostming about 2 years ago

bentoml - v1.2.6

What's Changed

docs: Add envs field in bentofile by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4555
docs: Add GitHub repo links for use case docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4556
docs: Update the vllm use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4558
docs: Add how to get deployment info doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4544
fix: rename the bentomlsdk.api module by @frostming in https://github.com/bentoml/BentoML/pull/4557
fix: wait until server ready when initializing client by @frostming in https://github.com/bentoml/BentoML/pull/4559
feat: Add server.on_deployment hook to execute deployment-specific logic by @frostming in https://github.com/bentoml/BentoML/pull/4561

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.5...v1.2.6

- Python
Published by frostming about 2 years ago

bentoml - v1.2.5

What's Changed

docs: Add client code examples without context manager by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4512
docs: Update docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4515
docs: Add authorization docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4517
docs: Change sample input to one line by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4518
docs: Update ControlNet use case docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4519
docs: Update the distributed services and get started docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4521
refactor(cli): make CLI commands available as modules by @frostming in https://github.com/bentoml/BentoML/pull/4487
docs: Refactor BentoCloud docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4525
docs: Update token docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4526
fix: fix list deployment exception by @FogDong in https://github.com/bentoml/BentoML/pull/4522
feat: add get endpoint url for deployment by @FogDong in https://github.com/bentoml/BentoML/pull/4516
docs: Add BentoCloud CLI and API ref docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4527
fix: add missing to dict for deployment config by @FogDong in https://github.com/bentoml/BentoML/pull/4529
feat(client): disallow media type other than json for HTTP clients by @frostming in https://github.com/bentoml/BentoML/pull/4520
chore: clean unneeded dependencies by @frostming in https://github.com/bentoml/BentoML/pull/4528
docs: Add call deployment endpoints doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4531
docs: Update all use cases by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4532
fix: correctly encode ValueError in response by @frostming in https://github.com/bentoml/BentoML/pull/4530
fix: improve CLI message by @frostming in https://github.com/bentoml/BentoML/pull/4533
chore(io): allow implicit pydantic type conversion by @aarnphm in https://github.com/bentoml/BentoML/pull/4534
fix: type convertion of io descriptor by @frostming in https://github.com/bentoml/BentoML/pull/4537
feat: add back service hooks by @frostming in https://github.com/bentoml/BentoML/pull/4524
docs: fix typo by @sauyon in https://github.com/bentoml/BentoML/pull/4540
docs: Add BentoSVD Use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4535
doc: Add adaptive batching doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4536
doc: allow reuse of existing directory by @jianshen92 in https://github.com/bentoml/BentoML/pull/4543
feat: cloud context from env vars by @frostming in https://github.com/bentoml/BentoML/pull/4542
docs: Update the clients doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4539
docs: Update adaptive batching docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4547
docs: Update vLLM use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4548
feat: export component context at top namespace by @frostming in https://github.com/bentoml/BentoML/pull/4551
feat: use starlette request directly in service context by @frostming in https://github.com/bentoml/BentoML/pull/4552
feat: mount asgi app and fastapi support by @frostming in https://github.com/bentoml/BentoML/pull/4538
fix(sdk): build envs into dockerfile by @bojiang in https://github.com/bentoml/BentoML/pull/4546
fix: rename server context attributes by @frostming in https://github.com/bentoml/BentoML/pull/4553
fix: crash when pulling bento from organization with proxy mode by @frostming in https://github.com/bentoml/BentoML/pull/4554
fix: add str in enum for deployment schema by @FogDong in https://github.com/bentoml/BentoML/pull/4549

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.4...v1.2.5

- Python
Published by frostming about 2 years ago

bentoml - v1.2.4

What's Changed

docs: Update the get started docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4513
fix(sdk): incorrect bento_path if not provided by @bojiang in https://github.com/bentoml/BentoML/pull/4514

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.3...v1.2.4

- Python
Published by bojiang over 2 years ago

bentoml - v1.2.3

What's Changed

ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4488
infra: update to use Ruff formatter by @aarnphm in https://github.com/bentoml/BentoML/pull/4269
feat: tpu_type support by @frostming in https://github.com/bentoml/BentoML/pull/4493
chore(cloud cli): add cluster and expose access_point by @bojiang in https://github.com/bentoml/BentoML/pull/4490
docs: Update the whisperx use case by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4495
docs: Update the readme by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4496
fix(sdk): current directory for built bentos by @bojiang in https://github.com/bentoml/BentoML/pull/4505
chore(cloud cli): rename cluster to region by @bojiang in https://github.com/bentoml/BentoML/pull/4508
doc: Add the lcm lora use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4510
fix(sdk): clean bentoml version by @bojiang in https://github.com/bentoml/BentoML/pull/4511
fix: bug: Dataframes not serializing correctly in the new API by @frostming in https://github.com/bentoml/BentoML/pull/4491

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.2...v1.2.3

- Python
Published by ssheng over 2 years ago

bentoml - v1.2.2

What's Changed

docs: Remove direct execution lines by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4480
docs: Refactor deployment docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4482
docs: Add build options doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4483
feat: lock dependencies by default considering the current bentoml version by @frostming in https://github.com/bentoml/BentoML/pull/4484
docs: Add distributed services doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4485
fix(server): clean the request resources after the response is consumed by @frostming in https://github.com/bentoml/BentoML/pull/4481

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.1...v1.2.2

- Python
Published by frostming over 2 years ago

bentoml - v1.2.1

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.1a1...v1.2.1

What's Changed

docs: Add more featured use cases by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4476
docs: Add API reference doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4475
docs: Clean API ref docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4477
fix: fix the cluster from display name to name and url by @FogDong in https://github.com/bentoml/BentoML/pull/4478
fix(cli): add start runner server back by @bojiang in https://github.com/bentoml/BentoML/pull/4479

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0...v1.2.1

- Python
Published by bojiang over 2 years ago

bentoml - v1.2.1a1

What's Changed

docs: Add more featured use cases by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4476
docs: Add API reference doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4475
docs: Clean API ref docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4477
fix: fix the cluster from display name to name and url by @FogDong in https://github.com/bentoml/BentoML/pull/4478
fix(cli): add start runner server back by @bojiang in https://github.com/bentoml/BentoML/pull/4479

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0...v1.2.1a1

- Python
Published by bojiang over 2 years ago

bentoml - v1.2.0

What's Changed

chore(deps): bump h2 from 0.3.20 to 0.3.24 in /grpc-client/rust by @dependabot in https://github.com/bentoml/BentoML/pull/4434
fix: Remove trailing character when building bento with API on Windows by @holzweber in https://github.com/bentoml/BentoML/pull/4455
fix: Replace backslahes by normal slashes, making bentoml pull possible on windows by @holzweber in https://github.com/bentoml/BentoML/pull/4456
fix(monitoring): Missing f string by @jianshen92 in https://github.com/bentoml/BentoML/pull/4463
feat: 1.2 staging by @bojiang in https://github.com/bentoml/BentoML/pull/4366
chore(deps): bump pdm-project/setup-pdm from 3 to 4 by @dependabot in https://github.com/bentoml/BentoML/pull/4457
fix(client): Convert string to Path if it isn't like a URL by @frostming in https://github.com/bentoml/BentoML/pull/4469
docs: Add Model Store doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4471
docs: Clean up BentoCloud doc and add get started doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4472
docs: Update the quickstart by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4474

New Contributors

@holzweber made their first contribution in https://github.com/bentoml/BentoML/pull/4455

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0rc1...v1.2.0

- Python
Published by ssheng over 2 years ago

bentoml - v1.2.0rc1

What's Changed

fix: correct schema for optional type by @frostming in https://github.com/bentoml/BentoML/pull/4459
fix: client sending request fail if the input is a list of files by @frostming in https://github.com/bentoml/BentoML/pull/4461
fix: worker number of start-http-server by @bojiang in https://github.com/bentoml/BentoML/pull/4462

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a7...v1.2.0rc1

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a7

What's Changed

feat: allow disabling GPU allocation via env by @frostming in https://github.com/bentoml/BentoML/pull/4453

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a6...v1.2.0a7

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a6

What's Changed

feat: add adaptive batch size histogram by @frostming in https://github.com/bentoml/BentoML/pull/4438
ci: Use nox as the task runner by @frostming in https://github.com/bentoml/BentoML/pull/4432
feat(cli): remove or hide unneeded commands by @frostming in https://github.com/bentoml/BentoML/pull/4441
docs: Add whisperx use case doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4436
docs: Add Docker deployment doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4443
docs: Add io types doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4444
fix: only inline display images by @frostming in https://github.com/bentoml/BentoML/pull/4442
docs: Update homepage by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4445
feat: skip access log for infra endpoints by @frostming in https://github.com/bentoml/BentoML/pull/4447
feat: add schema in bentoInfo by @xianml in https://github.com/bentoml/BentoML/pull/4439
fix: fix deployment cli log by @FogDong in https://github.com/bentoml/BentoML/pull/4450
fix: convert generator to async gen before response by @frostming in https://github.com/bentoml/BentoML/pull/4448

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a5...v1.2.0a6

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a5

What's Changed

fix: image encoding issue when the format is not specified by @frostming in https://github.com/bentoml/BentoML/pull/4435

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a4...v1.2.0a5

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a4

What's Changed

fix: make client timeout consistent with service config by @frostming in https://github.com/bentoml/BentoML/pull/4425
fix: fix env in deploy and create error msg by @FogDong in https://github.com/bentoml/BentoML/pull/4426
fix: port reservation causes bind error on Windows by @frostming in https://github.com/bentoml/BentoML/pull/4427
fix: add more status check in wait until ready by @FogDong in https://github.com/bentoml/BentoML/pull/4428
fix: use spinner log instead of console log by @FogDong in https://github.com/bentoml/BentoML/pull/4429
fix: add config dict to cli by @FogDong in https://github.com/bentoml/BentoML/pull/4424
docs: Add client and use case docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4430
fix(sdk): snake case bento names by @bojiang in https://github.com/bentoml/BentoML/pull/4431
docs: Add SDXL turbo use case by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4433
docs: Update readme doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4409

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a3...v1.2.0a4

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a3

What's Changed

fix(server): start-http-server for v1.0 bentos by @bojiang in https://github.com/bentoml/BentoML/pull/4421
fix: refactor cli & sdk for deployment by @FogDong in https://github.com/bentoml/BentoML/pull/4419
fix: fix get bento from cloud by @FogDong in https://github.com/bentoml/BentoML/pull/4423
fix: change entry_service to optional by @xianml in https://github.com/bentoml/BentoML/pull/4422

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a2...v1.2.0a3

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a2

What's Changed

feat: allow url as file input by @frostming in https://github.com/bentoml/BentoML/pull/4411
fix(sdk): validate service name; force lowercase by @bojiang in https://github.com/bentoml/BentoML/pull/4414
refactor(bento): add entry_service to bento info by @bojiang in https://github.com/bentoml/BentoML/pull/4417
fix: set content type from annotation by @frostming in https://github.com/bentoml/BentoML/pull/4416
fix(server): include system headers by @bojiang in https://github.com/bentoml/BentoML/pull/4418
fix: loading service from an absolute path by @frostming in https://github.com/bentoml/BentoML/pull/4415
docs: Add BLIP, CLIP, and sentence embedding use cases by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4420

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a1...v1.2.0a2

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a1

What's Changed

fix(docs): remove eager type evaluation for python 3.8 on openllm examples by @aarnphm in https://github.com/bentoml/BentoML/pull/4393
docs: fix typo in tritonserver_type parameter and code example by @ilyahabr in https://github.com/bentoml/BentoML/pull/4394

New Contributors

@ilyahabr made their first contribution in https://github.com/bentoml/BentoML/pull/4394

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.2.0a0...v1.2.0a1

- Python
Published by frostming over 2 years ago

bentoml - v1.2.0a0

What's Changed

feat: add preview feature for output by @jinyang1994 in https://github.com/bentoml/BentoML/pull/4319
feat: add feature for form validation by @jinyang1994 in https://github.com/bentoml/BentoML/pull/4322
fix: example and bento config by @FogDong in https://github.com/bentoml/BentoML/pull/4324
fix: set wrong default value when type is array by @jinyang1994 in https://github.com/bentoml/BentoML/pull/4332
feat: add v2 config and json override env by @FogDong in https://github.com/bentoml/BentoML/pull/4331
feat: config override by @frostming in https://github.com/bentoml/BentoML/pull/4334
feat: models.save api and tests by @MingLiangDai in https://github.com/bentoml/BentoML/pull/4307
fix: fix config migration by @FogDong in https://github.com/bentoml/BentoML/pull/4341
fix: async api call by @xianml in https://github.com/bentoml/BentoML/pull/4349
fix(config): typo on override by @Haivilo in https://github.com/bentoml/BentoML/pull/4351
feat: reorganize the new SDK package by @frostming in https://github.com/bentoml/BentoML/pull/4337
feat: support .python-version symlink by @aarnphm in https://github.com/bentoml/BentoML/pull/4354
feat: add loading status when form is submitting by @jinyang1994 in https://github.com/bentoml/BentoML/pull/4361
feat: add e2e tests for new SDK by @frostming in https://github.com/bentoml/BentoML/pull/4352
fix(with_config): annotate return type by @aarnphm in https://github.com/bentoml/BentoML/pull/4355
Chore: add supported gpu type by @xianml in https://github.com/bentoml/BentoML/pull/4363
fix(config): make sure to escape quotation for migration of services config by @aarnphm in https://github.com/bentoml/BentoML/pull/4369
fix(sdk): identify async by original func by @bojiang in https://github.com/bentoml/BentoML/pull/4370
chore(bento.yaml): move fields into services by @bojiang in https://github.com/bentoml/BentoML/pull/4372
fix: add services in manifest by @FogDong in https://github.com/bentoml/BentoML/pull/4373
chore(sdk): envs in bentofile by @bojiang in https://github.com/bentoml/BentoML/pull/4378
chore(cloud): include envs in manifest by @bojiang in https://github.com/bentoml/BentoML/pull/4379
chore: cherry-pick SSE utils into 1.2 branch by @aarnphm in https://github.com/bentoml/BentoML/pull/4375
fix: correct 1.2 model list and tag format when pushing bento by @Haivilo in https://github.com/bentoml/BentoML/pull/4381
feat: Update the bento yaml schema by @frostming in https://github.com/bentoml/BentoML/pull/4371
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4382
chore(sdk): able to specify service name by @bojiang in https://github.com/bentoml/BentoML/pull/4377
feat(bentocloud): deployment v2 api client + cli by @Haivilo in https://github.com/bentoml/BentoML/pull/4335
chore(deps): bump github/codeql-action from 2 to 3 by @dependabot in https://github.com/bentoml/BentoML/pull/4343
feat(sdk): use attribute chain as dependency import string by @frostming in https://github.com/bentoml/BentoML/pull/4385
chore(example): change name to avoid conflict by @bojiang in https://github.com/bentoml/BentoML/pull/4387
refactor(impl): 1.2 loader by @bojiang in https://github.com/bentoml/BentoML/pull/4388
fix: refactor deployment v2 client and cli by @FogDong in https://github.com/bentoml/BentoML/pull/4383

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.11...v1.2.0a0

- Python
Published by frostming over 2 years ago

bentoml - BentoML - v1.1.11

Bug fixes

Fix streaming for long payloads on remote runners. It will now always yield text and follow SSE protocol. We also provide SSE utils: ```python import bentoml from bentoml.io import SSE

class MyRunnable(bentoml.Runnable): @bentoml.Runnable.method() def streaming(self, text): yield "data: 1\n\n" yield "data: 12222222222222222222222222222\n\n"

runner = bentoml.Runner(MyRunnable)

svc = bentoml.Service("service", runners=[runner])

@svc.api() def infer(text): result = 0 async for it in runner.streaming.asyncstream(text): payload = SSE.fromiterator(it) result += int(payload.data) return result ```

What's Changed

docs: Add BentoCloud payment doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4286
docs: update quickstart with OpenLLM by @aarnphm in https://github.com/bentoml/BentoML/pull/4295
fix(docs): correct server implementation by @aarnphm in https://github.com/bentoml/BentoML/pull/4297
docs: Remove bill void status by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4299
docs: Update LLM quickstart format and wording by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4300
fix: citation link at README.md by @shenxiangzhuang in https://github.com/bentoml/BentoML/pull/4301
fix(transformers): support trust_remote_code and added unit tests by @MingLiangDai in https://github.com/bentoml/BentoML/pull/4271
docs: Add Bento Deployment details docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4304
test: remove outdated tests with pretrained_class parameter by @MingLiangDai in https://github.com/bentoml/BentoML/pull/4308
fix: Updated starlette to >= 0.24.0 by @jakthra in https://github.com/bentoml/BentoML/pull/4306
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4316
fix: syntax error on code snippet in bentoml.onnx.save_model docs by @lucasew in https://github.com/bentoml/BentoML/pull/4323
chore(deps): bump actions/setup-python from 4 to 5 by @dependabot in https://github.com/bentoml/BentoML/pull/4329
docs: fixed typo in file name benchmark README.md by @gazon1 in https://github.com/bentoml/BentoML/pull/4342
fix(stream): streaming enable to work with proxy by @jianshen92 in https://github.com/bentoml/BentoML/pull/4330
docs: fix typo in frameworks transformers guide by @IbrahimAmin1 in https://github.com/bentoml/BentoML/pull/4360
chore(sse): refactor sse utils with efficient buffering by @bojiang in https://github.com/bentoml/BentoML/pull/4362
fix(runner): fix DataFrame container header too long by @larme in https://github.com/bentoml/BentoML/pull/4364
chore(generated): new stubs for proto 4 by @aarnphm in https://github.com/bentoml/BentoML/pull/4374

New Contributors

@shenxiangzhuang made their first contribution in https://github.com/bentoml/BentoML/pull/4301
@jakthra made their first contribution in https://github.com/bentoml/BentoML/pull/4306
@lucasew made their first contribution in https://github.com/bentoml/BentoML/pull/4323
@gazon1 made their first contribution in https://github.com/bentoml/BentoML/pull/4342
@IbrahimAmin1 made their first contribution in https://github.com/bentoml/BentoML/pull/4360

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.10...v1.1.11

- Python
Published by aarnphm over 2 years ago

bentoml - BentoML - v1.1.10

Released a patch that set the upper bound for cattrs<23.2, which breaks our whole serialisation process both upstream and downstream.

What's Changed

fix: StreamingResponse compatibility issue by @xianml in https://github.com/bentoml/BentoML/pull/4248
bentocloud doc update --data field by @MingLiangDai in https://github.com/bentoml/BentoML/pull/4272
docs: Add repository and bento selection notes by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4280
fix(dispatcher): unbounded overload batch_size by @aarnphm in https://github.com/bentoml/BentoML/pull/4273
docs: update transformers example to include gpu options by @ssheng in https://github.com/bentoml/BentoML/pull/4281
fix monitoring docs configuration.yaml typo by @KimSoungRyoul in https://github.com/bentoml/BentoML/pull/4287
fix: runnable framework logic in transformers.py by @benfu-verses in https://github.com/bentoml/BentoML/pull/4291
docs: Update docs on supported CUDA versions by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4288
docs: Add docs for new transformers model import API by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4282
fix: Disable exception in serve-grpc on Windows in development mode by @zimka in https://github.com/bentoml/BentoML/pull/4294
fix(dependencies): lock cattrs<23.2 for now by @aarnphm in https://github.com/bentoml/BentoML/pull/4292

New Contributors

@benfu-verses made their first contribution in https://github.com/bentoml/BentoML/pull/4291
@zimka made their first contribution in https://github.com/bentoml/BentoML/pull/4294

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.9...v1.1.10

- Python
Published by aarnphm over 2 years ago

bentoml - BentoML - v1.1.9

Import Hugging Face Transformers Model: the bentoml.transformers.import_model API imports pretrained transformers models directly from HuggingFace. Using this API allows importing Transformers models into the BentoML model store without loading the model into memory. The bentoml.transformers.import_model API takes the first argument to be the model name in BentoML store, and the second argument to be the model_id on HuggingFace Hub.

```python import bentoml

bentomodel = bentoml.transformers.import_model("zephyr-7b-beta", "HuggingFaceH4/zephyr-7b-beta") ```

Standardize with nvidia-ml-py: BentoML now uses the official nvidia-ml-py package instead of pynvml to avoid conflict with other packages.
Define Environment Variable in Configuration: Within bentoml_configuration.yaml, values in the form of ${ENV_VAR} will be expanded at runtime to the value of the corresponding environment variable, but please note that this only supports string types.

What's Changed

docs: Update the deployment docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4260
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4264
feat: import model for transformers framework by @MingLiangDai in https://github.com/bentoml/BentoML/pull/4247
build: Use official nvidia-ml-py package instead of fork by @ecederstrand in https://github.com/bentoml/BentoML/pull/4208

New Contributors

@MingLiangDai made their first contribution in https://github.com/bentoml/BentoML/pull/4247
@ecederstrand made their first contribution in https://github.com/bentoml/BentoML/pull/4208

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.7...v1.1.9

- Python
Published by ssheng over 2 years ago

bentoml - BentoML - v1.1.8

What's Changed

docs: Add the OpenLLM Llama 2 Colab link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4235
docs: Add best practices doc for building and deploying Bentos by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4237
docs: Update the bento building and deployment best practices doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4242
docs: Add global token and token expiration in docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4243
fix: client API by @alexparker443 in https://github.com/bentoml/BentoML/pull/4245
docs: Update the Bentos doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4246
docs: Update OneDiffusion Colab link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4249
fix: suppress abstractmethod TypeError of TritonRunnerHandle by @netoou in https://github.com/bentoml/BentoML/pull/4251
fix: send request ID in response headers in all cases by @frostming in https://github.com/bentoml/BentoML/pull/4253
fix(client): prepend http if necessary in wait by @sauyon in https://github.com/bentoml/BentoML/pull/4255
fix: Bug 4252 Restore functioning benoml.ray.deployment by @jerryharrow in https://github.com/bentoml/BentoML/pull/4257
fix(client): add http if required to sync client wait by @sauyon in https://github.com/bentoml/BentoML/pull/4258
fix(ci): fix tests by @sauyon in https://github.com/bentoml/BentoML/pull/4259
feat: support storing secrets with env vars in config by @frostming in https://github.com/bentoml/BentoML/pull/4254

New Contributors

@alexparker443 made their first contribution in https://github.com/bentoml/BentoML/pull/4245
@netoou made their first contribution in https://github.com/bentoml/BentoML/pull/4251
@jerryharrow made their first contribution in https://github.com/bentoml/BentoML/pull/4257

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.7...v1.1.8

- Python
Published by ssheng over 2 years ago

bentoml - BentoML - v1.1.7

What's Changed

Update OTEL deps to 0.41b0 to address CVE for 0.39b0

General documentation client updates.

docs: Add the SDXL deployment quickstart by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4175
Update pytorch.rst by @piercus in https://github.com/bentoml/BentoML/pull/4176
chore(deps): bump actions/checkout from 3 to 4 by @dependabot in https://github.com/bentoml/BentoML/pull/4177
fix: parse tag from multiline output by @frostming in https://github.com/bentoml/BentoML/pull/4178
docs: Update the user management docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4186
fix(config): set default runner timeout to 15min by @sauyon in https://github.com/bentoml/BentoML/pull/4184
docs: Add observability to the BentoCloud overview docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4187
fix(framework): add args and kwargs to sklearn and xgboost methods by @jianshen92 in https://github.com/bentoml/BentoML/pull/4189
docs: fix typo in bento.rst and model.rst by @seedspirit in https://github.com/bentoml/BentoML/pull/4192
fix: Rename ASGIHTTPSender to BufferedASGISender for Ray compatibility. by @HamzaFarhan in https://github.com/bentoml/BentoML/pull/4191
fix(client): make get_client raise instead of logging by @sauyon in https://github.com/bentoml/BentoML/pull/4181
fix(cloud-client): delete unused field of schema by @Haivilo in https://github.com/bentoml/BentoML/pull/4196
chore(deps): bump docker/setup-buildx-action from 2 to 3 by @dependabot in https://github.com/bentoml/BentoML/pull/4195
chore(deps): bump docker/setup-qemu-action from 2 to 3 by @dependabot in https://github.com/bentoml/BentoML/pull/4194
chore: clientrequesthook type fix by @sauyon in https://github.com/bentoml/BentoML/pull/4199
docs: Add docs for the new bentoml.Server API by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4198
docs: Add the OneDiffusion Google Colab task by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4202
docs: Add best practices doc for cost optimization by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4200
docs: Update the Manage Models and Bentos docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4203
fix: do not use UDS on WSL by @frostming in https://github.com/bentoml/BentoML/pull/4204
docs: fix typos in help messages by @smidm in https://github.com/bentoml/BentoML/pull/4206
fix: subprocess not using same python as main process causing bentoml.bentos.build to crash by @nickolasrm in https://github.com/bentoml/BentoML/pull/4209
fix: allow WSL in the condition by @frostming in https://github.com/bentoml/BentoML/pull/4210
docs: Update manage access token docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4215
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4216
fix: EasyOCR integration docs mistake by @jianshen92 in https://github.com/bentoml/BentoML/pull/4214
fix: include mounted FastAPI app's OpenAPI components by @RobbieFernandez in https://github.com/bentoml/BentoML/pull/4212
UPDATE: model.py -> fix Model class Exepction message. by @JminJ in https://github.com/bentoml/BentoML/pull/4219
docs: Remove private access mention by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4221
docs: Change to sentence case by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4222
docs: Fix dead link by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4225
feat: support ipv6 addresses for serve by @sauyon in https://github.com/bentoml/BentoML/pull/3914
docs: Fix all dead links in BentoML docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4229
docs: Add the BYOC doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4223
docs: Update the Services doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4231
fix(client): type fixes by @sauyon in https://github.com/bentoml/BentoML/pull/4182
fix: correct the bento size to include the size of models by @frostming in https://github.com/bentoml/BentoML/pull/4226
fix: use httpx for usage tracking by @sauyon in https://github.com/bentoml/BentoML/pull/4228
fix(deps): bump otel for CVE by @aarnphm in https://github.com/bentoml/BentoML/pull/4233
feat: separate and optimize async and sync clients by @judahrand in https://github.com/bentoml/BentoML/pull/4116

New Contributors

@piercus made their first contribution in https://github.com/bentoml/BentoML/pull/4176
@seedspirit made their first contribution in https://github.com/bentoml/BentoML/pull/4192
@HamzaFarhan made their first contribution in https://github.com/bentoml/BentoML/pull/4191
@nickolasrm made their first contribution in https://github.com/bentoml/BentoML/pull/4209
@JminJ made their first contribution in https://github.com/bentoml/BentoML/pull/4219

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.6...v1.1.7

- Python
Published by aarnphm over 2 years ago

bentoml - BentoML - v1.1.6

What's Changed

fix(exception): catch exception for users' runners code by @aarnphm in https://github.com/bentoml/BentoML/pull/4150
docs: Add the streaming docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4164
ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in https://github.com/bentoml/BentoML/pull/4167
fix(httpclient): take into account trailing slash in from_url by @sauyon in https://github.com/bentoml/BentoML/pull/4169
docs: fix typo by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4173
fix: apply env map for distributed runner workers by @bojiang in https://github.com/bentoml/BentoML/pull/4174

New Contributors

@pre-commit-ci made their first contribution in https://github.com/bentoml/BentoML/pull/4167

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.5...v1.1.6

- Python
Published by ssheng over 2 years ago

bentoml - BentoML - v1.1.5

What's Changed

fix(type): explicit init for attrs Runner by @aarnphm in https://github.com/bentoml/BentoML/pull/4140
fix: typo in ALLOWEDCUDAVERSION_ARGS by @thomasjo in https://github.com/bentoml/BentoML/pull/4156
chore(deps): open Starlette version, to allow latest by @alexeyshockov in https://github.com/bentoml/BentoML/pull/4100
chore: lower bound for cloudpickle by @aarnphm in https://github.com/bentoml/BentoML/pull/4098
docs: Add embedded runners docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4157
fix cloud client types by @sauyon in https://github.com/bentoml/BentoML/pull/4160
fix: use closer-integrated callbackwrapper by @sauyon in https://github.com/bentoml/BentoML/pull/4161
chore(annotations): cleanup compat and fix ModelSignatureDict type by @aarnphm in https://github.com/bentoml/BentoML/pull/4162
fix(pull): correct use cloud_context for models pull by @aarnphm in https://github.com/bentoml/BentoML/pull/4163

New Contributors

@thomasjo made their first contribution in https://github.com/bentoml/BentoML/pull/4156
@alexeyshockov made their first contribution in https://github.com/bentoml/BentoML/pull/4100

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.4...v1.1.5

- Python
Published by ssheng over 2 years ago

bentoml - BentoML - v1.1.4

🍱 To better support LLM serving through response streaming, we are proud to introduce an experimental support of server-sent events (SSE) streaming support in this release of BentoML v1.14 and OpenLLM v0.2.27. See an example service definition for SSE streaming with Llama2.

Added response streaming through SSE to the bentoml.io.Text IO Descriptor type.
Added async generator support to both API Server and Runner to yield incremental text responses.
Added supported to ☁️ BentoCloud to natively support SSE streaming.

🦾 OpenLLM added token streaming capabilities to support streaming responses from LLMs.

Added /v1/generate_stream endpoint for streaming responses from LLMs.

bash curl -N -X 'POST' 'http://0.0.0.0:3000/v1/generate_stream' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{ "prompt": "### Instruction:\n What is the definition of time (200 words essay)?\n\n### Response:", "llm_config": { "use_llama2_prompt": false, "max_new_tokens": 4096, "early_stopping": false, "num_beams": 1, "num_beam_groups": 1, "use_cache": true, "temperature": 0.89, "top_k": 50, "top_p": 0.76, "typical_p": 1, "epsilon_cutoff": 0, "eta_cutoff": 0, "diversity_penalty": 0, "repetition_penalty": 1, "encoder_repetition_penalty": 1, "length_penalty": 1, "no_repeat_ngram_size": 0, "renormalize_logits": false, "remove_invalid_values": false, "num_return_sequences": 1, "output_attentions": false, "output_hidden_states": false, "output_scores": false, "encoder_no_repeat_ngram_size": 0, "n": 1, "best_of": 1, "presence_penalty": 0.5, "frequency_penalty": 0, "use_beam_search": false, "ignore_eos": false }, "adapter_name": null }'

What's Changed

docs: Update the models doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4145
docs: Add more workflows to the GitHub Actions doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4146
docs: Add text embedding example to readme by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4151
fix: bento build cache miss by @xianml in https://github.com/bentoml/BentoML/pull/4153
fix(buildx): parsing attestation on docker desktop by @aarnphm in https://github.com/bentoml/BentoML/pull/4155

New Contributors

@xianml made their first contribution in https://github.com/bentoml/BentoML/pull/4153

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.3...v1.1.4

- Python
Published by aarnphm almost 3 years ago

bentoml - BentoML - v1.1.2

Patch releases

BentoML now provides a new diffusers integration, bentoml.diffusers_simple.

This introduces two integration for stable_diffusion and stable_diffusion_xl model.

```python import bentoml

Create a Runner for a Stable Diffusion model

runner = bentoml.diffuserssimple.stablediffusion.create_runner("CompVis/stable-diffusion-v1-4")

Create a Runner for a Stable Diffusion XL model

runnerxl = bentoml.diffuserssimple.stablediffusionxl.create_runner("stabilityai/stable-diffusion-xl-base-1.0") ```

General bug fixes and documentation improvement

What's Changed

docs: Add the Overview and Quickstarts sections by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4088
chore(type): makes ModelInfo mypy-compatible by @aarnphm in https://github.com/bentoml/BentoML/pull/4094
feat(store): update annotations by @aarnphm in https://github.com/bentoml/BentoML/pull/4092
docs: Fix some relative links by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4097
docs: Add the Iris quickstart doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4096
docs: Add the yolo quickstart by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4099
docs: Code format fix by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4101
fix: respect environment during bentoml.bentos.build by @aarnphm in https://github.com/bentoml/BentoML/pull/4081
docs: replaced deprecated save to save_model in pytorch.rst by @EgShes in https://github.com/bentoml/BentoML/pull/4102
fix: Make the install command shorter by @frostming in https://github.com/bentoml/BentoML/pull/4103
docs: Update the BentoCloud Build doc by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4104
docs: Add quickstart repo link and move torch import in Yolo by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4106
docs: fix typo by @zhangwm404 in https://github.com/bentoml/BentoML/pull/4108
docs: fix typo by @zhangwm404 in https://github.com/bentoml/BentoML/pull/4109
fix: calculate Pandas DataFrame batch size correctly by @judahrand in https://github.com/bentoml/BentoML/pull/4110
fix(cli): fix CLI output to BentoCloud by @Haivilo in https://github.com/bentoml/BentoML/pull/4114
Fix sklearn example docs by @jianshen92 in https://github.com/bentoml/BentoML/pull/4121
docs: Add the BentoCloud Deployment creation and update page property explanations by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4105
fix: disable pyright for being too strict by @frostming in https://github.com/bentoml/BentoML/pull/4113
refactor(cli): change prompt of cloud cli to unify Yatai and BentoCloud by @Haivilo in https://github.com/bentoml/BentoML/pull/4124
fix(cli): change model to lower case by @Haivilo in https://github.com/bentoml/BentoML/pull/4126
chore(ci): remove codestyle jobs by @aarnphm in https://github.com/bentoml/BentoML/pull/4125
fix: don't pass column names twice by @judahrand in https://github.com/bentoml/BentoML/pull/4120
feat: SSE (Experimental) by @jianshen92 in https://github.com/bentoml/BentoML/pull/4083
docs: Restructure the get started section in BentoCloud docs by @Sherlock113 in https://github.com/bentoml/BentoML/pull/4129
docs: change monitoring image by @Haivilo in https://github.com/bentoml/BentoML/pull/4133
feat: Rust gRPC client by @aarnphm in https://github.com/bentoml/BentoML/pull/3368
feature(framework): diffusers lora and textual inversion support by @larme in https://github.com/bentoml/BentoML/pull/4086
feat(buildx): support for attestation and sbom with buildx by @aarnphm in https://github.com/bentoml/BentoML/pull/4132

New Contributors

@EgShes made their first contribution in https://github.com/bentoml/BentoML/pull/4102
@zhangwm404 made their first contribution in https://github.com/bentoml/BentoML/pull/4108

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.1...v1.1.2

- Python
Published by aarnphm almost 3 years ago

bentoml - BentoML - v1.1.1

🍱 Patched release 1.1.1

Added more extensive cloud config option for bentoml deployment CLI, Thanks @Haivilo. Note that bentoml deployment update now takes the name as a optional positional argument instead of the previous behaviour --name: bash bentoml deployment update DEPLOYMENT_NAME See #4087
Added documentation about bento release GitHub action, Thanks @frostming. See #4071

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.1.0...v1.1.1

- Python
Published by aarnphm almost 3 years ago

bentoml - BentoML - v1.1.0

🍱 We're thrilled to announce the release of BentoML v1.1.0, our first minor version update since the milestone v1.0.

Backward Compatibility: Rest assured that this release maintains full API backward compatibility with v1.0.
Official gRPC Support: We've transitioned gRPC support in BentoML from experimental to official status, expanding your toolkit for high-performance, low-latency services.
Ray Integration: Ray is a popular open-source compute framework that makes it easy to scale Python workloads. BentoML integrates natively with Ray Serve to enable users to deploy Bento applications in a Ray cluster without modifying code or configuration.
Enhanced Hugging Face Transformers and Diffusers Support: All Hugging Face Diffuser models and pipelines can be seamlessly imported and integrated into BentoML applications through the Transformers and Diffusers framework libraries.
Enhanced Model Version Management: Enjoy greater flexibility with the improved model version management, enabling flexible configuration and synchronization of model versions with your remote model store.

🦾 We are also excited to announce the launch of OpenLLM v0.2.0 featuring the support of Llama 2 models.

GPU and CPU Support: Running Llama is support on both GPU and CPU.
Model variations and parameter sizes: Support all model weights and parameter sizes on Hugging Face.

bash meta-llama/llama-2-70b-chat-hf meta-llama/llama-2-13b-chat-hf meta-llama/llama-2-7b-chat-hf meta-llama/llama-2-70b-hf meta-llama/llama-2-13b-hf meta-llama/llama-2-7b-hf openlm-research/open_llama_7b_v2 openlm-research/open_llama_3b_v2 openlm-research/open_llama_13b huggyllama/llama-65b huggyllama/llama-30b huggyllama/llama-13b huggyllama/llama-7b

Users can use any weights on HuggingFace (e.g. TheBloke/Llama-2-13B-chat-GPTQ), custom weights from local path (e.g. /path/to/llama-1), or fine-tuned weights as long as it adheres to LlamaModelForCausalLM.
Stay tuned for Fine-tuning capabilities in OpenLLM: Fine-tuning various Llama 2 models will be added in a future release. Try the experimental script for fine-tuning Llama-2 with QLoRA under OpenLLM playground.

python -m openllm.playground.llama2_qlora --help

- Python
Published by ssheng almost 3 years ago

bentoml - BentoML - v1.0.22

🍱 BentoML v1.0.22 release has brought a list of well-anticipated updates.

Added support for Pydantic 2 for better validate performance.
Added support for CUDA 12 versions in builds and containerization.
Introduced service lifecycle events allowing adding custom logic on_deployment, on_startup, and on_shutdown. States can be managed using the context ctx variable during the on_startup and on_shutdown events and during request serving in the API.

```python @svc.ondeployment def ondeployment(): pass

@svc.onstartup def onstartup(ctx: bentoml.Context): ctx.state["objectkey"] = createobject()

@svc.onshutdown def onshutdown(ctx: bentoml.Context): cleanupstate(ctx.state["objectkey"])

@svc.api def predict(inputdata, ctx): object = ctx.state["objectkey"] pass ```
Added support for traffic control for both API Server and Runners. Timeout and maximum concurrency can now be configured through configuration.

```bash apiserver: traffic: timeout: 10 # API Server request timeout in seconds maxconcurrency: 32 # Maximum concurrency requests in the API Server

runners: iris: traffic: timeout: 10 # Runner request timeout in seconds max_concurrency: 32 # Maximum concurrency requests in the Runner ```
Improved performance of bentoml push performance for large Bentos.

🚀 One more thing, the team is delighted to unveil our latest endeavor, OpenLLM. This innovative project allows you to effortless build with the state-of-the-art open source or fine-tuned Large Language Models.

Supports all variants of Flan-T5, Dolly V2, StarCoder, Falcon, StableLM, and ChatGLM out-of-box. Fully customizable with model specific arguments.

bash openllm start [falcon | flan_t5 | dolly_v2 | chatglm | stablelm | starcoder]
Exposes the familiar BentoML APIs and transforms LLMs seamlessly into Runners.

bash llm_runner = openllm.Runner("dolly-v2")
Builds LLM application into the Bento format that can be deployed to BentoCloud or containerized into OCI images.

bash openllm build [falcon | flan_t5 | dolly_v2 | chatglm | stablelm | starcoder]

Our dedicated team is working hard to pioneering more integrations of advanced models for our upcoming releases of OpenLLM. Stay tuned for the unfolding developments.

- Python
Published by ssheng almost 3 years ago

bentoml - BentoML - v1.0.20

🍱 BentoML v1.0.20 is released with improved usability and compatibility features.

Production Mode by Default: bentoml serve command will now run with the --production option by default. The change is made the simulate the production behavior during development. The --reload option will continue to with as expected. To achieve the serving behavior previously, use --development instead.
Optional Dependency for OpenTelemetry Exporter: The opentelemetry-exporter-otlp-proto-http dependency has been moved from a required dependency to an optional one to address a protobuf dependency incompatibility issue. ⚠️ If you are currently using the Model Monitoring and Inference Data Collection feature, you must install the package with the monitor-otlp ****option from this release onwards to include the necessary dependency.

python pip install "bentoml[monitor-otlp]"
OpenTelemetry Trace ID Configuration Option: A new configuration option has been added to return the OpenTelemetry Trace ID in the response. This feature is particularly helpful when tracing has not been initialized in the upstream caller, but the caller still wishes to log the Trace ID in case of an error.

yaml api_server: http: response: trace_id: True

Start from a Service: Added the ability to start a server from a bentoml.Service object. This is helpful for troubleshooting a project in a development environment where no Bentos has been built yet.

```python import bentoml

# import the Service defined in /clip_api_service/service.py file from clipapiservice.service import svc

if name == "main": # start a server: server = bentoml.HTTPServer(svc) server.start(blocking=False) client = server.get_client() client.predict(..) ```

What's Changed

fix(dispatcher): handling empty ostat in `triggerrefresh` by @larme in https://github.com/bentoml/BentoML/pull/3796
fix(framework): adjust diffusers device_map default behavior by @larme in https://github.com/bentoml/BentoML/pull/3779
chore(dispatcher): cancel jobs with a for loop by @sauyon in https://github.com/bentoml/BentoML/pull/3788
fix: correctly reraise CancelledError by @sauyon in https://github.com/bentoml/BentoML/pull/3801
use path as resource for non-OS paths by @sauyon in https://github.com/bentoml/BentoML/pull/3800
chore(deps): bump coverage[toml] from 7.2.3 to 7.2.4 by @dependabot in https://github.com/bentoml/BentoML/pull/3803
feat: embedded runner by @larme in https://github.com/bentoml/BentoML/pull/3735
feat(tensorflow): support list types inputs by @enmanuelmag in https://github.com/bentoml/BentoML/pull/3807
chore(deps): bump ruff from 0.0.263 to 0.0.264 by @dependabot in https://github.com/bentoml/BentoML/pull/3817
feat: subprocess build by @aarnphm in https://github.com/bentoml/BentoML/pull/3814
docs: update community slack links by @parano in https://github.com/bentoml/BentoML/pull/3824
chore(deps): bump pyarrow from 11.0.0 to 12.0.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3820
chore(deps): remove imageio by @aarnphm in https://github.com/bentoml/BentoML/pull/3812
chore(deps): bump tritonclient[all] from 2.32.0 to 2.33.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3795
ci: add Pillow to tests dependencies by @aarnphm in https://github.com/bentoml/BentoML/pull/3830
feat(observability): support service.name by @aarnphm in https://github.com/bentoml/BentoML/pull/3825
feat: optional returning trace_id in response by @aarnphm in https://github.com/bentoml/BentoML/pull/3827
chore: 3.11 support by @PeterJCLaw in https://github.com/bentoml/BentoML/pull/3792
fix: Eliminate the exception during shutdown by @frostming in https://github.com/bentoml/BentoML/pull/3826
chore: expose schedulingstrategy in torunner by @bojiang in https://github.com/bentoml/BentoML/pull/3831
feat: allow starting server with bentoml.Service instance by @parano in https://github.com/bentoml/BentoML/pull/3829
chore(deps): bump bufbuild/buf-setup-action from 1.17.0 to 1.18.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3838
fix: make sure to set content-type for file type by @aarnphm in https://github.com/bentoml/BentoML/pull/3837
docs: update default docs to use env as key:value instead of list type by @aarnphm in https://github.com/bentoml/BentoML/pull/3841
deps: move exporter-proto to optional by @aarnphm in https://github.com/bentoml/BentoML/pull/3840
feat(server): improve server APIs by @aarnphm in https://github.com/bentoml/BentoML/pull/3834

New Contributors

@enmanuelmag made their first contribution in https://github.com/bentoml/BentoML/pull/3807
@PeterJCLaw made their first contribution in https://github.com/bentoml/BentoML/pull/3792

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.0.19...v1.0.20

- Python
Published by ssheng about 3 years ago

bentoml - BentoML - v1.0.19

🍱 BentoML v1.0.19 is released with enhanced GPU utilization and expanded ML framework support.

Optimized GPU resource utilization: Enabled scheduling of multiple instances of the same runner using the workers_per_resource scheduling strategy configuration. The following configuration allows scheduling 2 instances of the “iris” runner per GPU instance. workers_per_resource is 1 by default.

yaml runners: iris: resources: nvidia.com/gpu: 1 workers_per_resource: 2
New ML framework support: We've added support for EasyOCR and Detectron2 to our growing list of supported ML frameworks.
Enhanced runner communication: Implemented PEP 574 out-of-band pickling to improve runner communication by eliminating memory copying, resulting in better performance and efficiency.
Backward compatibility for Hugging Face Transformers: Resolved compatibility issues with Hugging Face Transformers versions prior to v4.18, ensuring a seamless experience for users with older versions.

⚙️ With the release of Kubeflow 1.7, BentoML now has native integration with Kubeflow, allowing developers to leverage BentoML's cloud-native components. Prior, developers were limited to exporting and deploying Bento as a single container. With this integration, models trained in Kubeflow can easily be packaged, containerized, and deployed to a Kubernetes cluster as microservices. This architecture enables the individual models to run in their own pods, utilizing the most optimal hardware for their respective tasks and enabling independent scaling.

💡 With each release, we consistently update our blog, documentation and examples to empower the community in harnessing the full potential of BentoML.

Learn more scheduling strategy to get better resource utilization.
Learn more about model monitoring and drift detection in BentoML and integration with various monitoring framework.
Learn more about using Nvidia Triton Inference Server as a runner to improve your application’s performance and throughput.

What's Changed

fix(env): using python -m to run pip commands by @frostming in https://github.com/bentoml/BentoML/pull/3762
chore(deps): bump pytest from 7.3.0 to 7.3.1 by @dependabot in https://github.com/bentoml/BentoML/pull/3766
feat: lazy load bentoml.server by @aarnphm in https://github.com/bentoml/BentoML/pull/3763
fix(client): service route prefix by @aarnphm in https://github.com/bentoml/BentoML/pull/3765
chore: add test with many requests by @sauyon in https://github.com/bentoml/BentoML/pull/3768
fix: using http config for grpc server by @aarnphm in https://github.com/bentoml/BentoML/pull/3771
feat: apply pep574 out-of-band pickling to DefaultContainer by @larme in https://github.com/bentoml/BentoML/pull/3736
fix: passing serve_cmd and passthrough kwargs by @aarnphm in https://github.com/bentoml/BentoML/pull/3764
feat: Detectron by @aarnphm in https://github.com/bentoml/BentoML/pull/3711
chore(dispatcher): (re-)factor out training code by @sauyon in https://github.com/bentoml/BentoML/pull/3767
feat: EasyOCR by @aarnphm in https://github.com/bentoml/BentoML/pull/3712
feat(build): support 3.11 by @aarnphm in https://github.com/bentoml/BentoML/pull/3774
patch: backports module availability for transformers<4.18 by @aarnphm in https://github.com/bentoml/BentoML/pull/3775
fix(dispatcher): set wait to 0 while training by @sauyon in https://github.com/bentoml/BentoML/pull/3664
chore(deps): bump ruff from 0.0.261 to 0.0.262 by @dependabot in https://github.com/bentoml/BentoML/pull/3778
feat: add model#load_model method by @parano in https://github.com/bentoml/BentoML/pull/3780
feat: Allow spawning more than 1 worker on each resource by @frostming in https://github.com/bentoml/BentoML/pull/3776
docs: Fix TensorFlow save_model parameter order by @ssheng in https://github.com/bentoml/BentoML/pull/3781
chore(deps): bump yamllint from 1.30.0 to 1.31.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3782
chore(deps): bump imageio from 2.27.0 to 2.28.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3783
chore(deps): bump ruff from 0.0.262 to 0.0.263 by @dependabot in https://github.com/bentoml/BentoML/pull/3790
fix: allow import service defined under a Python package by @parano in https://github.com/bentoml/BentoML/pull/3794

New Contributors

@frostming made their first contribution in https://github.com/bentoml/BentoML/pull/3762

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.0.18...v1.0.19

- Python
Published by ssheng about 3 years ago

bentoml - BentoML - v1.0.18

🍱 BentoML v1.0.18 brings a new way of creating the server and client natively from Python.

Start an HTTP or gRPC server and client asynchronously with a context manager.

```python server = HTTPServer("iris_classifier:latest", production=True, port=3000)

Start the server in a separate process and connect to it using a client

with server.start() as client: res = client.classify(np.array([[4.9, 3.0, 1.4, 0.2]])) ```
Start an HTTP or gRPC server synchronously.

python server = HTTPServer("iris_classifier:latest", production=True, port=3000) server.start(blocking=True)
As always, a client can be created and connected to an running server.

python client = Client.from_url("http://localhost:3000") res = client.classify(np.array([[4.9, 3.0, 1.4, 0.2]]))

What's Changed
chore(deps): bump coverage[toml] from 7.2.2 to 7.2.3 by @dependabot in https://github.com/bentoml/BentoML/pull/3746
bugs: Fix an f-string bug in Tranformers framework. by @ssheng in https://github.com/bentoml/BentoML/pull/3753
chore(deps): bump pytest from 7.2.2 to 7.3.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3751
chore(deps): bump bufbuild/buf-setup-action from 1.16.0 to 1.17.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3750
fix: BufferError when pushing model to BentoCloud by @aarnphm in https://github.com/bentoml/BentoML/pull/3737
chore: remove codecov dependencies by @aarnphm in https://github.com/bentoml/BentoML/pull/3754
feat: implement new serve API by @sauyon in https://github.com/bentoml/BentoML/pull/3696
examples: Add a client example to quickstart by @ssheng in https://github.com/bentoml/BentoML/pull/3752

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.0.17...v1.0.18

- Python
Published by ssheng about 3 years ago

bentoml - BentoML - v1.0.17

🍱 We are excited to announce the release of BentoML v1.0.17, which includes support for 🤗 Hugging Face Transformers pre-trained instances. Prior to this release, only pipelines could be saved and loaded using the bentoml.transformers APIs. However, based on the community's demand to work with pre-trained models, tokenizers, preprocessors, etc., without pipelines, we have expanded our capabilities in bentoml.transformers APIs. With this release, all pre-trained instances can be saved and loaded into either built-in Transformers framework runners or custom runners. This update opens up new possibilities for users to work with pre-trained models, and we are thrilled to see what the community will create using this feature. To learn more, visit BentoML Transformers framework documentation.

Pre-trained models and instances, such as tokenizers, preprocessors, and feature extractors, can also be saved as standalone models using the bentoml.transformers.save_model API.

```python import bentoml from transformers import AutoTokenizer

processor = SpeechT5Processor.frompretrained("microsoft/speecht5tts") model = SpeechT5ForTextToSpeech.frompretrained("microsoft/speecht5tts") vocoder = SpeechT5HifiGan.frompretrained("microsoft/speecht5hifigan")

bentoml.transformers.savemodel("speecht5ttsprocessor", processor) bentoml.transformers.savemodel("speecht5ttsmodel", model, signatures={"generatespeech": {"batchable": False}}) bentoml.transformers.savemodel("speecht5ttsvocoder", vocoder) ```
Pre-trained models and instances can be run either independently as Transformers framework runners or jointly in a custom runner. To use pre-trained models and instances as individual framework runners, simply get the models reference and convert them to runners using the to_runner method.

```python import bentoml import torch

from bentoml.io import Text, NumpyNdarray from datasets import load_dataset

proccessorrunner = bentoml.transformers.get("speecht5ttsprocessor").torunner() modelrunner = bentoml.transformers.get("speecht5ttsmodel").torunner() vocoderrunner = bentoml.transformers.get("speecht5ttsvocoder").torunner() embeddingsdataset = loaddataset("Matthijs/cmu-arctic-xvectors", split="validation") speakerembeddings = torch.tensor(embeddingsdataset[7306]["xvector"]).unsqueeze(0)

svc = bentoml.Service("text2speech", runners=[proccessorrunner, modelrunner, vocoder_runner])

@svc.api(input=Text(), output=NumpyNdarray()) def generatespeech(inp: str): inputs = proccessorrunner.run(text=inp, returntensors="pt") speech = modelrunner.generatespeech.run(inputids=inputs["inputids"], speakerembeddings=speakerembeddings, vocoder=vocoderrunner.run) return speech.numpy() ```
To use the pre-trained models and instances together in a custom runner, use the bentoml.transformers.get API to get the models references and load them in a custom runner. The pretrained instances can then be used for inference in the custom runner.

```python import bentoml import torch

from datasets import load_dataset

processorref = bentoml.models.get("speecht5ttsprocessor:latest") modelref = bentoml.models.get("speecht5ttsmodel:latest") vocoderref = bentoml.models.get("speecht5tts_vocoder:latest")

class SpeechT5Runnable(bentoml.Runnable):
```
def __init__(self):
    self.processor = bentoml.transformers.load_model(processor_ref)
    self.model = bentoml.transformers.load_model(model_ref)
    self.vocoder = bentoml.transformers.load_model(vocoder_ref)
    self.embeddings_dataset = load_dataset("Matthijs/cmu-arctic-xvectors", split="validation")
    self.speaker_embeddings = torch.tensor(self.embeddings_dataset[7306]["xvector"]).unsqueeze(0)

@bentoml.Runnable.method(batchable=False)
def generate_speech(self, inp: str):
    inputs = self.processor(text=inp, return_tensors="pt")
    speech = self.model.generate_speech(inputs["input_ids"], self.speaker_embeddings, vocoder=self.vocoder)
    return speech.numpy()
```
text2speechrunner = bentoml.Runner(SpeechT5Runnable, name="speecht5runner", models=[processorref, modelref, vocoderref]) svc = bentoml.Service("talkgpt", runners=[text2speech_runner])

@svc.api(input=bentoml.io.Text(), output=bentoml.io.NumpyNdarray()) async def generatespeech(inp: str): return await text2speechrunner.generatespeech.asyncrun(inp) ```

What's Changed

feat(containerize): caching pip/conda installation layers by @smidm in https://github.com/bentoml/BentoML/pull/3673
docs(batching): update docs to 503 by @sauyon in https://github.com/bentoml/BentoML/pull/3677
chore(deps): bump ruff from 0.0.255 to 0.0.256 by @dependabot in https://github.com/bentoml/BentoML/pull/3676
fix(type): annotate PdSeries with pandas-stubs by @aarnphm in https://github.com/bentoml/BentoML/pull/3466
chore(dispatcher): refactor out training code by @sauyon in https://github.com/bentoml/BentoML/pull/3663
fix: makes containerize for triton examples to all amd64 by @aarnphm in https://github.com/bentoml/BentoML/pull/3678
chore(deps): bump coverage[toml] from 7.2.1 to 7.2.2 by @dependabot in https://github.com/bentoml/BentoML/pull/3679
revert: "chore(dispatcher): refactor out training code (#3663)" by @sauyon in https://github.com/bentoml/BentoML/pull/3680
doc: add more links to Bentoml/examples by @larme in https://github.com/bentoml/BentoML/pull/3631
perf: serialization optimization by @larme in https://github.com/bentoml/BentoML/pull/3606
examples: Kubeflow by @ssheng in https://github.com/bentoml/BentoML/pull/3656
chore(deps): bump pytest-asyncio from 0.20.3 to 0.21.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3688
chore(deps): bump ruff from 0.0.256 to 0.0.257 by @dependabot in https://github.com/bentoml/BentoML/pull/3689
chore(deps): bump imageio from 2.26.0 to 2.26.1 by @dependabot in https://github.com/bentoml/BentoML/pull/3690
chore(deps): bump yamllint from 1.29.0 to 1.30.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3694
fix: remove duplicate dependabot check for pip by @aarnphm in https://github.com/bentoml/BentoML/pull/3691
chore(deps): bump ruff from 0.0.257 to 0.0.258 by @dependabot in https://github.com/bentoml/BentoML/pull/3699
docs: Update the Kubeflow example by @ssheng in https://github.com/bentoml/BentoML/pull/3703
chore(deps): bump ruff from 0.0.258 to 0.0.259 by @dependabot in https://github.com/bentoml/BentoML/pull/3709
docs: add link to pyfilesystem plugins by @sauyon in https://github.com/bentoml/BentoML/pull/3716
docs: Kubeflow integration documentation by @ssheng in https://github.com/bentoml/BentoML/pull/3704
docs: replace loadrunner() to get().torunner() by @KimSoungRyoul in https://github.com/bentoml/BentoML/pull/3715
chore(deps): bump imageio from 2.26.1 to 2.27.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3720
fix(readme): format markdown table by @aarnphm in https://github.com/bentoml/BentoML/pull/3722
fix: copy files before running setup_script by @aarnphm in https://github.com/bentoml/BentoML/pull/3713
chore: remove experimental warning for bentoml.metrics by @aarnphm in https://github.com/bentoml/BentoML/pull/3725
ci: temporary disable coverage by @aarnphm in https://github.com/bentoml/BentoML/pull/3726
chore(deps): bump ruff from 0.0.259 to 0.0.260 by @dependabot in https://github.com/bentoml/BentoML/pull/3734
chore(deps): bump tritonclient[all] from 2.31.0 to 2.32.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3730
fix(type): bentoml.container.build should accept multiple image_tag by @pmayd in https://github.com/bentoml/BentoML/pull/3719
chore(deps): bump bufbuild/buf-setup-action from 1.15.1 to 1.16.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3738
feat: add query params to request context by @sauyon in https://github.com/bentoml/BentoML/pull/3717
chore(dispatcher): use attr class instead of a tuple by @sauyon in https://github.com/bentoml/BentoML/pull/3731
fix: Make it so the configured maxbatchsize is respected when batching inference requests together by @RShang97 in https://github.com/bentoml/BentoML/pull/3741
feat(transformers): pretrained protocol support by @aarnphm in https://github.com/bentoml/BentoML/pull/3684
fix(tests): broken CI by @aarnphm in https://github.com/bentoml/BentoML/pull/3742
chore(deps): bump ruff from 0.0.260 to 0.0.261 by @dependabot in https://github.com/bentoml/BentoML/pull/3744
docs: Transformers documentation on pre-trained instances support by @ssheng in https://github.com/bentoml/BentoML/pull/3745

New Contributors

@smidm made their first contribution in https://github.com/bentoml/BentoML/pull/3673
@pmayd made their first contribution in https://github.com/bentoml/BentoML/pull/3719
@RShang97 made their first contribution in https://github.com/bentoml/BentoML/pull/3741

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.0.16...v1.0.17

- Python
Published by ssheng about 3 years ago

bentoml - BentoML - v1.0.16

🍱 BentoML v1.0.16 release is here featuring the introduction of the bentoml.triton framework. With this integration, BentoML now supports running NVIDIA Triton Inference Server as a Runner. See Triton Inference Server documentation to learn more!

Triton Inference Server can be configured as a Runner in BentoML with its model repository and CLI arguments specified as parameters.

```python import bentoml

tritonrunner = bentoml.triton.Runner( "tritonrunner", modelrepository="s3://bucket/path/to/modelrepository", cliargs=["--load-model=torchscripyolov5s", "--model-control-mode=explicit"], ) ```
Models served by the Triton Inference Server Runner can be called as a method on the runner handle both synchronously and asynchronously.

python @svc.api( input=bentoml.io.Image.from_sample("./data/0.png"), output=bentoml.io.NumpyNdarray() ) async def bentoml_torchscript_mnist_infer(im: Image) -> NDArray[t.Any]: arr = np.array(im) / 255.0 arr = np.expand_dims(arr, (0, 1)).astype("float32") InferResult = await triton_runner.torchscript_mnist.async_run(arr) return InferResult.as_numpy("OUTPUT__0")
Build bentos and containerize images with Triton Runners by specifying nvcr.io/nvidia/tritonserver base image in bentofile.yaml.

```yaml service: service:svc include:
- /model_repository
- /data/*.png
- /*.py exclude:
- /pycache
- /venv
- /train.py
- /build_bento.py
- /containerize_bento.py python: packages:
  - bentoml[triton] docker: base_image: nvcr.io/nvidia/tritonserver:22.12-py3 ```

💡 If you are an existing Triton user, the integration provides simpler ways to add custom logics in Python, deploy distributed multi-model inference graph, unify model management across different ML frameworks and workflows, and standardize model packaging format with versioning and collaboration features. If you are an existing BentoML user, the integration improves the runner efficiency and throughput under high load thanks to Triton’s efficient C++ runtime.

What's Changed

fix(container): podman virtual machine healthcheck (#3575) by @timc in https://github.com/bentoml/BentoML/pull/3576
chore(aiohttp): remove deprecated verify_ssl to ssl by @aarnphm in https://github.com/bentoml/BentoML/pull/3574
feat(triton): support HTTP client by @aarnphm in https://github.com/bentoml/BentoML/pull/3502
fix(grpc): handle backward protocol version by @aarnphm in https://github.com/bentoml/BentoML/pull/3332
chore(deps): bump ruff from 0.0.246 to 0.0.247 by @dependabot in https://github.com/bentoml/BentoML/pull/3579
chore(test): using container API for testing by @aarnphm in https://github.com/bentoml/BentoML/pull/3582
fix(serve-cli): Make sure to use BENTOML_CONFIG value by @aarnphm in https://github.com/bentoml/BentoML/pull/3597
docs: Update documentation with an examples link by @ssheng in https://github.com/bentoml/BentoML/pull/3599
chore: lock starlette version by @sauyon in https://github.com/bentoml/BentoML/pull/3600
feature(diffusers): support enable_attention_slicing by @larme in https://github.com/bentoml/BentoML/pull/3598
chore(cli): figlet to show on CLI only by @aarnphm in https://github.com/bentoml/BentoML/pull/3603
chore(cli): using default background as color by @aarnphm in https://github.com/bentoml/BentoML/pull/3608
feat: Flax by @aarnphm in https://github.com/bentoml/BentoML/pull/3123
feat(gRPC): client implementation by @aarnphm in https://github.com/bentoml/BentoML/pull/3280
fix: invalid option dtype=True for pd.read_csv by @parano in https://github.com/bentoml/BentoML/pull/3601
chore(deps): bump coverage[toml] from 7.1.0 to 7.2.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3616
chore(deps): bump ruff from 0.0.247 to 0.0.252 by @dependabot in https://github.com/bentoml/BentoML/pull/3617
docs: containerisation API by @aarnphm in https://github.com/bentoml/BentoML/pull/3518
chore(deps): bump coverage[toml] from 7.2.0 to 7.2.1 by @dependabot in https://github.com/bentoml/BentoML/pull/3621
chore(deps): bump imageio from 2.25.1 to 2.26.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3620
fix(docs): missing space bug causes table not to render by @aarnphm in https://github.com/bentoml/BentoML/pull/3622
chore(deps): bump ruff from 0.0.252 to 0.0.253 by @dependabot in https://github.com/bentoml/BentoML/pull/3624
feat: enable cork for non-batched workloads by @sauyon in https://github.com/bentoml/BentoML/pull/3602
docs: Fix typo in concepts/service by @FelixSchuSi in https://github.com/bentoml/BentoML/pull/3627
chore(deps): bump tritonclient[all] from 2.30.0 to 2.31.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3628
fix(docs): broken inline docstring by @aarnphm in https://github.com/bentoml/BentoML/pull/3538
fix: use a semaphore to limit runner connections by @sauyon in https://github.com/bentoml/BentoML/pull/3607
fix: make inference_api handle None type by @aarnphm in https://github.com/bentoml/BentoML/pull/3611
fix: make sure not to override user set values for from_sample by @aarnphm in https://github.com/bentoml/BentoML/pull/3610
docs: add exceptions API section by @aarnphm in https://github.com/bentoml/BentoML/pull/3609
revert(pyproject): add back pytest plugins by @aarnphm in https://github.com/bentoml/BentoML/pull/3633
fix(configuration): CORS docs, allow_origins and allow_headers by @larme in https://github.com/bentoml/BentoML/pull/3643
chore(deps): bump ruff from 0.0.253 to 0.0.254 by @dependabot in https://github.com/bentoml/BentoML/pull/3641
chore(deps): bump pytest from 7.2.1 to 7.2.2 by @dependabot in https://github.com/bentoml/BentoML/pull/3642
chore: http client healthcheck by @denyszhak in https://github.com/bentoml/BentoML/pull/3636
docs: typo in configuration.rst by @davkime in https://github.com/bentoml/BentoML/pull/3644
docs: correct links to configuration source code by @davkime in https://github.com/bentoml/BentoML/pull/3645
example: add fraud detection and benchmark examples by @parano in https://github.com/bentoml/BentoML/pull/3647
fix(containerize): remove autoconfig for buildctl by @aarnphm in https://github.com/bentoml/BentoML/pull/3484
feat: name in bentofile.yaml by @aarnphm in https://github.com/bentoml/BentoML/pull/3604
chore: ensure all labels are dict[str,str] by @aarnphm in https://github.com/bentoml/BentoML/pull/3605
fix(triton): enable runtime options by @aarnphm in https://github.com/bentoml/BentoML/pull/3649
docs: Triton Inference Server by @aarnphm in https://github.com/bentoml/BentoML/pull/3519
example: Triton Inference Server by @aarnphm in https://github.com/bentoml/BentoML/pull/3471
chore(deps): bump pytest from 7.2.1 to 7.2.2 in /requirements by @dependabot in https://github.com/bentoml/BentoML/pull/3639
chore(deps): bump bufbuild/buf-setup-action from 1.14.0 to 1.15.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3638
fix: some missing logics for triton examples by @aarnphm in https://github.com/bentoml/BentoML/pull/3650
fix: use async implementation by @characat0 in https://github.com/bentoml/BentoML/pull/3654
feat: add ray deploy support by @parano in https://github.com/bentoml/BentoML/pull/3632
chore(deps): bump pytest-xdist[psutil] from 3.2.0 to 3.2.1 by @dependabot in https://github.com/bentoml/BentoML/pull/3659
chore(deps): bump bufbuild/buf-setup-action from 1.15.0 to 1.15.1 by @dependabot in https://github.com/bentoml/BentoML/pull/3655
fix: update scheme logic using ssl.enabled by @aarnphm in https://github.com/bentoml/BentoML/pull/3660
feat: from_sample docstring by @aarnphm in https://github.com/bentoml/BentoML/pull/3318
fix(ci): locking starlette for container tests by @aarnphm in https://github.com/bentoml/BentoML/pull/3666
chore: better exception for numpy by @sauyon in https://github.com/bentoml/BentoML/pull/3665
feat: make file io descriptor allow any mime type by default by @sauyon in https://github.com/bentoml/BentoML/pull/3626
fix(docs): broken link by @aarnphm in https://github.com/bentoml/BentoML/pull/3537
chore(stubs): remove unused by @aarnphm in https://github.com/bentoml/BentoML/pull/3612
docs: Update Triton documentation and examples by @ssheng in https://github.com/bentoml/BentoML/pull/3668
chore(deps): bump ruff from 0.0.254 to 0.0.255 by @dependabot in https://github.com/bentoml/BentoML/pull/3671
docs: Update integration docs by @ssheng in https://github.com/bentoml/BentoML/pull/3672

New Contributors

@FelixSchuSi made their first contribution in https://github.com/bentoml/BentoML/pull/3627
@denyszhak made their first contribution in https://github.com/bentoml/BentoML/pull/3636
@davkime made their first contribution in https://github.com/bentoml/BentoML/pull/3644

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.0.15...v1.0.16

- Python
Published by ssheng about 3 years ago

bentoml - BentoML - v1.0.15

🍱 BentoML v1.0.15 release is here featuring the introduction of the bentoml.diffusers framework.

Learn more about the capabilities of the bentoml.diffusers framework in the Creating Stable Diffusion 2.0 Service With BentoML And Diffusers blog and BentoML Diffusers example project.
Import a diffusion model with the bentoml.diffusers.import_model API.

```python import bentoml

bentoml.diffusers.import_model( "sd2", "stabilityai/stable-diffusion-2", ) ```
Create a text2img service using a Stable Diffusion 2.0 model runner with the familiar to_runner API from the bentoml.diffuser framework.

```python import torch from diffusers import StableDiffusionPipeline

import bentoml from bentoml.io import Image, JSON, Multipart

bentomodel = bentoml.diffusers.get("sd2:latest") stablediffusionrunner = bentomodel.to_runner()

svc = bentoml.Service("stablediffusionv2", runners=[stablediffusionrunner])

@svc.api(input=JSON(), output=Image()) def txt2img(inputdata): images, _ = stablediffusionrunner.run(**inputdata) return images[0] ```

🍱 Fixed a incompatibility change introduced in starlette==0.25.0 result in the type MultiPartMessage not being found in starlette.formparsers. ImportError: cannot import name 'MultiPartMessage' from 'starlette.formparsers' (/opt/miniconda3/envs/bentoml/lib/python3.10/site-packages/starlette/formparsers.py)

What's Changed

chore(deps): bump pytest-xdist[psutil] from 3.1.0 to 3.2.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3536
fix: include dockerfile_template to Bento for containerize by @aarnphm in https://github.com/bentoml/BentoML/pull/3501
chore: add missing logger and fix types by @aarnphm in https://github.com/bentoml/BentoML/pull/3453
chore(rtd): disable epub and pdf as format by @aarnphm in https://github.com/bentoml/BentoML/pull/3544
feat(torchscript): support _extra_files by @aarnphm in https://github.com/bentoml/BentoML/pull/3480
refactor(ci): make sure to run types on py,pyi files by @aarnphm in https://github.com/bentoml/BentoML/pull/3545
fix(server): deprecate client and cache get_client by @aarnphm in https://github.com/bentoml/BentoML/pull/3547
chore(serve): update options for triton_options by @aarnphm in https://github.com/bentoml/BentoML/pull/3503
tools(linter): Ruff by @aarnphm in https://github.com/bentoml/BentoML/pull/3539
chore(deps): bump ruff from 0.0.243 to 0.0.244 by @dependabot in https://github.com/bentoml/BentoML/pull/3548
chore(type): remove cattr type ignore by @aarnphm in https://github.com/bentoml/BentoML/pull/3550
chore: bumping otlp deps to 1.15 by @aarnphm in https://github.com/bentoml/BentoML/pull/3351
docs: Add an example index by @ssheng in https://github.com/bentoml/BentoML/pull/3551
revert: "chore: bumping otlp deps to 1.15" by @bojiang in https://github.com/bentoml/BentoML/pull/3553
chore(deps): bump bufbuild/buf-setup-action from 1.13.1 to 1.14.0 by @dependabot in https://github.com/bentoml/BentoML/pull/3554
chore(deps): bump ruff from 0.0.244 to 0.0.246 by @dependabot in https://github.com/bentoml/BentoML/pull/3559
chore(deps): bump imageio from 2.25.0 to 2.25.1 by @dependabot in https://github.com/bentoml/BentoML/pull/3557
chore: update README.md by @timliubentoml in https://github.com/bentoml/BentoML/pull/3565
feat(containerization): support 11.7 by @aarnphm in https://github.com/bentoml/BentoML/pull/3567
chore: remove deprecation warning when building bentos by @CheeksTheGeek in https://github.com/bentoml/BentoML/pull/3566
feature(framework): diffusers by @larme in https://github.com/bentoml/BentoML/pull/3534
fix: update formparser for new starlette by @sauyon in https://github.com/bentoml/BentoML/pull/3569

New Contributors

@CheeksTheGeek made their first contribution in https://github.com/bentoml/BentoML/pull/3566

Full Changelog: https://github.com/bentoml/BentoML/compare/v1.0.14...v1.0.15

- Python
Published by ssheng over 3 years ago

bentoml - BentoML - v1.0.14

🍱 Fixed the backward incompatibility introduced in starlette version 0.24.0. Upgrade BentoML to v1.0.14 if you encounter the error related to content_type like below.

Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/server/service_app.py", line 305, in api_func input_data = await api.input.from_http_request(request) File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/io_descriptors/multipart.py", line 208, in from_http_request reqs = await populate_multipart_requests(request) File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/utils/formparser.py", line 188, in populate_multipart_requests form = await multipart_parser.parse() File "/usr/local/lib/python3.8/dist-packages/bentoml/_internal/utils/formparser.py", line 158, in parse multipart_file = UploadFile( TypeError: __init__() got an unexpected keyword argument 'content_type'

- Python
Published by ssheng over 3 years ago

Recent Releases of bentoml

bentoml - v1.4.22

What's Changed

bentoml - v1.4.21

What's Changed

bentoml - v1.4.20

What's Changed

bentoml - v1.4.19

What's Changed

New Contributors

bentoml - v1.4.18

What's Changed

bentoml - v1.4.17

What's Changed

bentoml - v1.4.16

What's Changed

bentoml - v1.4.15

What's Changed

bentoml - v1.4.14

What's Changed

New Contributors

bentoml - v1.4.13

What's Changed

bentoml - v1.4.12

What's Changed

bentoml - v1.4.10

What's Changed

New Contributors

bentoml - v1.4.8

What's Changed

bentoml - v1.4.7

What's Changed

bentoml - v1.4.6

What's Changed

New Contributors

bentoml - v1.4.5

What's Changed

bentoml - v1.4.4

What's Changed

New Contributors

bentoml - v1.4.3

What's Changed

bentoml - v1.4.2

What's Changed

New Contributors

bentoml - v1.4.1

What's Changed

bentoml - v1.4.0

🚀 20x faster iteration with Codespaces

🐍 New Python SDK for runtime configurations

⚡ Accelerated model loading with safetensors

🌍 External deployment dependencies

⚠️ Legacy Service API deprecation

What's Changed

bentoml - v1.4.0a2

What's Changed

bentoml - v1.4.0a1

What's Changed

bentoml - v1.3.22

What's Changed

bentoml - v1.3.21

What's Changed

New Contributors

bentoml - v1.3.20

What's Changed

New Contributors

bentoml - v1.3.19

What's Changed

bentoml - v1.3.18

What's Changed

bentoml - v1.3.17

What's Changed

bentoml - v1.3.16

What's Changed

bentoml - v1.3.15

What's Changed

New Contributors

bentoml - v1.3.14

What's Changed

New Contributors