flaml

A fast library for AutoML and tuning. Join our Discord: https://discord.gg/Cppx2vSPVP.

Science Score: 64.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org
✓
Committers with academic emails
3 of 96 committers (3.1%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (14.6%) to scientific vocabulary

Keywords

automated-machine-learning automl classification data-science deep-learning finetuning hyperparam hyperparameter-optimization jupyter-notebook machine-learning natural-language-generation natural-language-processing python random-forest regression scikit-learn tabular-data timeseries-forecasting tuning

Keywords from Contributors

agents langchain observability mlops model-management mlflow llmops llm-evaluation ai-governance agentops

Last synced: 6 months ago · JSON representation ·

Repository

A fast library for AutoML and tuning. Join our Discord: https://discord.gg/Cppx2vSPVP.

Basic Info

Host: GitHub
Owner: microsoft
License: mit
Language: Jupyter Notebook
Default Branch: main
Homepage: https://microsoft.github.io/FLAML/
Size: 209 MB

Statistics

Stars: 4,199
Watchers: 58
Forks: 542
Open Issues: 231
Releases: 77

Topics

Created over 5 years ago · Last pushed 6 months ago

Metadata Files

Readme License Code of conduct Citation Security

README.md

Conda version

A Fast Library for Automated Machine Learning & Tuning

:fire: FLAML supports AutoML and Hyperparameter Tuning in Microsoft Fabric Data Science. In addition, we've introduced Python 3.11 support, along with a range of new estimators, and comprehensive integration with MLflow—thanks to contributions from the Microsoft Fabric product team.

:fire: Heads-up: We have migrated AutoGen into a dedicated github repository. Alongside this move, we have also launched a dedicated Discord server and a website for comprehensive documentation.

:fire: The automated multi-agent chat framework in AutoGen is in preview from v2.0.0.

:fire: FLAML is highlighted in OpenAI's cookbook.

:fire: autogen is released with support for ChatGPT and GPT-4, based on Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference.

What is FLAML

FLAML is a lightweight Python library for efficient automation of machine learning and AI operations. It automates workflow based on large language models, machine learning models, etc. and optimizes their performance.

FLAML enables building next-gen GPT-X applications based on multi-agent conversations with minimal effort. It simplifies the orchestration, automation and optimization of a complex GPT-X workflow. It maximizes the performance of GPT-X models and augments their weakness.
For common machine learning tasks like classification and regression, it quickly finds quality models for user-provided data with low computational resources. It is easy to customize or extend. Users can find their desired customizability from a smooth range.
It supports fast and economical automatic tuning (e.g., inference hyperparameters for foundation models, configurations in MLOps/LMOps workflows, pipelines, mathematical/statistical models, algorithms, computing experiments, software configurations), capable of handling large search space with heterogeneous evaluation cost and complex constraints/guidance/early stopping.

FLAML is powered by a series of research studies from Microsoft Research and collaborators such as Penn State University, Stevens Institute of Technology, University of Washington, and University of Waterloo.

FLAML has a .NET implementation in ML.NET, an open-source, cross-platform machine learning framework for .NET.

Installation

FLAML requires Python version >= 3.9. It can be installed from pip:

bash pip install flaml

Minimal dependencies are installed without extra options. You can install extra options based on the feature you need. For example, use the following to install the dependencies needed by the autogen package.

bash pip install "flaml[autogen]"

Find more options in Installation. Each of the notebook examples may require a specific option to be installed.

Quickstart

(New) The autogen package enables the next-gen GPT-X applications with a generic multi-agent conversation framework. It offers customizable and conversable agents which integrate LLMs, tools and human. By automating chat among multiple capable agents, one can easily make them collectively perform tasks autonomously or with human feedback, including tasks that require using tools via code. For example,

```python from flaml import autogen

assistant = autogen.AssistantAgent("assistant") userproxy = autogen.UserProxyAgent("userproxy") userproxy.initiatechat( assistant, message="Show me the YTD gain of 10 largest technology companies as of today.", )

This initiates an automated chat between the two agents to solve the task

```

Autogen also helps maximize the utility out of the expensive LLMs such as ChatGPT and GPT-4. It offers a drop-in replacement of openai.Completion or openai.ChatCompletion with powerful functionalites like tuning, caching, templating, filtering. For example, you can optimize generations by LLM with your own tuning data, success metrics and budgets.

```python

perform tuning

config, analysis = autogen.Completion.tune( data=tunedata, metric="success", mode="max", evalfunc=evalfunc, inferencebudget=0.05, optimizationbudget=3, numsamples=-1, )

perform inference for a test instance

response = autogen.Completion.create(context=test_instance, **config) ```

With three lines of code, you can start using this economical and fast AutoML engine as a scikit-learn style estimator.

```python from flaml import AutoML

automl = AutoML() automl.fit(Xtrain, ytrain, task="classification") ```

You can restrict the learners and use FLAML as a fast hyperparameter tuning tool for XGBoost, LightGBM, Random Forest etc. or a customized learner.

python automl.fit(X_train, y_train, task="classification", estimator_list=["lgbm"])

You can also run generic hyperparameter tuning for a custom function.

python from flaml import tune tune.run(evaluation_function, config={…}, low_cost_partial_config={…}, time_budget_s=3600)

Zero-shot AutoML allows using the existing training API from lightgbm, xgboost etc. while getting the benefit of AutoML in choosing high-performance hyperparameter configurations per task.

```python from flaml.default import LGBMRegressor

Use LGBMRegressor in the same way as you use lightgbm.LGBMRegressor.

estimator = LGBMRegressor()

The hyperparameters are automatically set according to the training data.

estimator.fit(Xtrain, ytrain) ```

Documentation

You can find a detailed documentation about FLAML here.

In addition, you can find:

Research and blogposts around FLAML.
Discord.
Contributing guide.
ML.NET documentation and tutorials for Model Builder, ML.NET CLI, and AutoML API.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

If you are new to GitHub here is a detailed help source on getting involved with development on GitHub.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Contributors Wall

Owner

Name: Microsoft
Login: microsoft
Kind: organization
Email: opensource@microsoft.com
Location: Redmond, WA

Website: https://opensource.microsoft.com
Twitter: OpenAtMicrosoft
Repositories: 7,257
Profile: https://github.com/microsoft

Open source projects and samples from Microsoft

Citation (CITATION.cff)

preferred-citation:
  type: inproceedings
  authors:
  - family-names: "Wang"
    given-names: "Chi"
    affiliation: "Microsoft Research, Redmond WA USA"
  - family-names: "Wu"
    given-names: "Qingyun"
    affiliation: "Microsoft Research, Redmond WA USA"
  - family-names: "Weimer"
    given-names: "Markus"
    affiliation: "Microsoft Corporation, Redmond WA USA"
  - family-names: "Zhu"
    given-names: "Eric"
    affiliation: "Microsoft Research, Redmond WA USA"
  booktitle: "Proceedings of the 4th MLSys Conference"
  title: "FLAML: A Fast and Lightweight AutoML Library"
  year: 2021

GitHub Events

Total

Create event: 33
Release event: 7
Issues event: 67
Watch event: 304
Delete event: 30
Member event: 2
Issue comment event: 130
Push event: 120
Pull request event: 95
Pull request review comment event: 31
Pull request review event: 87
Fork event: 41

Last Year

Create event: 33
Release event: 7
Issues event: 67
Watch event: 304
Delete event: 30
Member event: 2
Issue comment event: 130
Push event: 120
Pull request event: 95
Pull request review comment event: 31
Pull request review event: 87
Fork event: 41

Committers

Last synced: 9 months ago

All Time

Total Commits: 706
Total Committers: 96
Avg Commits per committer: 7.354
Development Distribution Score (DDS): 0.659

Past Year

Commits: 71
Committers: 17
Avg Commits per committer: 4.176
Development Distribution Score (DDS): 0.62

Top Committers

Name	Email	Commits
Chi Wang	w**i@m**m	241
Xueqing Liu	l****9	72
Qingyun Wu	q**y@v**u	53
Li Jiang	b**i@g**m	47
skzhang1	s**9@g**m	41
dependabot[bot]	4****]	32
Anonymous-submission-repo	h**1@1**m	29
Chi Wang (MSR)	c**w@m**m	20
Jirka Borovec	6****a	11
Kevin Chen	7****s	11
Daniel Grindrod	d**6@g**m	9
Yiran Wu	3****a	8
Sayan Roy	s**y@g**m	6
Jing Dong	3****0	5
Zvi Baratz	z**z@g**m	4
Antoni Baum	a**m@p**m	4
Andrea W	a**i@a**m	4
Gian Pio Domiziani	5****i	3
Gian Pio Domiziani	g**i@g**m	3
EgorKraevTransferwise	6****e	3
levscaut	5****t	3
vvijayalakshmi21	3****1	3
Luis Quintanilla	4****a	2
Michal	m**k@h**m	2
Prajwal Borkar	4****r	2
murunlin	m**n@b**n	2
jmrichardson	j****n	2
Yang, Bo	p**y@g**m	2
Vijaya Lakshmi Venkatraman	v**n@s**m	2
Rui Zhuang	7****w	2
and 66 more...

Committer Domains (Top 20 + Academic)

qq.com: 2 microsoft.com: 2 yandex.ru: 1 stevens.edu: 1 unitary.ai: 1 gitter.im: 1 smartcoin.co.in: 1 socgen.com: 1 bupt.edu.cn: 1 heidelbergcement.com: 1 ammagamma.com: 1 163.com: 1 virginia.edu: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 208
Total pull requests: 286
Average time to close issues: 2 months
Average time to close pull requests: 17 days
Total issue authors: 107
Total pull request authors: 57
Average comments per issue: 2.37
Average comments per pull request: 1.19
Merged pull requests: 207
Bot issues: 0
Bot pull requests: 29

Past Year

Issues: 44
Pull requests: 76
Average time to close issues: 16 days
Average time to close pull requests: 3 days
Issue authors: 25
Pull request authors: 14
Average comments per issue: 1.5
Average comments per pull request: 1.0
Merged pull requests: 59
Bot issues: 0
Bot pull requests: 17

View more stats

Top Authors

Issue Authors

sonichi (30)
dannycg1996 (11)
bbudescu (10)
thinkall (10)
Sandy4321 (8)
qingyun-wu (7)
lizhuoq (4)
Programmer-RD-AI (3)
harupy (3)
tsoernes (3)
flippercy (3)
kevin666aa (3)
lucazav (3)
yuhorun (3)
azachar (2)

Pull Request Authors

thinkall (77)
sonichi (49)
dependabot[bot] (40)
dannycg1996 (16)
kevin666aa (15)
qingyun-wu (14)
Programmer-RD-AI (14)
Borda (12)
liususan091219 (7)
skzhang1 (7)
murunlin (6)
Stickic-cyber (4)
Atry (4)
Animaholic (4)
dhruvthakur2000 (4)

Top Labels

Issue Labels

enhancement (26) bug (26) good first issue (17) question (13) documentation (11) help wanted (8) communication (2) dependencies (2) wontfix (1) need more info (1)

Pull Request Labels

dependencies (43) enhancement (20) documentation (16) javascript (11) bug (7) communication (2) question (2) need more info (1)

Packages

Total packages: 4
Total downloads:
- pypi 372,597 last-month
Total docker downloads: 21,948,847

Total dependent packages: 15
(may contain duplicates)
Total dependent repositories: 389
(may contain duplicates)
Total versions: 276
Total maintainers: 1

pypi.org: flaml

A fast library for automated machine learning and tuning

Homepage: https://github.com/microsoft/FLAML
Documentation: https://flaml.readthedocs.io/
License: MIT License
Latest release: 2.3.6
published 6 months ago

Versions: 101
Dependent Packages: 14
Dependent Repositories: 388
Downloads: 372,597 Last month
Docker Downloads: 21,948,847

Rankings

Downloads: 0.6%

Dependent repos count: 0.7%

Dependent packages count: 0.8%

Stargazers count: 1.3%

Average: 1.5%

Forks count: 2.5%

Docker downloads count: 3.2%

Maintainers (1)

automl

Last synced: 6 months ago

proxy.golang.org: github.com/microsoft/flaml

Documentation: https://pkg.go.dev/github.com/microsoft/flaml#section-documentation
License: mit
Latest release: v2.3.6+incompatible
published 6 months ago

Versions: 72
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 6.5%

Average: 6.7%

Dependent repos count: 7.0%

Last synced: 6 months ago

proxy.golang.org: github.com/microsoft/FLAML

Documentation: https://pkg.go.dev/github.com/microsoft/FLAML#section-documentation
License: mit
Latest release: v2.3.6+incompatible
published 6 months ago

Versions: 72
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 6.5%

Average: 6.7%

Dependent repos count: 7.0%

Last synced: 6 months ago

conda-forge.org: flaml

Homepage: https://github.com/microsoft/FLAML
License: MIT
Latest release: 1.0.14
published over 3 years ago

Versions: 31
Dependent Packages: 1
Dependent Repositories: 1

Rankings

Stargazers count: 8.4%

Forks count: 8.9%

Average: 17.7%

Dependent repos count: 24.4%

Dependent packages count: 29.0%

Last synced: 7 months ago

Dependencies

website/package.json npm

@docusaurus/core 0.0.0-4193
@docusaurus/preset-classic 0.0.0-4193
@easyops-cn/docusaurus-search-local ^0.21.1
@mdx-js/react ^1.6.21
@svgr/webpack ^5.5.0
clsx ^1.1.1
file-loader ^6.2.0
hast-util-is-element 1.1.0
react ^17.0.1
react-dom ^17.0.1
rehype-katex 4
remark-math 3
trim ^0.0.3
url-loader ^4.1.1

website/yarn.lock npm

1087 dependencies

setup.py pypi

NumPy >=1.17.0rc1
lightgbm >=2.3.1
pandas >=1.1.4
scikit-learn >=0.24
scipy >=1.4.1
xgboost >=0.90

.github/workflows/CD.yml actions

actions/cache v3 composite
actions/checkout v3 composite
conda-incubator/setup-miniconda v2 composite

.github/workflows/deploy-website.yml actions

actions/checkout v3 composite
actions/setup-node v3 composite
actions/setup-python v4 composite
peaceiris/actions-gh-pages v3 composite

.github/workflows/openai.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite
codecov/codecov-action v3 composite

.github/workflows/pre-commit.yml actions

actions/cache v3 composite
actions/checkout v3 composite
actions/setup-python v4 composite
pre-commit/action v3.0.0 composite

.github/workflows/python-package.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite
codecov/codecov-action v3 composite

.devcontainer/Dockerfile docker

mcr.microsoft.com/vscode/devcontainers/python 0-3.9 build

Dockerfile docker

python 3.7 build

pyproject.toml pypi

test/pipeline_tuning_example/requirements.txt pypi

azure-ml-component ==0.9.10.post1 test
azureml-core ==1.39.0 test
azureml-dataset-runtime ==1.39.0 test
flaml ==1.0.9 test
hydra-core ==1.1.1 test