**Awesome LLMOps**
An awesome & curated list of the best LLMOps tools for developers.
**Contribute**
Contributions are most welcome, please adhere to the [contribution guidelines](contributing.md).
Table of Contents
=================
* [Table of Contents](#table-of-contents)
* [Model](#model)
* [Large Language Model](#large-language-model)
* [CV Foundation Model](#cv-foundation-model)
* [Audio Foundation Model](#audio-foundation-model)
* [Serving](#serving)
* [Large Model Serving](#large-model-serving)
* [Frameworks/Servers for Serving](#frameworksservers-for-serving)
* [Observability](#observability)
* [LLMOps](#llmops)
* [Search](#search)
* [Vector search](#vector-search)
* [Code AI](#code-ai)
* [Training](#training)
* [IDEs and Workspaces](#ides-and-workspaces)
* [Foundation Model Fine Tuning](#foundation-model-fine-tuning)
* [Frameworks for Training](#frameworks-for-training)
* [Experiment Tracking](#experiment-tracking)
* [Visualization](#visualization)
* [Data](#data)
* [Data Management](#data-management)
* [Data Storage](#data-storage)
* [Data Tracking](#data-tracking)
* [Feature Engineering](#feature-engineering)
* [Data/Feature enrichment](#datafeature-enrichment)
* [Large Scale Deployment](#large-scale-deployment)
* [ML Platforms](#ml-platforms)
* [Workflow](#workflow)
* [Scheduling](#scheduling)
* [Model Management](#model-management)
* [Performance](#performance)
* [ML Compiler](#ml-compiler)
* [Profiling](#profiling)
* [AutoML](#automl)
* [Optimizations](#optimizations)
* [Federated ML](#federated-ml)
* [Awesome Lists](#awesome-lists)
# Model
## Large Language Model
- [Alpaca](https://github.com/tatsu-lab/stanford_alpaca)  - Code and documentation to train Stanford's Alpaca models, and generate the data.
- [BELLE](https://github.com/LianjiaTech/BELLE)  - A 7B Large Language Model fine-tune by 34B Chinese Character Corpus, based on LLaMA and Alpaca.
- [Bloom](https://github.com/bigscience-workshop/model_card)  - BigScience Large Open-science Open-access Multilingual Language Model
- [dolly](https://github.com/databrickslabs/dolly)  - Databricks Dolly, a large language model trained on the Databricks Machine Learning Platform
- [GLM-6B (ChatGLM)](https://github.com/THUDM/ChatGLM-6B)  - An Open Bilingual Pre-Trained Model, quantization of ChatGLM-130B, can run on consumer-level GPUs.
- [GLM-130B (ChatGLM)](https://github.com/THUDM/GLM-130B)  - An Open Bilingual Pre-Trained Model (ICLR 2023)
- [GPT-NeoX](https://github.com/EleutherAI/gpt-neox)  - An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
- [Luotuo](https://github.com/LC1332/Luotuo-Chinese-LLM)  - A Chinese LLM, Based on LLaMA and fine tune by Stanford Alpaca, Alpaca LoRA, Japanese-Alpaca-LoRA.
- [StableLM](https://github.com/Stability-AI/StableLM)  - StableLM: Stability AI Language Models
**[ back to ToC](#table-of-contents)**
## CV Foundation Model
- [disco-diffusion](https://github.com/alembics/disco-diffusion)  - A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations.
- [midjourney](https://www.midjourney.com/home/) - Midjourney is an independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species.
- [segment-anything (SAM)](https://github.com/facebookresearch/segment-anything)  - produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image.
- [stable-diffusion](https://github.com/CompVis/stable-diffusion)  - A latent text-to-image diffusion model
- [stable-diffusion v2](https://github.com/Stability-AI/stablediffusion)  - High-Resolution Image Synthesis with Latent Diffusion Models
**[ back to ToC](#table-of-contents)**
## Audio Foundation Model
- [bark](https://github.com/suno-ai/bark)  - Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects.
- [whisper](https://github.com/openai/whisper)  - Robust Speech Recognition via Large-Scale Weak Supervision
# Serving
## Large Model Serving
- [Alpaca-LoRA-Serve](https://github.com/deep-diver/Alpaca-LoRA-Serve)  - Alpaca-LoRA as Chatbot service
- [DeepSpeed-MII](https://github.com/microsoft/DeepSpeed-MII)  - MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
- [FlexGen](https://github.com/FMInference/FlexGen)  - Running large language models on a single GPU for throughput-oriented scenarios.
- [Flowise](https://github.com/FlowiseAI/Flowise)  - Drag & drop UI to build your customized LLM flow using LangchainJS.
- [llama.cpp](https://github.com/ggerganov/llama.cpp)  - Port of Facebook's LLaMA model in C/C++
- [whisper.cpp](https://github.com/ggerganov/whisper.cpp)  - Port of OpenAI's Whisper model in C/C++
- [x-stable-diffusion](https://github.com/stochasticai/x-stable-diffusion)  - Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention.
**[ back to ToC](#table-of-contents)**
## Frameworks/Servers for Serving
- [BentoML](https://github.com/bentoml/BentoML)  - The Unified Model Serving Framework
- [Mosec](https://github.com/mosecorg/mosec)  - A machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface.
- [TFServing](https://github.com/tensorflow/serving)  - A flexible, high-performance serving system for machine learning models.
- [Torchserve](https://github.com/pytorch/serve)  - Serve, optimize and scale PyTorch models in production
- [Triton Server (TRTIS)](https://github.com/triton-inference-server/server)  - The Triton Inference Server provides an optimized cloud and edge inferencing solution.
**[ back to ToC](#table-of-contents)**
## Observability
- [Deepchecks](https://github.com/deepchecks/deepchecks)  - Tests for Continuous Validation of ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.
- [Evidently](https://github.com/evidentlyai/evidently)  - Evaluate and monitor ML models from validation to production.
- [Great Expectations](https://github.com/great-expectations/great_expectations)  - Always know what to expect from your data.
- [whylogs](https://github.com/whylabs/whylogs)  - The open standard for data logging
**[ back to ToC](#table-of-contents)**
# LLMOps
- [deeplake](https://github.com/activeloopai/deeplake)  - Stream large multimodal datasets to achieve near 100% GPU utilization. Query, visualize, & version control data. Access data w/o the need to recompute the embeddings for the model finetuning.
- [GPTCache](https://github.com/zilliztech/GPTCache)  - Creating semantic cache to store responses from LLM queries.
- [Haystack](https://github.com/deepset-ai/haystack)  - Quickly compose applications with LLM Agents, semantic search, question-answering and more.
- [langchain](https://github.com/hwchase17/langchain)  - Building applications with LLMs through composability
- [LlamaIndex](https://github.com/jerryjliu/llama_index)  - Provides a central interface to connect your LLMs with external data.
- [xTuring](https://github.com/stochasticai/xturing)  - Build and control your personal LLMs with fast and efficient fine-tuning.
- [ZenML](https://github.com/zenml-io/zenml)  - Open-source framework for orchestrating, experimenting and deploying production-grade ML solutions, with built-in `langchain` & `llama_index` integrations.
**[ back to ToC](#table-of-contents)**
# Search
## Vector search
- [AquilaDB](https://github.com/Aquila-Network/AquilaDB)  - An easy to use Neural Search Engine. Index latent vectors along with JSON metadata and do efficient k-NN search.
- [Chroma](https://github.com/chroma-core/chroma)  - the open source embedding database
- [Jina](https://github.com/jina-ai/jina)  - Build multimodal AI services via cloud native technologies Neural Search Generative AI Cloud Native
- [Marqo](https://github.com/marqo-ai/marqo)  - Tensor search for humans.
- [Milvus](https://github.com/milvus-io/milvus)  - Vector database for scalable similarity search and AI applications.
- [Pinecone](https://www.pinecone.io/) - The Pinecone vector database makes it easy to build high-performance vector search applications. Developer-friendly, fully managed, and easily scalable without infrastructure hassles
- [Qdrant](https://github.com/qdrant/qdrant)  - Vector Search Engine and Database for the next generation of AI applications. Also available in the cloud
- [txtai](https://github.com/neuml/txtai)  - Build AI-powered semantic search applications
- [Vald](https://github.com/vdaas/vald)  - A Highly Scalable Distributed Vector Search Engine
- [Vearch](https://github.com/vearch/vearch)  - A distributed system for embedding-based vector retrieval
- [Weaviate](https://github.com/semi-technologies/weaviate)  - Weaviate is an open source vector search engine that stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients.
**[ back to ToC](#table-of-contents)**
# Code AI
- [CodeGen](https://github.com/salesforce/CodeGen)  - CodeGen is an open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.
- [fauxpilot](https://github.com/fauxpilot/fauxpilot)  - An open-source alternative to GitHub Copilot server
- [tabby](https://github.com/TabbyML/tabby)  - Self-hosted AI coding assistant. An opensource / on-prem alternative to GitHub Copilot.
# Training
## IDEs and Workspaces
- [code server](https://github.com/coder/code-server)  - Run VS Code on any machine anywhere and access it in the browser.
- [conda](https://github.com/conda/conda)  - OS-agnostic, system-level binary package manager and ecosystem.
- [Docker](https://github.com/moby/moby)  - Moby is an open-source project created by Docker to enable and accelerate software containerization.
- [envd](https://github.com/tensorchord/envd)  - Reproducible development environment for AI/ML.
- [Jupyter Notebooks](https://github.com/jupyter/notebook)  - The Jupyter notebook is a web-based notebook environment for interactive computing.
- [Kurtosis](https://github.com/kurtosis-tech/kurtosis)  - A build, packaging, and run system for ephemeral multi-container environments.
**[ back to ToC](#table-of-contents)**
## Foundation Model Fine Tuning
- [alpaca-lora](https://github.com/tloen/alpaca-lora)  - Instruct-tune LLaMA on consumer hardware
- [LMFlow](https://github.com/OptimalScale/LMFlow)  - An Extensible Toolkit for Finetuning and Inference of Large Foundation Models
- [Lora](https://github.com/cloneofsimo/lora)  - Using Low-rank adaptation to quickly fine-tune diffusion models.
- [peft](https://github.com/huggingface/peft)  - State-of-the-art Parameter-Efficient Fine-Tuning.
- [p-tuning-v2](https://github.com/THUDM/P-tuning-v2)  - An optimized prompt tuning strategy achieving comparable performance to fine-tuning on small/medium-sized models and sequence tagging challenges. [(ACL 2022)](https://arxiv.org/abs/2110.07602)
**[ back to ToC](#table-of-contents)**
## Frameworks for Training
- [Apache MXNet](https://github.com/apache/mxnet)  - Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler.
- [Caffe](https://github.com/BVLC/caffe)  - A fast open framework for deep learning.
- [ColossalAI](https://github.com/hpcaitech/ColossalAI)  - An integrated large-scale model training system with efficient parallelization techniques.
- [DeepSpeed](https://github.com/microsoft/DeepSpeed)  - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
- [Horovod](https://github.com/horovod/horovod)  - Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
- [Jax](https://github.com/google/jax)  - Autograd and XLA for high-performance machine learning research.
- [Kedro](https://github.com/kedro-org/kedro)  - Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code.
- [Keras](https://github.com/keras-team/keras)  - Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow.
- [LightGBM](https://github.com/microsoft/LightGBM)  - A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
- [MegEngine](https://github.com/MegEngine/MegEngine)  - MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto-differentiation.
- [metric-learn](https://github.com/scikit-learn-contrib/metric-learn)  - Metric Learning Algorithms in Python.
- [MindSpore](https://github.com/mindspore-ai/mindspore)  - MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
- [Oneflow](https://github.com/Oneflow-Inc/oneflow)  - OneFlow is a performance-centered and open-source deep learning framework.
- [PaddlePaddle](https://github.com/PaddlePaddle/Paddle)  - Machine Learning Framework from Industrial Practice.
- [PyTorch](https://github.com/pytorch/pytorch)  - Tensors and Dynamic neural networks in Python with strong GPU acceleration.
- [PyTorchLightning](https://github.com/PyTorchLightning/pytorch-lightning)  - The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
- [XGBoost](https://github.com/dmlc/xgboost)  - Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library.
- [scikit-learn](https://github.com/scikit-learn/scikit-learn)  - Machine Learning in Python.
- [TensorFlow](https://github.com/tensorflow/tensorflow)  - An Open Source Machine Learning Framework for Everyone.
- [VectorFlow](https://github.com/Netflix/vectorflow)  - A minimalist neural network library optimized for sparse data and single machine environments.
**[ back to ToC](#table-of-contents)**
## Experiment Tracking
- [Aim](https://github.com/aimhubio/aim)  - an easy-to-use and performant open-source experiment tracker.
- [ClearML](https://github.com/allegroai/clearml)  - Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management
- [Guild AI](https://github.com/guildai/guildai)  - Experiment tracking, ML developer tools.
- [MLRun](https://github.com/mlrun/mlrun)  - Machine Learning automation and tracking.
- [Kedro-Viz](https://github.com/kedro-org/kedro-viz)  - Kedro-Viz is an interactive development tool for building data science pipelines with Kedro. Kedro-Viz also allows users to view and compare different runs in the Kedro project.
- [LabNotebook](https://github.com/henripal/labnotebook)  - LabNotebook is a tool that allows you to flexibly monitor, record, save, and query all your machine learning experiments.
- [Sacred](https://github.com/IDSIA/sacred)  - Sacred is a tool to help you configure, organize, log and reproduce experiments.
**[ back to ToC](#table-of-contents)**
## Visualization
- [Maniford](https://github.com/uber/manifold)  - A model-agnostic visual debugging tool for machine learning.
- [netron](https://github.com/lutzroeder/netron)  - Visualizer for neural network, deep learning, and machine learning models.
- [OpenOps](https://github.com/ThePlugJumbo/openops)  - Bring multiple data streams into one dashboard.
- [TensorBoard](https://github.com/tensorflow/tensorboard)  - TensorFlow's Visualization Toolkit.
- [TensorSpace](https://github.com/tensorspace-team/tensorspace)  - Neural network 3D visualization framework, build interactive and intuitive model in browsers, support pre-trained deep learning models from TensorFlow, Keras, TensorFlow.js.
- [dtreeviz](https://github.com/parrt/dtreeviz)  - A python library for decision tree visualization and model interpretation.
- [Zetane Viewer](https://github.com/zetane/viewer)  - ML models and internal tensors 3D visualizer.
- [Zeno](https://github.com/zeno-ml/zeno)  - AI evaluation platform for interactively exploring data and model outputs.
**[ back to ToC](#table-of-contents)**
# Data
## Data Management
- [ArtiVC](https://github.com/InfuseAI/ArtiVC)  - A version control system to manage large files.
Lake is a dataset format with a simple API for creating, storing, and collaborating on AI datasets of any size.
- [Dolt](https://github.com/dolthub/dolt)  - Git for Data.
- [DVC](https://github.com/iterative/dvc)  - Data Version Control | Git for Data & Models | ML Experiments Management.
- [Delta-Lake](https://github.com/delta-io/delta)  - Storage layer that brings scalable, ACID transactions to Apache Spark and other engines.
- [Pachyderm](https://github.com/pachyderm/pachyderm)  - Pachyderm is a version control system for data.
- [Quilt](https://github.com/quiltdata/quilt)  - A self-organizing data hub for S3.
**[ back to ToC](#table-of-contents)**
## Data Storage
- [JuiceFS](https://github.com/juicedata/juicefs)  - A distributed POSIX file system built on top of Redis and S3.
- [LakeFS](https://github.com/treeverse/lakeFS)  - Git-like capabilities for your object storage.
- [Lance](https://github.com/eto-ai/lance)  - Modern columnar data format for ML implemented in Rust.
**[ back to ToC](#table-of-contents)**
## Data Tracking
- [Piperider](https://github.com/InfuseAI/piperider)  - A CLI tool that allows you to build data profiles and write assertion tests for easily evaluating and tracking your data's reliability over time.
- [LUX](https://github.com/lux-org/lux)  - A Python library that facilitates fast and easy data exploration by automating the visualization and data analysis process.
**[ back to ToC](#table-of-contents)**
## Feature Engineering
- [Featureform](https://github.com/featureform/featureform)  - The Virtual Feature Store. Turn your existing data infrastructure into a feature store.
- [FeatureTools](https://github.com/Featuretools/featuretools)  - An open source python framework for automated feature engineering
**[ back to ToC](#table-of-contents)**
## Data/Feature enrichment
- [Upgini](https://github.com/upgini/upgini)  - Free automated data & feature enrichment library for machine learning: automatically searches through thousands of ready-to-use features from public and community shared data sources and enriches your training dataset with only the accuracy improving features
- [Feast](https://github.com/feast-dev/feast)  - An open source feature store for machine learning.
**[ back to ToC](#table-of-contents)**
# Large Scale Deployment
## ML Platforms
- [ClearML](https://github.com/allegroai/clearml)  - Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management.
- [MLflow](https://github.com/mlflow/mlflow)  - Open source platform for the machine learning lifecycle.
- [MLRun](https://github.com/mlrun/mlrun)  - An open MLOps platform for quickly building and managing continuous ML applications across their lifecycle.
- [ModelFox](https://github.com/modelfoxdotdev/modelfox)  - ModelFox is a platform for managing and deploying machine learning models.
- [Kserve](https://github.com/kserve/kserve)  - Standardized Serverless ML Inference Platform on Kubernetes
- [Kubeflow](https://github.com/kubeflow/kubeflow)  - Machine Learning Toolkit for Kubernetes.
- [PAI](https://github.com/microsoft/pai)  - Resource scheduling and cluster management for AI.
- [Polyaxon](https://github.com/polyaxon/polyaxon)  - Machine Learning Management & Orchestration Platform.
- [Primehub](https://github.com/InfuseAI/primehub)  - An effortless infrastructure for machine learning built on the top of Kubernetes.
- [Seldon-core](https://github.com/SeldonIO/seldon-core)  - An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
**[ back to ToC](#table-of-contents)**
## Workflow
- [Airflow](https://airflow.apache.org/)  - A platform to programmatically author, schedule and monitor workflows.
- [aqueduct](https://github.com/aqueducthq/aqueduct)  - An Open-Source Platform for Production Data Science
- [Argo Workflows](https://github.com/argoproj/argo-workflows)  - Workflow engine for Kubernetes.
- [Flyte](https://github.com/flyteorg/flyte)  - Kubernetes-native workflow automation platform for complex, mission-critical data and ML processes at scale.
- [Kubeflow Pipelines](https://github.com/kubeflow/pipelines)  - Machine Learning Pipelines for Kubeflow.
- [Metaflow](https://github.com/Netflix/metaflow)  - Build and manage real-life data science projects with ease!
- [Ploomber](https://github.com/ploomber/ploomber)  - The fastest way to build data pipelines. Develop iteratively, deploy anywhere.
- [Prefect](https://github.com/PrefectHQ/prefect)  - The easiest way to automate your data.
- [VDP](https://github.com/instill-ai/vdp)  - An open-source unstructured data ETL tool to streamline the end-to-end unstructured data processing pipeline.
- [ZenML](https://github.com/zenml-io/zenml)  - MLOps framework to create reproducible pipelines.
**[ back to ToC](#table-of-contents)**
## Scheduling
- [Kueue](https://github.com/kubernetes-sigs/kueue)  - Kubernetes-native Job Queueing.
- [PAI](https://github.com/microsoft/pai)  - Resource scheduling and cluster management for AI (Open-sourced by Microsoft).
- [Slurm](https://github.com/SchedMD/slurm)  - A Highly Scalable Workload Manager.
- [Volcano](https://github.com/volcano-sh/volcano)  - A Cloud Native Batch System (Project under CNCF).
- [Yunikorn](https://github.com/apache/yunikorn-core)  - Light-weight, universal resource scheduler for container orchestrator systems.
**[ back to ToC](#table-of-contents)**
## Model Management
- [dvc](https://github.com/iterative/dvc)  - Data Version Control | Git for Data & Models | ML Experiments Management
- [ModelDB](https://github.com/VertaAI/modeldb)  - Open Source ML Model Versioning, Metadata, and Experiment Management
- [MLEM](https://github.com/iterative/mlem)  - A tool to package, serve, and deploy any ML model on any platform.
- [ormb](https://github.com/kleveross/ormb)  - Docker for Your ML/DL Models Based on OCI Artifacts
**[ back to ToC](#table-of-contents)**
# Performance
## ML Compiler
- [ONNX-MLIR](https://github.com/onnx/onnx-mlir)  - Compiler technology to transform a valid Open Neural Network Exchange (ONNX) graph into code that implements the graph with minimum runtime support.
- [TVM](https://github.com/apache/tvm)  - Open deep learning compiler stack for cpu, gpu and specialized accelerators
**[ back to ToC](#table-of-contents)**
## Profiling
- [octoml-profile](https://github.com/octoml/octoml-profile)  - octoml-profile is a python library and cloud service designed to provide the simplest experience for assessing and optimizing the performance of PyTorch models on cloud hardware with state-of-the-art ML acceleration technology.
- [scalene](https://github.com/plasma-umass/scalene)  - a high-performance, high-precision CPU, GPU, and memory profiler for Python
**[ back to ToC](#table-of-contents)**
# AutoML
- [Archai](https://github.com/microsoft/archai)  - a platform for Neural Network Search (NAS) that allows you to generate efficient deep networks for your applications.
- [autoai](https://github.com/blobcity/autoai)  - A framework to find the best performing AI/ML model for any AI problem.
- [AutoGL](https://github.com/THUMNLab/AutoGL)  - An autoML framework & toolkit for machine learning on graphs
- [AutoGluon](https://github.com/awslabs/autogluon)  - AutoML for Image, Text, and Tabular Data.
- [automl-gs](https://github.com/minimaxir/automl-gs)  - Provide an input CSV and a target field to predict, generate a model + code to run it.
- [autokeras](https://github.com/keras-team/autokeras)  - AutoML library for deep learning.
- [Auto-PyTorch](https://github.com/automl/Auto-PyTorch)  - Automatic architecture search and hyperparameter optimization for PyTorch.
- [auto-sklearn](https://github.com/automl/auto-sklearn)  - an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator.
- [Dragonfly](https://github.com/dragonfly/dragonfly)  - An open source python library for scalable Bayesian optimisation.
- [Determined](https://github.com/determined-ai/determined)  - scalable deep learning training platform with integrated hyperparameter tuning support; includes Hyperband, PBT, and other search methods.
- [DEvol (DeepEvolution)](https://github.com/joeddav/devol)  - a basic proof of concept for genetic architecture search in Keras.
- [EvalML](https://github.com/alteryx/evalml)  - An open source python library for AutoML.
- [FEDOT](https://github.com/nccr-itmo/FEDOT)  - AutoML framework for the design of composite pipelines.
- [FLAML](https://github.com/microsoft/FLAML)  - Fast and lightweight AutoML ([paper](https://www.microsoft.com/en-us/research/publication/flaml-a-fast-and-lightweight-automl-library/)).
- [Goptuna](https://github.com/c-bata/goptuna)  - A hyperparameter optimization framework, inspired by Optuna.
- [HpBandSter](https://github.com/automl/HpBandSter)  - a framework for distributed hyperparameter optimization.
- [HPOlib2](https://github.com/automl/HPOlib2)  - a library for hyperparameter optimization and black box optimization benchmarks.
- [Hyperband](https://github.com/zygmuntz/hyperband)  - open source code for tuning hyperparams with Hyperband.
- [Hypernets](https://github.com/DataCanvasIO/Hypernets)  - A General Automated Machine Learning Framework.
- [Hyperopt](https://github.com/hyperopt/hyperopt)  - Distributed Asynchronous Hyperparameter Optimization in Python.
- [hyperunity](https://github.com/gdikov/hypertunity)  - A toolset for black-box hyperparameter optimisation.
- [Katib](https://github.com/kubeflow/katib)  - Katib is a Kubernetes-native project for automated machine learning (AutoML).
- [Keras Tuner](https://github.com/keras-team/keras-tuner)  - Hyperparameter tuning for humans.
- [learn2learn](https://github.com/learnables/learn2learn)  - PyTorch Meta-learning Framework for Researchers.
- [Ludwig](https://github.com/uber/ludwig)  - a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code.
- [MOE](https://github.com/Yelp/MOE)  - a global, black box optimization engine for real world metric optimization by Yelp.
- [Model Search](https://github.com/google/model_search)  - a framework that implements AutoML algorithms for model architecture search at scale.
- [NASGym](https://github.com/gomerudo/nas-env)  - a proof-of-concept OpenAI Gym environment for Neural Architecture Search (NAS).
- [NNI](https://github.com/Microsoft/nni)  - An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
- [Optuna](https://github.com/optuna/optuna)  - A hyperparameter optimization framework.
- [Pycaret](https://github.com/pycaret/pycaret)  - An open-source, low-code machine learning library in Python that automates machine learning workflows.
- [Ray Tune](github.com/ray-project/ray)  - Scalable Hyperparameter Tuning.
- [REMBO](https://github.com/ziyuw/rembo)  - Bayesian optimization in high-dimensions via random embedding.
- [RoBO](https://github.com/automl/RoBO)  - a Robust Bayesian Optimization framework.
- [scikit-optimize(skopt)](https://github.com/scikit-optimize/scikit-optimize)  - Sequential model-based optimization with a `scipy.optimize` interface.
- [Spearmint](https://github.com/HIPS/Spearmint)  - a software package to perform Bayesian optimization.
- [TPOT](http://automl.info/tpot/)  - one of the very first AutoML methods and open-source software packages.
- [Torchmeta](https://github.com/tristandeleu/pytorch-meta)  - A Meta-Learning library for PyTorch.
- [Vegas](https://github.com/huawei-noah/vega)  - an AutoML algorithm tool chain by Huawei Noah's Arb Lab.
**[ back to ToC](#table-of-contents)**
# Optimizations
- [FeatherCNN](https://github.com/Tencent/FeatherCNN)  - FeatherCNN is a high performance inference engine for convolutional neural networks.
- [Forward](https://github.com/Tencent/Forward)  - A library for high performance deep learning inference on NVIDIA GPUs.
- [NCNN](https://github.com/Tencent/ncnn)  - ncnn is a high-performance neural network inference framework optimized for the mobile platform.
- [PocketFlow](https://github.com/Tencent/PocketFlow)  - use AutoML to do model compression.
- [TensorFlow Model Optimization](https://github.com/tensorflow/model-optimization)  - A suite of tools that users, both novice and advanced, can use to optimize machine learning models for deployment and execution.
- [TNN](https://github.com/Tencent/TNN)  - A uniform deep learning inference framework for mobile, desktop and server.
**[ back to ToC](#table-of-contents)**
# Federated ML
- [EasyFL](https://github.com/EasyFL-AI/EasyFL)  - An Easy-to-use Federated Learning Platform
- [FATE](https://github.com/FederatedAI/FATE)  - An Industrial Grade Federated Learning Framework
- [FedML](https://github.com/FedML-AI/FedML)  - The federated learning and analytics library enabling secure and collaborative machine learning on decentralized data anywhere at any scale. Supporting large-scale cross-silo federated learning, cross-device federated learning on smartphones/IoTs, and research simulation.
- [Flower](https://github.com/adap/flower)  - A Friendly Federated Learning Framework
- [Harmonia](https://github.com/ailabstw/harmonia)  - Harmonia is an open-source project aiming at developing systems/infrastructures and libraries to ease the adoption of federated learning (abbreviated to FL) for researches and production usage.
- [TensorFlow Federated](https://github.com/tensorflow/federated)  - A framework for implementing federated learning
**[ back to ToC](#table-of-contents)**
# Awesome Lists
- [Awesome Argo](https://github.com/terrytangyuan/awesome-argo)  - A curated list of awesome projects and resources related to Argo
- [Awesome AutoDL](https://github.com/D-X-Y/Awesome-AutoDL)  - Automated Deep Learning: Neural Architecture Search Is Not the End (a curated list of AutoDL resources and an in-depth analysis)
- [Awesome AutoML](https://github.com/windmaple/awesome-AutoML)  - Curating a list of AutoML-related research, tools, projects and other resources
- [Awesome AutoML Papers](https://github.com/hibayesian/awesome-automl-papers)  - A curated list of automated machine learning papers, articles, tutorials, slides and projects
- [Awesome Federated Learning](https://github.com/chaoyanghe/Awesome-Federated-Learning)  - A curated list of federated learning publications, re-organized from Arxiv (mostly)
- [awesome-federated-learning](https://github.com/weimingwill/awesome-federated-learning)  - All materials you need for Federated Learning: blogs, videos, papers, and softwares, etc.
- [Awesome Open MLOps](https://github.com/fuzzylabs/awesome-open-mlops)  - This is the Fuzzy Labs guide to the universe of free and open source MLOps tools.
- [Awesome Production Machine Learning](https://github.com/EthicalML/awesome-production-machine-learning)  - A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
- [Awesome Tensor Compilers](https://github.com/merrymercy/awesome-tensor-compilers)  - A list of awesome compiler projects and papers for tensor computation and deep learning.
- [kelvins/awesome-mlops](https://github.com/kelvins/awesome-mlops)  - A curated list of awesome MLOps tools.
- [visenger/awesome-mlops](https://github.com/visenger/awesome-mlops)  - An awesome list of references for MLOps - Machine Learning Operations
- [currentslab/awesome-vector-search](https://github.com/currentslab/awesome-vector-search)  - A curated list of awesome vector search framework/engine, library, cloud service and research papers to vector similarity search.
**[ back to ToC](#table-of-contents)**