openfederatedlearning

An Open Framework for Federated Learning.

https://github.com/securefederatedai/openfederatedlearning

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 2 DOI reference(s) in README
  • Academic publication links
    Links to: arxiv.org, iop.org
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (12.4%) to scientific vocabulary

Keywords

federated-analytics federated-evaluation federated-learning privacy-preserving-ml python trusted-execution-environment
Last synced: 6 months ago · JSON representation ·

Repository

An Open Framework for Federated Learning.

Basic Info
  • Host: GitHub
  • Owner: securefederatedai
  • License: apache-2.0
  • Language: Python
  • Default Branch: develop
  • Homepage: https://openfl.readthedocs.io
  • Size: 132 MB
Statistics
  • Stars: 802
  • Watchers: 19
  • Forks: 231
  • Open Issues: 78
  • Releases: 14
Topics
federated-analytics federated-evaluation federated-learning privacy-preserving-ml python trusted-execution-environment
Created about 5 years ago · Last pushed 6 months ago
Metadata Files
Readme Contributing License Code of conduct Citation Security Governance Roadmap

README.md

[![PyPI version](https://img.shields.io/pypi/v/openfl)](https://pypi.org/project/openfl/) [![Conda version](https://img.shields.io/conda/vn/conda-forge/openfl?color=green)](https://anaconda.org/conda-forge/openfl) [![Downloads](https://pepy.tech/badge/openfl)](https://pepy.tech/project/openfl) [![Documentation Status](https://readthedocs.org/projects/openfl/badge/?version=latest)](https://openfl.readthedocs.io/en/latest/?badge=latest) [](https://join.slack.com/t/openfl/shared_invite/zt-ovzbohvn-T5fApk05~YS_iZhjJ5yaTw) [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/6599/badge)](https://bestpractices.coreinfrastructure.org/projects/6599) [![Product Quality Pipeline](https://github.com/securefederatedai/openfl/actions/workflows/pq_pipeline.yml/badge.svg)](https://github.com/securefederatedai/openfl/actions/workflows/pq_pipeline.yml) Coverity Scan Build Status [**Overview**](#overview) | [**Installation**](#installation) | [**Features**](#features) | [**Changelog**](https://openfl.readthedocs.io/en/latest/releases.html) | [**Documentation**](https://openfl.readthedocs.io/en/latest/)

Open Federated Learning (formerly known as OpenFL) is a Python framework for Federated Learning. It enables organizations to train and validate machine learning models on sensitive data. It increases privacy by allowing collaborative model training or validation across local private datasets without ever sharing that data with a central server. OpenFL is hosted by The Linux Foundation.

Looking for the Open Flash Library project also referred to as OpenFL? Find it here!

Overview

Federated Learning is a distributed machine learning approach that enables collaborative training and evaluation of models without sharing sensitive data such as, personal information, patient records, financial data, or classified information. The minimum data movement needed across a Federated Training experiment, is solely the model parameters and their updates. This is in contrast to a Centralized Learning regime, where all data needs to be moved to a central server or a datacenter for massively parallel training.

Federated Learning

Open Federated Learning builds on a collaboration between Intel and the Bakas lab at the University of Pennsylvania (UPenn) to develop the Federated Tumor Segmentation (FeTS) platform (grant award number: U01-CA242871).

The grant for FeTS was awarded from the Informatics Technology for Cancer Research (ITCR) program of the National Cancer Institute (NCI) of the National Institutes of Health (NIH), to Dr. Spyridon Bakas (Principal Investigator) when he was affiliated with the Center for Biomedical Image Computing and Analytics (CBICA) at UPenn and now heading the Division of Computational Pathology at Indiana University (IU).

FeTS is a real-world medical federated learning platform with international collaborators. The original OpenFederatedLearning project and Open Federated Learning are designed to serve as the backend for the FeTS platform, and Open Federated Learning developers and researchers continue to work very closely with IU on the FeTS project. An example is the FeTS-AI/Front-End, which integrates the group’s medical AI expertise with Open Federated Learning framework to create a federated learning solution for medical imaging.

Although initially developed for use in medical imaging, Open Federated Learning is designed to be agnostic to the use-case, the industry, and the machine learning framework.

For more information, here is a list of relevant publications.

Installation

Install via PyPI (latest stable release):

pip install -U openfl Or via conda: conda install conda-forge::openfl For more installation options, checkout the installation guide.

Features

Ways to set up an FL experiment

OpenFL supports two ways to set up a Federated Learning experiment:

  • TaskRunner API: This API uses short-lived components like the Aggregator and Collaborator, which terminate at the end of an FL experiment. TaskRunner supports mTLS-based secure communication channels, and TEE-based confidential computing environments.

  • Workflow API: This API allows for experiments beyond the traditional horizontal federated learning paradigm using a pythonic interface. It allows an experiment to be simulated locally, and then to be seamlessly scaled to a federated setting by switching from a local runtime to a distributed, federated runtime.

    Note: This is experimental capability.

Framework Compatibility

OpenFL is backend-agnostic. It comes with support for popular NumPy-based ML frameworks like TensorFlow, PyTorch and Jax which should be installed separately. Users may extend the list of supported backends if needed.

Aggregation Algorithms

Open Federated Learning supports popular aggregation algorithms out-of-the-box, with more algorithms coming soon. | | Reference | PyTorch backend | TensorFlow backend | NumPy backend | | -------------- | ----- | :--------------------: | :-----------------------: | :----------------------------: | | FedAvg | McMahan et al., 2017 | yes | yes | yes | | FedOpt | Reddi et al., 2020 | yes | yes | yes | | FedProx | Li et al., 2020 | yes | yes | - | | FedCurv | Shoham et al., 2019 | yes | - | - |

Contributing

We welcome contributions! Please refer to the contributing guidelines.

The OpenFL community is expanding, and we encourage you to join us. Connect with other enthusiasts, share knowledge, and contribute to the advancement of federated learning by joining our Slack channel.

Stay updated by subscribing to the Open Federated Learning mailing list: openfl-announce@lists.lfaidata.foundation.

License

This project is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

Citation

@article{openfl_citation, author={Foley, Patrick and Sheller, Micah J and Edwards, Brandon and Pati, Sarthak and Riviera, Walter and Sharma, Mansi and Moorthy, Prakash Narayana and Wang, Shi-han and Martin, Jason and Mirhaji, Parsa and Shah, Prashant and Bakas, Spyridon}, title={OpenFL: the open federated learning library}, journal={Physics in Medicine \& Biology}, url={http://iopscience.iop.org/article/10.1088/1361-6560/ac97d9}, year={2022}, doi={10.1088/1361-6560/ac97d9}, publisher={IOP Publishing} }

Owner

  • Name: securefederatedai
  • Login: securefederatedai
  • Kind: organization

Citation (CITATION)

@article{openfl_citation,
	author={Foley, Patrick and Sheller, Micah J and Edwards, Brandon and Pati, Sarthak and Riviera, Walter and Sharma, Mansi and Moorthy, Prakash Narayana and Wang, Shi-han and Martin, Jason and Mirhaji, Parsa and Shah, Prashant and Bakas, Spyridon},
	title={OpenFL: the open federated learning library},
	journal={Physics in Medicine \& Biology},
	url={http://iopscience.iop.org/article/10.1088/1361-6560/ac97d9},
	year={2022},
	doi={10.1088/1361-6560/ac97d9},
	publisher={IOP Publishing}
}

GitHub Events

Total
  • Watch event: 4
  • Delete event: 1
  • Issue comment event: 1
  • Pull request event: 1
  • Fork event: 1
Last Year
  • Watch event: 4
  • Delete event: 1
  • Issue comment event: 1
  • Pull request event: 1
  • Fork event: 1

Dependencies

docs/requirements-docs.txt pypi
  • sphinx-prompt *
  • sphinx-rtd-theme *
  • sphinx_substitution_extensions *
  • sphinxcontrib-kroki *
openfl-tutorials/interactive_api/MXNet_landmarks/envoy/sd_requirements.txt pypi
  • kaggle *
  • pynvml *
openfl-tutorials/interactive_api/MXNet_landmarks/workspace/requirements.txt pypi
  • kaggle *
  • matplotlib *
openfl-tutorials/interactive_api/PyTorch_DogsCats_ViT/envoy/sd_requirements.txt pypi
  • kaggle *
  • numpy *
  • pillow *
  • pynvml *
openfl-tutorials/interactive_api/PyTorch_DogsCats_ViT/workspace/requirements.txt pypi
  • linformer ==0.2.1
  • torchvision ==0.8.1
  • vit-pytorch ==0.22.0
openfl-tutorials/interactive_api/PyTorch_Histology/envoy/requirements.txt pypi
  • Pillow ==8.3.2
  • numpy ==1.19.1
  • tqdm ==4.48.2
openfl-tutorials/interactive_api/PyTorch_Histology_FedCurv/envoy/requirements.txt pypi
  • Pillow ==9.0.1
  • numpy ==1.19.1
  • tqdm ==4.48.2
openfl-tutorials/interactive_api/PyTorch_Huggingface_transformers_SUPERB/envoy/sd_requirements.txt pypi
  • datasets ==1.14
  • librosa *
  • numpy ==1.22.0
  • pynvml *
openfl-tutorials/interactive_api/PyTorch_Kvasir_UNet/envoy/sd_requirements.txt pypi
  • numpy *
  • pillow *
  • pynvml *
openfl-tutorials/interactive_api/PyTorch_Lightning_MNIST_GAN/envoy/sd_requirements.txt pypi
  • numpy *
  • pillow *
  • pynvml ==11.4.1
  • torch ==1.9.1
  • torchvision ==0.10.1
openfl-tutorials/interactive_api/PyTorch_MVTec_PatchSVDD/envoy/sd_requirements.txt pypi
  • imageio *
  • numpy *
  • pillow *
openfl-tutorials/interactive_api/PyTorch_Market_Re-ID/envoy/requirements.txt pypi
  • Pillow ==8.3.2
  • gdown ==3.13.0
openfl-tutorials/interactive_api/PyTorch_Market_Re-ID/workspace/requirements.txt pypi
  • torch ==1.9.0
  • torchvision ==0.10.0
openfl-tutorials/interactive_api/PyTorch_MedMNIST_2D/envoy/requirements.txt pypi
  • medmnist *
openfl-tutorials/interactive_api/PyTorch_MedMNIST_3D/envoy/requirements.txt pypi
  • ACSConv *
  • medmnist *
openfl-tutorials/interactive_api/PyTorch_TinyImageNet/envoy/requirements.txt pypi
  • Pillow ==8.3.2
openfl-tutorials/interactive_api/PyTorch_TinyImageNet/workspace/requirements.txt pypi
  • torch ==1.7.0
  • torchvision ==0.8.1
openfl-tutorials/interactive_api/Tensorflow_Word_Prediction/envoy/sd_requirements.txt pypi
  • gdown ==3.13.0
  • numpy ==1.19.5
  • pandas ==1.3.3
  • pyarrow ==5.0.0
openfl-tutorials/interactive_api/Tensorflow_Word_Prediction/workspace/requirements.txt pypi
  • numpy ==1.19.5
  • tensorflow ==2.7.2
openfl-tutorials/interactive_api/jax_linear_regression/envoy/requirements.txt pypi
  • jax ==0.3.13
  • jaxlib ==0.3.10
  • openfl ==1.3
openfl-tutorials/interactive_api/numpy_linear_regression/envoy/requirements.txt pypi
  • numpy *
  • openfl ==1.2.1
openfl-tutorials/interactive_api/numpy_linear_regression/workspace/requirements.txt pypi
  • jupyterlab *
  • matplotlib *
  • numpy *
  • openfl ==1.2.1
openfl-workspace/keras_cnn_mnist/requirements.txt pypi
  • tensorflow ==2.7.0
openfl-workspace/keras_cnn_with_compression/requirements.txt pypi
  • tensorflow ==2.7.0
openfl-workspace/keras_nlp/requirements.txt pypi
  • tensorflow ==2.7.0
openfl-workspace/keras_nlp_gramine_ready/requirements.txt pypi
  • tensorflow-cpu ==2.7.0
openfl-workspace/tf_2dunet/requirements.txt pypi
  • nibabel *
  • tensorflow ==2.7.0
openfl-workspace/tf_3dunet_brats/requirements.txt pypi
  • nibabel *
  • numpy *
  • tensorflow >=2
openfl-workspace/tf_cnn_histology/requirements.txt pypi
  • pillow *
  • tensorflow ==2.7.2
  • tensorflow-datasets *
openfl-workspace/torch_cnn_histology/src/requirements.txt pypi
  • Pillow ==8.3.2
  • numpy ==1.19.1
  • torchvision ==0.7.0
  • tqdm ==4.48.2
openfl-workspace/torch_cnn_histology_gramine_ready/requirements.txt pypi
  • torch ==1.6.0
  • torchvision ==0.7.0
openfl-workspace/torch_cnn_mnist/requirements.txt pypi
  • tensorboard *
  • torch ==1.8.1
  • torchvision ==0.9.1
openfl-workspace/torch_cnn_mnist_straggler_check/requirements.txt pypi
  • tensorboard *
  • torch ==1.8.1
  • torchvision ==0.9.1
openfl-workspace/torch_unet_kvasir/requirements.txt pypi
  • scikit-image ==0.17.2
  • torch ==1.7.1
  • torchvision ==0.8.2
openfl-workspace/torch_unet_kvasir_gramine_ready/requirements.txt pypi
  • scikit-image ==0.17.2
  • torch ==1.7.1
  • torchvision ==0.8.2
requirements-linters.txt pypi
  • flake8 *
  • flake8-broken-line *
  • flake8-bugbear *
  • flake8-builtins *
  • flake8-comprehensions *
  • flake8-copyright *
  • flake8-docstrings *
  • flake8-eradicate *
  • flake8-import-order *
  • flake8-import-single *
  • flake8-quotes *
  • flake8-use-fstring *
  • pep8-naming *
requirements-test.txt pypi
  • pytest ==6.1.2 test
  • pytest-mock ==3.4.0 test
setup.py pypi
  • Click ==8.0.1
  • PyYAML >=5.4.1
  • cloudpickle *
  • cryptography >=3.4.6
  • docker *
  • dynaconf ==3.1.7
  • flatten_json *
  • grpcio *
  • ipykernel *
  • jupyterlab *
  • numpy *
  • pandas *
  • protobuf ==3.19.4
  • requests *
  • rich *
  • scikit-learn *
  • tensorboard *
  • tensorboardX *
  • tqdm *
tests/github/interactive_api_director/experiments/pytorch_kvasir_unet/envoy/sd_requirements.txt pypi
  • numpy * test
  • pillow * test
  • scikit-image * test
tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy/sd_requirements.txt pypi
  • tensorflow * test
.github/workflows/dockerization.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/double_ws_export.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/experimental_workflow_tests.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/fets-challenge.yml actions
  • actions/checkout v3 composite
  • actions/checkout master composite
  • actions/setup-python v3 composite
.github/workflows/interactive-kvasir.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/interactive-tensorflow.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/lint.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/pki.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/pytest_coverage.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/straggler-handling.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/taskrunner.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/taskrunner_eden_pipeline.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/ubuntu.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/windows.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
openfl-tutorials/experimental/Global_DP/requirements_global_dp.txt pypi
  • cloudpickle *
  • matplotlib ==3.6.0
  • numpy ==1.23.3
  • opacus ==1.2.0
  • pillow ==9.3.0
  • pyyaml ==6.0
  • torch ==1.13.1
  • torchvision ==0.13.1
openfl-tutorials/experimental/Privacy_Meter/requirements_privacy_meter.txt pypi
  • cloudpickle *
  • matplotlib *
  • numpy ==1.23.5
  • opacus ==1.3.0
  • pillow *
  • scikit-learn *
  • torch ==1.13.1
  • torchvision ==0.14.0
openfl-tutorials/experimental/requirements_workflow_interface.txt pypi
  • dill ==0.3.6
  • metaflow ==2.7.15
  • ray ==2.2.0
openfl-tutorials/interactive_api/Flax_CNN_CIFAR/requirements.txt pypi
  • jax ==0.3.13
  • jaxlib ==0.3.10
  • tensorflow ==2.9.3
  • tensorflow-datasets ==4.6.0
openfl-tutorials/interactive_api/HPU/PyTorch_Kvasir_UNet/envoy/sd_requirements.txt pypi
  • numpy *
  • pillow *
openfl-tutorials/interactive_api/HPU/PyTorch_MedMNIST_2D/envoy/requirements.txt pypi
  • medmnist *
  • setuptools >=65.5.1
  • wheel >=0.38.0
openfl-tutorials/interactive_api/HPU/PyTorch_TinyImageNet/envoy/requirements.txt pypi
  • Pillow ==9.3.0
openfl-workspace/torch_cnn_mnist_eden_compression/requirements.txt pypi
  • tensorboard *
  • torch ==1.13.1
  • torchvision ==0.14.1
  • wheel >=0.38.0
.github/workflows/scans.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v3 composite
.github/workflows/trivy.yml actions
  • actions/checkout v3 composite
  • aquasecurity/trivy-action master composite
  • github/codeql-action/upload-sarif v2 composite
openfl-tutorials/interactive_api/PyTorch_FedProx_MNIST/envoy/requirements.txt pypi
  • medmnist *
  • setuptools >=65.5.1
  • wheel >=0.38.0
openfl-tutorials/interactive_api/PyTorch_LinearRegression/envoy/requirements.txt pypi
  • mistune >=2.0.3
  • numpy >=1.13.3
  • openfl >=1.2.1
  • scikit-learn >=0.24.1
  • setuptools >=65.5.1
  • torch >=1.13.1
  • wheel >=0.38.0
openfl-tutorials/interactive_api/PyTorch_LinearRegression/workspace/requirements.txt pypi
  • jupyterlab *
  • numpy >=1.13.3
  • openfl >=1.2.1
  • scikit-learn >=0.24.1
  • setuptools >=65.5.1
  • torch >=1.13.1
  • wheel >=0.38.0
openfl-tutorials/interactive_api/scikit_learn_linear_regression/envoy/requirements.txt pypi
  • matplotlib >=2.0.0
  • mistune >=2.0.3
  • numpy >=1.13.3
  • openfl >=1.2.1
  • scikit-learn >=0.24.1
  • setuptools >=65.5.1
  • wheel >=0.38.0
openfl-tutorials/interactive_api/scikit_learn_linear_regression/workspace/requirements.txt pypi
  • matplotlib >=2.0.0
  • mistune >=2.0.3
  • numpy >=1.13.3
  • openfl >=1.2.1
  • scikit-learn >=0.24.1
  • setuptools >=65.5.1
  • wheel >=0.38.0
openfl-workspace/default/requirements.txt pypi
openfl-workspace/gandlf_seg_test/requirements.txt pypi
  • onnx ==1.13.0 test
openfl-workspace/torch_cnn_histology/requirements.txt pypi