New Features

Add documentation for CADRE (Customizable Assurance of Data Readiness) modules
Add tutorials for running APPFL on FLamby datasets, both on AWS and HPCs
Add tutorials for Globus-based authentication

- Python
Published by Zilinghan 8 months ago

New Features

Add data readiness agent to APPFL for the assurance of AI data readiness before FL training.
Support running APPFL tutorials on AWS SageMaker.
Integrate APPFL github workflow with ALCF Gitlab CI for testing on Polaris with GPUs.

- Python
Published by Zilinghan 10 months ago

New Features

Add a new tutorial for using shared Globus Compute endpoints at here.
Add a new tutorial for joining APPFL-hosted international federation at here.
Support colab notebooks for running APPFL on Google Colab with a colab connector for easy data transmission.

- Python
Published by Zilinghan 11 months ago

New Features

Add Ray into communicator, with documentation available here. (#271)
Make mpi4py optional during installation. (#264)

Deprecated

Deprecate the usage of comm_configs.globus_compute_configs for AWS S3 configurations, which is replaced by comm_configs.s3_configs.

- Python
Published by Zilinghan 12 months ago

New Features

Make MPI installation optional

- Python
Published by Zilinghan 12 months ago

New Features

Integrate APPFL with MONAI to use MONAI bundles for federated learning, with documentation available here. #251
Add support for Multi-GPU training using PyTorch DDP, with documentation available here. #254
Integrate ProxyStore into Globus Compute and gRPC communication protocols for data transmission, with documentation available here. #252, #259
Add three colab-based tutorials at here for running APPFL on Google Colab #255

Full Changelog: https://github.com/APPFL/APPFL/compare/v1.2.1...v1.3.0

- Python
Published by Zilinghan about 1 year ago

New Features

Enhance safety for Globus Compute by only sending a trigger function. #227
Remove redundant experiment configurations. #228

Full Changelog: https://github.com/APPFL/APPFL/compare/v1.2.0...v1.2.1

- Python
Published by Zilinghan about 1 year ago

New Features

Improve client name display for running FL experiments by specifying client_id in the client configuration file.
Add documentation for using APPFL on ALCF Polaris at here.
Allow users to send payload of arbitrary size for custom actions in gRPC communication.
Add more tests for FL experiments under different scenarios: serial, MPI, batched MPI, and gRPC.
Integrate wandb for logging training metadata such as training and validation losses into client trainer, with documentation available here.

Bug Fixes

Fix path issues when running APPFL on Windows.
Fix batched MPI issue with compression.
Fix some other small bugs and bump the version of few dependencies.

- Python
Published by Zilinghan about 1 year ago

New Features

Support batched MPI, with documentation available here.
Add more data readiness metrics such as PCA plot in PR #208
Backend support for service.appfl.ai.
Add documentation for service.appfl.ai at here.
Add logging capabilities to the server side to log the training metadata such as the training and validation losses.
Change documentation theme to furo.

Community Standards

Add pull request template and issue templates
Add contribution guidance
Add dependabot for auto github action version check

- Python
Published by Zilinghan about 1 year ago

New Features

Add the feature to generate data readiness reports on all client data by from @kaveenh (PR #202)
Update the documentation for adding custom action at here.

- Python
Published by Zilinghan over 1 year ago

New Features

Add documentation for using APPFL with Globus Compute for secure distributed training at here.

Bug Fixes

Fix an issue with Globus Compute at this commit.

- Python
Published by Zilinghan over 1 year ago

New Features

Add trackback information to the gRPC server to help debug the server-side errors.
Add a video tutorials for installing APPFL on AWS, creating SSL-encrypted gRPC server, and using APPFL to finetune a ViT.

Bug Fixes

Handle corner cases for server aggregators when the keys in client local models are not consistent with the global model keys.

- Python
Published by Zilinghan over 1 year ago

New Features

Add a new command line interface (CLI), appfl-setup-ssl, to create necessary certificates for creating SSL-secured gRPC connections between the server and clients.
Add a tutorial on how to use the CLI, appfl-setup-ssl to create certificates for the server and clients, and enable SSL-secured gRPC connections between the server and clients at here.
Add a detailed step-by-step tutorial on how to define custom action with an example to generate a data readiness report on all client data at here.
Add a APPFL YouTube channel to provide video tutorials on how to use APPFL for federated learning research in the future.

Bug Fixes

Fix the issue regarding client gradient clipping. The clipping is now applied before weights update.

- Python
Published by Zilinghan over 1 year ago

New Features

For the aggregators, the model architecture is set to be an optional initialization parameter, and the aggregators only aggregate the parameters sent by the clients instead of the whole set of model parameters. This is useful when doing federated fine-tuning or federated transfer learning where only part of model parameters are updated / the model architecture is unknown to the aggregator.
Support easy integration of custom trainer/aggregator: user only needs to provide the custom trainer/aggregator class name and the path to the definition file in the configuration file to use it, instead of modifying the source code.
Add a detailed step-by-step tutorial on how to use APPFL to fine-tune a ViT model with a custom trainer at https://appfl.ai/en/latest/tutorials/examplesvitfinetuning.html

- Python
Published by Zilinghan over 1 year ago

Overview

Version 1.0.0 of appfl is a major release that refactors the entire codebase to make it more modular, extensible, and functional, while remains backward compatibility with the previous version. The release also included the following changes:

New Features

Define server and client agents to act on behalf of the FL server and clients to conduct FL experiments.
Simplify the configuration process for launching FL experiments by only providing a single YAML file for the server and a YAML file for each client.
Rebuild the communicator module, supporting MPI, gRPC, and Globus Compute, to robustly exchange model parameters as well as task metadata between the server and clients in both synchronous and asynchronous FL experiment settings.
Implement Globus-based authentication for secure distributed training with gRPC and Globus Compute - only members within the same specific Globus group can participate in the FL experiment.
Integrate several lossy and error-bounded lossless compressors to the communicator module for efficient model compression.

Add documentation for the new version available at appfl.ai

Deprecated

The previous version of appfl is still seamlessly supported but deprecated and no longer maintained. Users are encouraged to upgrade to the new version for better performance, functionality, and extensibility. Examples and tutorials for the previous version are still available in the examples/examples_legacy directory of the Github appfl repository.

- Python
Published by Zilinghan over 1 year ago

appfl - v0.4.2

What's Changed

Data readiness report generation by @kaveenh in https://github.com/APPFL/APPFL/pull/163
Fix Issue #164 by @Zilinghan in https://github.com/APPFL/APPFL/pull/165
Make dr_metrics optional by @Zilinghan in https://github.com/APPFL/APPFL/pull/168
Major updates to docs by @kibaekkim in https://github.com/APPFL/APPFL/pull/172
Easy installation of SZ compressor using appfl-install-compressor by @Zilinghan

New Contributors

@kaveenh made their first contribution in https://github.com/APPFL/APPFL/pull/163

Full Changelog: https://github.com/APPFL/APPFL/compare/v0.4.1...v0.4.2

- Python
Published by Zilinghan almost 2 years ago

appfl - v0.4.1

What's Changed

Integrate authenticators (NaiveAuthenticator and GlobusAuthenticator) for gRPC communication protocol for secure federation #162
Add documentations for authentication designs
Add documentations for asynchronous FL algorithms

Full Changelog: https://github.com/APPFL/APPFL/compare/v0.4.0...v0.4.1

- Python
Published by Zilinghan about 2 years ago

appfl - v0.4.0

What's Changed

Add four variants of lossy compressors (SZ2, SZ3, SZx, ZFP) to improve FL communication efficiency #155 from @grantwilkins
Resolve file system race condition for globus compute #158 from @yim0331
Resolve invalid load key error for MPI communicator in https://github.com/APPFL/APPFL/commit/d5f2573beb86e816746b5180d1e692f90630335b by @Zilinghan

- Python
Published by Zilinghan about 2 years ago

appfl - v0.3.2

Resolve a new bug ModuleNotFoundError: No module named 'appfl.comm.globus_compute.utils'

- Python
Published by Zilinghan over 2 years ago

appfl - v0.3.1

What's Changed

In summary, this is a minor release mainly with the focus on resolving the problem that appfl.comm is not in the current released version, together with few other minor changes with details below

Rename the package from Argonne Privacy-Preserving Federated Learning to Advanced Privacy-Preserving Federated Learning #154
Resolve the device error when using DP on GPU #150
Resolve appfl.comm ModuleNotFoundError #156

- Python
Published by Zilinghan over 2 years ago

appfl -

What's Changed

In summary, this is a release of a major change of the APPFL repository with the refactor of the codebase and the addition of several new capabilities. In details: * Add examples on CELEBA and FEMNIST datasets with a new MPI communicator for large models by @yim0331 * Add asynchronous FL algorithms FedAsync, FedBuffer, and FedCompass by @Zilinghan and @ShellyRiver * Add example for personalized FL by @shourya01 * Add globus compute (formerly funcX) as a communicator by @Zilinghan and @hthieu166 * Allow use to use custom loss and custom evaluation metric in the FL experiments by @Zilinghan * Document improvements by @Zilinghan and @minseok-ryu

- Python
Published by Zilinghan over 2 years ago

appfl - v0.2.1

What's Changed

Inititalised test_dataloader to None by @samg2003 in https://github.com/APPFL/APPFL/pull/87
Adding example by @samg2003 in https://github.com/APPFL/APPFL/pull/88
Fix the batch normalization layers updates and training loss computation by @hthieu166, @minseok-ryu in https://github.com/APPFL/APPFL/pull/86
Fix the initial point by @hthieu166, @minseok-ryu in https://github.com/APPFL/APPFL/pull/98

Full Changelog: https://github.com/APPFL/APPFL/compare/v0.2.0...v0.2.1

- Python
Published by minseok-ryu over 3 years ago

appfl - v0.2.0

What's Changed

Revise run_grpc_client.py by @minseok-ryu in https://github.com/APPFL/APPFL/pull/68
Add Federated Adaptive Methods at Server-Side by @minseok-ryu in https://github.com/APPFL/APPFL/pull/59
Additional Clean Up by @minseok-ryu in https://github.com/APPFL/APPFL/pull/70
add tensorboard by @HongdaChen in https://github.com/APPFL/APPFL/pull/76
Revisit the validation by @minseok-ryu in https://github.com/APPFL/APPFL/pull/72
Add a tensorboard contributed by HongdaChen by @minseok-ryu in https://github.com/APPFL/APPFL/pull/77
Clean up by @minseok-ryu in https://github.com/APPFL/APPFL/pull/79
customized loss function by @minseok-ryu in https://github.com/APPFL/APPFL/pull/82
Documentation and release for 0.2.0 by @kibaekkim in https://github.com/APPFL/APPFL/pull/83

New Contributors

@HongdaChen made their first contribution in https://github.com/APPFL/APPFL/pull/76

Full Changelog: https://github.com/APPFL/APPFL/compare/v0.1.2...v0.2.0

- Python
Published by kibaekkim over 3 years ago

appfl - v0.1.2

Corrected a version tag

Full Changelog: https://github.com/APPFL/APPFL/compare/v0.1.1...v0.1.2

- Python
Published by kibaekkim almost 4 years ago

appfl - v0.1.1

What's Changed

Generalize the loss function by @minseok-ryu in https://github.com/APPFL/APPFL/pull/62
update document for the loss function by @minseok-ryu in https://github.com/APPFL/APPFL/pull/63
Revise the code for training with "torch.nn.BCELoss()" by @minseok-ryu in https://github.com/APPFL/APPFL/pull/64

Full Changelog: https://github.com/APPFL/APPFL/compare/v0.1.0...v0.1.1

- Python
Published by kibaekkim almost 4 years ago

appfl - v0.1.0

What's Changed

Loading and Saving PyTorch Model; Use of logging by @minseok-ryu in https://github.com/APPFL/APPFL/pull/50
Updates to gRPC with testing on Google Cloud Platform by @kibaekkim in https://github.com/APPFL/APPFL/pull/54

Full Changelog: https://github.com/APPFL/APPFL/compare/v0.0.1...v0.1.0

- Python
Published by kibaekkim almost 4 years ago

appfl - v0.0.1

What's Changed

Initial design by @kibaekkim in https://github.com/APPFL/APPFL/pull/1
first doc by @kibaekkim in https://github.com/APPFL/APPFL/pull/5
Documentation by @kibaekkim in https://github.com/APPFL/APPFL/pull/7
Major development and redesign by @minseok-ryu in https://github.com/APPFL/APPFL/pull/8
Refactoring the data reading part and documentation by @minseok-ryu in https://github.com/APPFL/APPFL/pull/11
Add differential privacy by @minseok-ryu in https://github.com/APPFL/APPFL/pull/16
Added ICEADMM, IIADMM, and client weight computation by @minseok-ryu in https://github.com/APPFL/APPFL/pull/17
Improve IIADMM by introducing adaptive penalty by @minseok-ryu in https://github.com/APPFL/APPFL/pull/25
Restructuring directories and setuptools for PyPI by @minseok-ryu in https://github.com/APPFL/APPFL/pull/26
Differential privacy using the gradient clipping by @minseok-ryu in https://github.com/APPFL/APPFL/pull/31
Add residual calculation and adaptive penalty by @minseok-ryu in https://github.com/APPFL/APPFL/pull/33
Replacing the configuration system by @kibaekkim in https://github.com/APPFL/APPFL/pull/35
Documentation for the first release by @kibaekkim in https://github.com/APPFL/APPFL/pull/22
Fix run_serial for a single GPU computing by @minseok-ryu in https://github.com/APPFL/APPFL/pull/41
fixed #43 by @kibaekkim in https://github.com/APPFL/APPFL/pull/44
Adding initial PyTest by @kibaekkim in https://github.com/APPFL/APPFL/pull/46
Typo in documentation? by @minseok-ryu in https://github.com/APPFL/APPFL/pull/45
Docs for how to contribute and release the package by @kibaekkim in https://github.com/APPFL/APPFL/pull/48
fixes a bug #39. by @youngdae in https://github.com/APPFL/APPFL/pull/47

New Contributors

@kibaekkim made their first contribution in https://github.com/APPFL/APPFL/pull/1
@minseok-ryu made their first contribution in https://github.com/APPFL/APPFL/pull/8
@youngdae made their first contribution in https://github.com/APPFL/APPFL/pull/47

Full Changelog: https://github.com/APPFL/APPFL/commits/v0.0.1

- Python
Published by kibaekkim about 4 years ago

Recent Releases of appfl

appfl - v1.6.1

New Features

appfl - v1.5.0

New Features

appfl - v1.4.1

New Features

appfl - v1.4.0

New Features

Deprecated

appfl - v1.3.2

New Features

appfl - v1.3.0

New Features

appfl - v1.2.1

New Features

appfl - v1.2.0

New Features

Bug Fixes

appfl - v1.1.0

New Features

Community Standards

appfl - v1.0.5

New Features

appfl - v1.0.4

New Features

Bug Fixes

appfl - v1.0.3

New Features

Bug Fixes

appfl - v1.0.2

New Features

Bug Fixes

appfl - v1.0.1

New Features

appfl - v1.0.0

Overview

New Features

Deprecated

appfl - v0.4.2

What's Changed

New Contributors

appfl - v0.4.1

What's Changed

appfl - v0.4.0

What's Changed

appfl - v0.3.2

appfl - v0.3.1

What's Changed

appfl -

What's Changed

appfl - v0.2.1

What's Changed

appfl - v0.2.0

What's Changed

New Contributors

appfl - v0.1.2

appfl - v0.1.1

What's Changed

appfl - v0.1.0

What's Changed

appfl - v0.0.1

What's Changed

New Contributors