fl-demonstrator

A set of tools for demonstrating machine learning in a federated environment

https://github.com/dlr-ki/fl-demonstrator

Science Score: 75.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 2 DOI reference(s) in README
  • Academic publication links
    Links to: zenodo.org
  • Academic email domains
  • Institutional organization owner
    Organization dlr-ki has institutional domain (www.dlr.de)
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (16.4%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

A set of tools for demonstrating machine learning in a federated environment

Basic Info
Statistics
  • Stars: 2
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 2
Created almost 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme License Citation

README.md

Federated Learning Demonstrator

DOI

This repository contains the Federated Learning Demonstrator, a comprehensive suite of tools designed for machine learning applications in a Federated context. This server component that handles the orchestration and notification of all training participants and ensuring smooth and efficient operation.

The demonstrator further provides capabilities for diverse model aggregation methods, merging the model inputs from each participant. It also offers model inference, enabling the generation of predictions using the trained models. The repository also includes utilities for quantifying uncertainty, which provide a measure of the reliability of the model's predictions.

This project is the server component of the Federated Learning (FL) platform, serving as a proof of concept for the Catena-X project. The FL platform aims to demonstrate the potential of federated learning in a practical, real-world context.

For a comprehensive understanding of the FL platform, please refer to the official FL platform documentation.

A complete list of all repositories relevant to the FL platform can be found here.

Get Started

This README.md is primarily intended for developers and contributors, providing necessary information for setup, installation, and contribution guidelines. If you're interested in using or testing this project, we recommend starting with the GitHub pages. They offer a more user-friendly interface and comprehensive guides to get you started.

Requirements

Python > 3.10

```bash sudo apt install python

If not available you can add latest versions with

sudo add-apt-repository ppa:deadsnakes/ppa ```

Venv

```bash

Install venv

sudo apt install python-venv ```

Virtualenv (alternative to venv)

```bash

python 3.10 or later

which python

virtualenv or venv

pip install -U virtualenv ```

Install

```bash

create virtual environment

virtualenv -p $(which python) .venv

or

python -m venv .venv

activate our virtual environment

source .venv/bin/activate

update pip (optional)

python -m pip install -U pip

install

./dev install -U -e ".[all]" ```

Helpers

```txt $ ./dev --help usage: ./dev [options]

positional arguments: {celery,clean,collectstatic,coverage,coverage-report,db-reset,doc,doc-build,docker-build,help,install,licenses,licenses-check,lint,lint-code,lint-doc,lint-scripts,makemigrations,manage,migrate,mypy,safety-check,start,superuser,test,version,versions} Available sub commands help Show this help message and exit start Run the application celery Run celery worker migrate Run database migrations makemigrations Create new database migrations manage Run django manage.py superuser Create superuser collectstatic Collect static files db-reset Reset database docker-build Build docker images for local development test Run all tests lint Run all linter lint-code Run code linter lint-doc Run documentation linter lint-scripts Run bash script linter mypy Run type checker coverage Run unit tests coverage-report Generate test coverage report doc Start documentation server doc-build Build documentation licenses Generate licenses licenses-check Check licenses safety-check Check dependencies for known security vulnerabilities install Install package clean Clean up local files version Show package version versions Show versions

options: --no-http-serve Do not serve the action result via HTTP ```

Contribution

  • Type-Save and linting with mypy+flake8
  • Scripts and examples for linux, wsl (bash)

Documentation

This projects is using the Docstring style from Google. At least public classes, methods, fields, ... should be documented.

```python """ This is the single line short description.

This is the multiline or long description. Note, that the whole Docstring support markdown styling.

The long description can also contains multiple paragraphs.

Args: logfilepath (str): Log file path. ensurelog_dir (bool, optional): Create directory for the log file if not exists. Defaults to True.

Returns: Dict[str, Any]: logging configuration dict """ ```

Credits

DLR DLR
Catena-X Catena-X
European Union European Union BMWK

Owner

  • Name: DLR Institute of AI Safety and Security
  • Login: DLR-KI
  • Kind: organization

Citation (CITATION.cff)

# SPDX-FileCopyrightText: 2024 Benedikt Franke <benedikt.franke@dlr.de>
# SPDX-FileCopyrightText: 2024 Florian Heinrich <florian.heinrich@dlr.de>
#
# SPDX-License-Identifier: Apache-2.0

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: DLR Federated Learning Logging Base
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Florian
    family-names: Heinrich
    email: florian.heinrich@dlr.de
    affiliation: German Aerospace Center (DLR)
    orcid: 'https://orcid.org/0009-0007-0792-7908'
  - given-names: Benedikt
    family-names: Franke
    email: benedikt.franke@dlr.de
    affiliation: German Aerospace Center (DLR)
    orcid: 'https://orcid.org/0000-0003-0202-2804'
identifiers:
  - type: doi
    value: 10.5281/zenodo.13951205  
repository-code: 'https://github.com/DLR-KI/fl-demonstrator'
url: 'https://dlr-ki.github.io/fl-demonstrator/'
keywords:
  - Federaed Machine Learning
  - Python
date-released: 2024-07-31

GitHub Events

Total
  • Release event: 1
  • Push event: 7
  • Create event: 1
Last Year
  • Release event: 1
  • Push event: 7
  • Create event: 1

Dependencies

docker/celery/Dockerfile docker
  • python 3.10-slim build
docker/django/Dockerfile docker
  • python 3.10-slim build
docker-compose.yml docker
  • ghcr.io/dlr-ki/fl-demonstrator main
  • ghcr.io/dlr-ki/fl-demonstrator-celery main
  • postgres latest
  • redis latest
pyproject.toml pypi
  • Faker ~=20.1.0
  • celery ~=5.3.0
  • django ~=4.0.0
  • django-cors-headers ~=4.1.0
  • django-polymorphic ~=3.1.0
  • django-redis ~=5.2.0
  • djangorestframework ~=3.14.0
  • dlr-logging @ git+https://github.com/DLR-KI/fl-logging-base.git@main
  • docstring-parser ~=0.15
  • drf-spectacular ~=0.26.3
  • marshmallow ~=3.19.0
  • numpy ~=1.24.3
  • psycopg2-binary ~=2.9.6
  • requests ~=2.31.0
  • torch ~=2.0.1
.github/workflows/main.yml actions
  • actions/checkout v4 composite
  • actions/deploy-pages v3 composite
  • actions/setup-node v4 composite
  • actions/setup-python v5 composite
  • actions/upload-pages-artifact v2 composite
  • docker/build-push-action v5 composite
  • docker/login-action v3 composite
  • docker/metadata-action v5 composite
  • postgres latest docker
  • redis latest docker