Data

Tools

Indexes

Applications

Experiments

Open Source Science

kedro-umbrella

DesCartes Builder back-end

https://github.com/cps-research-group/kedro-umbrella

Science Score: 52.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ✓
    Institutional organization owner
    Organization cps-research-group has institutional domain (www.ntu.edu.sg)
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.6%) to scientific vocabulary
Last synced: 6 months ago · JSON representation ·

Repository

DesCartes Builder back-end

Basic Info
  • Host: GitHub
  • Owner: CPS-research-group
  • License: mit
  • Language: Python
  • Default Branch: main
  • Size: 8.87 MB
Statistics
  • Stars: 1
  • Watchers: 1
  • Forks: 0
  • Open Issues: 2
  • Releases: 0
Created 11 months ago · Last pushed 7 months ago
Metadata Files
Readme License Citation

README.md

DesCartes Builder: Back-end implementation

Overview

This package provides facilitates to specify ML pipeline as employed in Digital Twin design. This is a part of the DesCartes Builder, see [https://descartes.cnrsatcreate.cnrs.fr/wp9-augmented-engineering/]. Digital Twin (DT) often have (1) complex & diverse pipelines, but despite their variety, there are common building blocks. Besides, (2) DTs are often composed of several functions.

The Function+Data Flow (FDF) implemented here extends a traditional data-flow in two ways:

  1. incorporating functions as first-class citizens: FDF allows know whether a function or data is generated, allowing to reuse/export functions explicitly
  2. defining application-specific boxes representing different processing steps that are required for DT design.

This package is implemented leveraging Kedro for executing and describing the pipeline is a simple way. We extend the Kedro Node to implement the FDF boxes and we can use Kedro pipeline as is for execution. We also include a library of commonly used functions for each application-specific code.

Installation & development

Here are instructions to install the library. Using a conda environment is highly recommended.

```bash

Clone repo

git clone git@github.com:CPS-research-group/kedro-umbrella.git

Create development env

conda create -n builderdev python=3.10.8 conda activate builderdev

Install library

make install ```

Tests for key features are in ./tests: execute with make test.

Examples

Some examples and case studies using FDF are in the ./examples folder. Each example is documented in its own folder in a README.md file. Different variants of the case study pipeline can be executed with Makefile rules (briefly described in the Makefile itself):

  • Run all examples: cd examples && make.
  • Run a single example: cd examples/<example_dir> && make <example_rule>

Owner

  • Name: CPS@NTU
  • Login: CPS-research-group
  • Kind: organization
  • Location: Nanyang Technological University

Cyber-Physical Systems Research Group in SCSE, NTU.

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
title: DesCartes Builder

preferred-citation:
  type: conference-paper
  authors:
    - family-names: "de Conto"
      given-names: "Eduardo"
    - family-names: "Genest"
      given-names: "Blaise"
    - family-names: "Easwaran"
      given-names: "Arvind"
  title: "Function+Data Flow: A Framework to Specify Machine Learning Pipelines for Digital Twinning"
  year: 2024
  doi: "10.1145/3664646.3664759"
  isbn: "9798400706851"
  url: "https://doi.org/10.1145/3664646.3664759"
  conference:
    name: "1st ACM International Conference on AI-Powered Software"
    location: "Porto de Galinhas, Brazil"
    series: "AIware 2024"
  publisher: "Association for Computing Machinery"
  address: "New York, NY, USA"
  keywords:
    - "dataflow"
    - "digital twins"
    - "machine learning pipeline"

GitHub Events

Total
  • Issues event: 2
  • Push event: 1
  • Public event: 1
  • Create event: 1
Last Year
  • Issues event: 2
  • Push event: 1
  • Public event: 1
  • Create event: 1

Dependencies

.github/workflows/tests.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v5 composite
examples/pipe-deformation/pyproject.toml pypi
examples/pipe-deformation/src/setup.py pypi
examples/spring-system/pyproject.toml pypi
examples/spring-system/src/pyproject.toml pypi
kedro_umbrella/setup.py pypi
kedro_umbrella/template/builder-spring/{{ cookiecutter.repo_name }}/pyproject.toml pypi
kedro_umbrella/template/builder-spring/{{ cookiecutter.repo_name }}/src/pyproject.toml pypi
pyproject.toml pypi
requirements.txt pypi
  • black *
  • captum ==0.7.0
  • casadi ==3.6.7
  • cvxpy ==1.6.0
  • cvxpylayers ==0.1.9
  • dill ==0.3.9
  • flake8 >=3.7.9,<5.0
  • fmpy ==0.3.18
  • graphviz ==0.20.3
  • ipython >=7.31.1,<8.0
  • ipython *
  • isort *
  • jupyter *
  • jupyterlab *
  • jupyterlab_server >=2.11.1,<2.16.0
  • kedro ==0.19.10
  • kedro-datasets *
  • kedro-viz *
  • lightning ==2.5.0.post0
  • lightning-utilities ==0.11.9
  • mat73 ==0.62
  • matplotlib ==3.8.4
  • nbstripout *
  • neuromancer ==1.5.1
  • numpy ==1.23.5
  • optuna ==3.6.1
  • pandas ==2.2.2
  • pre-commit >=2.9.2
  • pytest *
  • pytest-cov *
  • pytest-mock >=1.7.1,<2.0
  • pythonfmu3 ==0.1.11
  • pyts ==0.13.0
  • scikit-learn ==1.4.0
  • scipy ==1.15.1
  • setuptools ==67.7.2
  • six ==1.17.0
  • toml ==0.10.2
  • torch ==2.5.1
  • torchdiffeq ==0.2.5
  • torchmetrics ==1.6.1
  • torchsde ==0.2.6
  • wandb ==0.18.7