deeptsf

The DeepTSF time series forecasting repository developed by EPU NTUA within the DeployAI project

https://github.com/epu-ntua/deeptsf

Science Score: 57.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
    Found 7 DOI reference(s) in README
  • Academic publication links
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (15.7%) to scientific vocabulary

Keywords

darts deep-learning docker machine-learning mlflow mlops time-series-forecasting
Last synced: 6 months ago · JSON representation ·

Repository

The DeepTSF time series forecasting repository developed by EPU NTUA within the DeployAI project

Basic Info
  • Host: GitHub
  • Owner: epu-ntua
  • License: gpl-3.0
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 80.4 MB
Statistics
  • Stars: 2
  • Watchers: 2
  • Forks: 1
  • Open Issues: 0
  • Releases: 2
Topics
darts deep-learning docker machine-learning mlflow mlops time-series-forecasting
Created over 1 year ago · Last pushed 7 months ago
Metadata Files
Readme License Citation

README.md

DeepTSF

✨ DeepTSF is designed to enable codeless machine learning operations for time series forecasting ✨

🙌 Refer to https://github.com/epu-ntua/DeepTSF/wiki/DeepTSF-documentation for the documentation 📖

License: GPL v3 [DOI](https://doi.org/10.1016/j.softx.2024.101758)

Installation

To set up DeepTSF on your local system, you need clone the main branch of this repository:

git clone https://github.com/epu-ntua/DeepTSF.git

Alternatively you can use the dedicated Github release instead of cloning the main branch.

After that you need to navigate to the root directory of DeepTSF:

cd /path/to/repo/of/DeepTSF

Το enable the communication of the client with the logging servers (MLflow, Minio, Postgres), a .env file is needed. An example (.env.example) is provided, with default environment variables.

After that, you can set up a full deployment of DeepTSF using Docker.

Set up locally using Docker

To set up locally using docker first go to DeepTSF's root directory (inside deeptsf_backend) and rename .env.example to .env. Then run the following command in DeepTSF's root directory:

docker-compose up

DeepTSF is up and running. Navigate to http://localhost:3000 and start your experiments!

Dagster UI for advanced users

For users that require advanced pipeline parameterization and functionalities such as hyperparameter tuning, a dagster based pipeline is provided. By modifying the config of deeptsfdagsterjob, the user can set all parameters described in the extensive documentation of DeepTSF. An example config file is given below:

resources: config: config: a: 0.3 analyze_with_shap: false convert_to_local_tz: true country: PT cut_date_test: "20210101" cut_date_val: "20200101" darts_model: LightGBM database_name: rdn_load_data device: gpu eval_method: ts_ID eval_series: eval_series evaluate_all_ts: true experiment_name: dagster_test forecast_horizon: 24 format: long from_database: false future_covs_csv: None future_covs_uri: None grid_search: false hyperparams_entrypoint: lags: [-1, -2, -14] ignore_previous_runs: true imputation_method: linear loss_function: mape m_mase: 1 max_thr: -1 min_non_nan_interval: 24 multiple: false n_trials: 100 num_samples: 1 num_workers: 4 opt_test: false order: 1 parent_run_name: dagster_test past_covs_csv: None past_covs_uri: None pv_ensemble: false resampling_agg_method: averaging resolution: 1h retrain: false rmv_outliers: true scale: true scale_covs: true series_csv: dataset-storage/Italy.csv series_uri: None shap_data_size: 100 shap_input_length: -1 std_dev: 4.5 stride: -1 test_end_date: None time_covs: false ts_used_id: None wncutoff: 0.000694 ycutoff: 3 ydcutoff: 30 year_range: None

For a more complete guide check the extensive documentation.

This application can also be deployed in a kubernetes enviroment.

Set up mlflow tracking server

To run DeepTSF on your system you first have to install the mlflow tracking and minio server.

git clone https://github.com/epu-ntua/mlflow-tracking-server.git

cd mlflow-server

After that, you need to get the server to run

docker-compose up

The MLflow server and client may run on different computers. In this case, remember to change the addresses on the .env file.

For the extensive DeepTSF documentation please navigate to our Wiki.

📺 DeepTSF — Video Demonstration

Also, a video demonstration of DeepTSF is avaialble on Youtube.

Watch on YouTube

References

[1] S. Pelekis et al., “DeepTSF: Codeless machine learning operations for time series forecasting,” SoftwareX, vol. 27, p. 101758, Sep. 2024, doi: 10.1016/J.SOFTX.2024.101758.

Owner

  • Name: EPU NTUA
  • Login: epu-ntua
  • Kind: organization
  • Location: Athens

The Decision Support Systems Laboratory of the National Technical University of Athens, Greece.

Citation (CITATION.cff)

@article{PELEKIS2024101758,
title = {DeepTSF: Codeless machine learning operations for time series forecasting},
journal = {SoftwareX},
volume = {27},
pages = {101758},
year = {2024},
issn = {2352-7110},
doi = {https://doi.org/10.1016/j.softx.2024.101758},
url = {https://www.sciencedirect.com/science/article/pii/S2352711024001298},
author = {Sotiris Pelekis and Theodosios Pountridis and Georgios Kormpakis and George Lampropoulos and Evangelos Karakolis and Spiros Mouzakitis and Dimitris Askounis},
keywords = {Codeless, Deep learning, Machine learning operations, Time series forecasting},
abstract = {This paper presents DeepTSF, a comprehensive machine learning operations (MLOps) framework aiming to innovate time series forecasting through workflow automation and codeless modeling. DeepTSF automates key aspects of the machine learning (ML) lifecycle, making it an ideal tool for data scientists and MLops engineers engaged in ML and deep learning (DL)-based forecasting. DeepTSF empowers users with a robust and user-friendly solution, while it is designed to seamlessly integrate with existing data analysis workflows, providing enhanced productivity and compatibility. The framework offers a front-end user interface (UI) suitable for data scientists, as well as other higher-level stakeholders, enabling comprehensive understanding through insightful visualizations and evaluation metrics. DeepTSF also prioritizes security through identity management and access authorization mechanisms. The application of DeepTSF in real-life use cases of the I-NERGY project has already proven DeepTSF’s efficacy in DL-based load forecasting, showcasing its significant added value in the electrical power and energy systems domain.}
}

GitHub Events

Total
  • Watch event: 1
  • Push event: 31
  • Gollum event: 26
  • Pull request event: 1
  • Create event: 3
Last Year
  • Watch event: 1
  • Push event: 31
  • Gollum event: 26
  • Pull request event: 1
  • Create event: 3

Dependencies

Dockerfile docker
  • nvidia/cuda 11.6.2-base-ubuntu20.04 build
dagster-deeptsf/config/Dockerfile docker
  • continuumio/miniconda3 23.5.2-0 build
docker-compose-keycloak.yml docker
docker-compose.yml docker
  • inergy2020iccs/deeptsf_dashboard 1.0.2
  • inergy2020iccs/mlflow_server 2.11.3
  • inergy2020iccs/pgdb 16.2
  • minio/minio RELEASE.2024-03-30T09-41-56Z
dagster-deeptsf/pyproject.toml pypi
dagster-deeptsf/setup.py pypi
  • dagster *
  • dagster-cloud *