TemporalContext-BayesDL

Enhancing Time-Series Prediction with Temporal Context Modeling: A Bayesian and Deep Learning Synergy

Table of Contents

• Description
• Dependencies
• Installation
• Usage
• Models
• Example
• Contributing
• Citation

Description

This repository contains Python code using a combination of Deep Learning and Bayesian Models. Alongside training a deep learning model, we construct a Conditional Probability Table (CPT) during training to capture label transitions. During inference, these CPTs are utilized to adjust the predicted class probabilities of each window, taking into account the predictions of preceding windows. Our experimental analysis, focused on Human Activity Recognition (HAR) time series datasets, demonstrates that this approach not only surpasses the baseline performance of standalone deep learning models but also outperforms contemporary state-of-the-art methods that integrate temporal context into time series prediction.

Dependencies

• Python 3.10
• TensorFlow 2.x
• NumPy
• SciPy
• Matplotlib
• Seaborn
• Pandas
• Scikit-learn
• tsai
• PyTorch

Installation

To install and run Temporal Context in Bayesian Deep Learning, follow these steps:

```bash git clone https://github.com/imics-lab/TemporalContext-BayesDL.git cd TemporalContext-BayesDL conda env create -f environment.yml

```

Usage

To use this project, run the main script after installation:

```bash python scripts/main.py

```

Models

• Baseline CNN
• BayesDL
• InceptionTime
• TST
• LSTNet

Example

Training:

```python from loaddata import getdataset from BayesMethod import learn_cpts

xtrain, ytrain, xvalid, yvalid, xtest, ytest, ksize, EPOCHS, tnames = getdataset(dataset) y = np.argmax(ytrain, axis=-1) k = 20 # Number of previous states to consider cpts = learncpts(y, k) # Learning CPTs with open('cpts.pickle', 'wb') as handle: pickle.dump(cpts, handle, protocol=pickle.HIGHESTPROTOCOL) #save the CPTs from the training phase and use them later in the inference phase ``` Inference:

```python from BayesMethod import Bayesianprobabilities, combineprobabilities from utils import tunelambdavalue

with open('cpts.pickle', 'rb') as handle: cpts = pickle.load(handle) numclasses = ytrain.shape[1] # Number of classes sequence = ytest dlprobs = loadedprobabilities[dataset] # Deep learning probabilities lambdavalues = np.linspace(0, 1, 11) # Example list of lambda values lambdavalue = tunelambdavalue(xvalid, yvalid, cpts, dlprobsvalid, lambdavalues) bayesianprobs = Bayesianprobabilities(cpts, sequence, numclasses) # Calculating Bayesian probabilities combinedprobs = combineprobabilities(dlprobs, bayesianprobs, lambdavalue) # Combining probabilities ```

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

Citation

bibtex @inproceedings{irani2024enhancing, title={Enhancing Time-Series Prediction with Temporal Context Modeling: A Bayesian and Deep Learning Synergy}, author={Irani, Habib and Metsis, Vangelis}, booktitle={The International FLAIRS Conference Proceedings}, volume={37}, year={2024} }