transtimegrad

TransTimeGrad is a PyTorch probabilistic time series model which is a transformer based version of the TimeGrad model. It utilizes GluonTS backend API.

Quick Start

Installation

To install the TransTimeGrad library with pip, you can simply proceed with

bash pip install transtimegrad

Also you may want to download the library directly from the GitHub repo as follows

bash pip install git+https://github.com/nbukhanchenko/transtimegrad

Usage

To build a trained model you may follow the pipepline from examples/experiments.ipynb. Otherwise, proceed with following steps: 1. Prepare your datasets: * if you want to use ready-to-go datasets, exploit get_dataset from GluonTS, for instance, get_dataset("solar_nips") in order to get the Solar dataset * split your data into train and test with MultivariateGrouper from GluonTS * an example of your code might look something like this:

```python def preparedataset(datasetname): # you don't need this line if you already have a dataset dataset = getdataset(datasetname, regenerate=False)

train_grouper = MultivariateGrouper(
    max_target_dim=int(dataset.metadata.feat_static_cat[0].cardinality)
)
test_grouper = MultivariateGrouper(
    num_test_dates=int(len(dataset.test) / len(dataset.train)),
    max_target_dim=int(dataset.metadata.feat_static_cat[0].cardinality),
)

return {
    "train": train_grouper(dataset.train),
    "test": test_grouper(dataset.test),
    "metadata": dataset.metadata
}

```

1. Prepare your predictor:
2. choose the model to work with: rnn_timegrad (standard TimeGrad model) or trans_timegrad (our newly proposed TransTimeGrad model)
3. choose scheduler from diffusers library, for instance DEISMultistepScheduler
4. specify parameters of scheduler and estimator
5. an example of your code might look something like this:

```python def preparepredictor(dataset, mode, maxepochs, accelerator, numtraintimesteps=100, betastart=1e-4, betaend=0.1, betaschedule="linear", contextlengthcoef=3, numlayers=2, hiddensize=64, lr=1e-3, weightdecay=1e-8, dropoutrate=0.1, lagsseq=[1, 7, 14, 24, 168], numinferencesteps=99, batchsize=64, numbatchesperepoch=100): model = { "rnntimegrad": TimeGradEstimator, "transtimegrad": TransTimeGradEstimator, }

scheduler = DEISMultistepScheduler(
    num_train_timesteps=num_train_timesteps,
    beta_start=beta_start,
    beta_end=beta_end,
    beta_schedule=beta_schedule,
)

estimator = model[mode](
    freq=dataset["metadata"].freq,
    prediction_length=dataset["metadata"].prediction_length,
    input_size=int(dataset["metadata"].feat_static_cat[0].cardinality),
    scheduler=scheduler,
    context_length=context_length_coef*dataset["metadata"].prediction_length,
    num_layers=num_layers,
    hidden_size=hidden_size,
    lr=lr,
    weight_decay=weight_decay,
    dropout_rate=dropout_rate,
    scaling="mean",
    lags_seq=lags_seq,
    num_inference_steps=num_inference_steps,
    batch_size=batch_size,
    num_batches_per_epoch=num_batches_per_epoch,
    trainer_kwargs=dict(max_epochs=max_epochs, accelerator=accelerator, devices="1"),
)

return estimator.train(
    training_data=dataset["train"],
    cache_data=True,
    shuffle_buffer_length=1024,
)

```

1. Run the model:
2. prepare metrics for num_samples runs at the test set, using MultivariateEvaluator, make_evaluation_predictions and evaluator from GluonTS:

```python def preparemetrics(dataset, predictor, numsamples=100): evaluator = MultivariateEvaluator( quantiles=(np.arange(20) / 20.0)[1:], targetaggfuncs={"sum": np.sum} )

forecast_it, ts_it = make_evaluation_predictions(
    dataset=dataset["test"], predictor=predictor, num_samples=num_samples
)
forecasts, targets = list(forecast_it), list(ts_it)
agg_metric, _ = evaluator(targets, forecasts, num_series=len(dataset["test"]))

return forecasts, targets, agg_metric

```

• finally, prepare statistics and visualize predictions, for example, like this (you can find definition of the plot function at examples/experiments.ipynb):

```python def preparestatistics(dataset, forecasts, targets, aggmetric, frame=0): metrics = { "CRPS": "mean_wQuantileLoss", "ND": "ND", "NRMSE": "NRMSE", "MSE": "MSE" }

for name in metrics:
    print(f"{name}: {round(agg_metric[metrics[name]], 4)}")
    print(f"{name + '-Sum'}: {round(agg_metric['m_sum_' + metrics[name]], 4)}")
    print("-" * 64)

plot(
    target=targets[frame],
    forecast=forecasts[frame],
    prediction_length=dataset["metadata"].prediction_length,
)
plt.show()

```

The full training-inference pipeline for a given dataset then looks like this:

python predictor = prepare_predictor( dataset=dataset, mode="rnn_timegrad", max_epochs=40, accelerator="gpu", num_train_timesteps=100, beta_start=1e-4, beta_end=0.1, beta_schedule="linear", context_length_coef=1, num_layers=3, hidden_size=40, lr=1e-3, weight_decay=1e-8, dropout_rate=0.1, lags_seq=[1, 7, 14, 24, 168], num_inference_steps=99, batch_size=64, num_batches_per_epoch=100) forecasts, targets, agg_metric = prepare_metrics(dataset, predictor) prepare_statistics( dataset["dataset"], dataset["forecasts"], dataset["targets"], dataset["agg_metric"] )

Acknowledgments

We thank the following repositories, papers and their authors and do not claim any authorship on their achievments.

• Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting, GitHub repo

License

MIT License