Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/astrogilda/tspfnbeats

https://github.com/astrogilda/tspfnbeats

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.7%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: astrogilda
  • Default Branch: main
  • Size: 6.1 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Fork of ARM-software/tspfnbeats
Created over 4 years ago · Last pushed over 5 years ago

https://github.com/astrogilda/tspfnbeats/blob/main/ 

# Experiments with N-BEATS (Neural basis expansion analysis for interpretable time series forecasting)
Project with a TensorFlow implementation of [N-BEATS](https://arxiv.org/pdf/1905.10437.pdf) and experiments with ensembles.

##### Implementation 
TensorFlow implementation of N-BEATS adapted from PyTorch variant found [here](https://github.com/philipperemy/n-beats)


##### Generic
![Generic Model Outputs](./images/GenericModelOutputs.png)

##### Interpretable
![Interpretable Model Outputs](./images/InterpretableModelOutputs.png)


##### Weighted-Ensembling
If we view each ensemble member as an expert forecaster, then a gating network [1] can be used to select an expert or blend their outputs. 
In the blended approach the error of the output affects the weight updates of all the experts, and as such there is *co-adaptation* between them.
I explore one way to reduce co-adaptations by using a [dropout](https://arxiv.org/pdf/1207.0580.pdf) layer, implemented as Weighted-Ensemble of N-BEATS (WE N-BEATS). 

During training, the (random) zeroing out of the gates outputs cancels the contribution of a member to the final error. 
The backpropagated gradient of loss would then update only those members which contributed to the loss as well as the gate network itself, thus learning proper blending weights and reducing co-adaptations.
 
![WE-NBEATS](./images/WeightEnsemble.png)


#### Ensemble Example Predictions
![Example Predictions](./images/ExamplePredictions.png)



##### Dependencies
This project has been developed with the following packages
- tensorflow (2.3)
- pyyaml (5.3.1)
- pandas (1.1.1)
- scikit-learn (0.23.2)
- tqdm (4.48.2)
- matplotlib (3.3.1)

[1]: R. A. Jacobs, M. I. Jordan, S. J. Nowlan, G. E. Hinton, Adaptive Mixtures of Local Experts, Neural Computation, 3: 79-87, 1991

[2]: L. Breiman, Bagging predictors, Machine Learning, 24(2):123140, Aug 1996

Owner

  • Name: Sankalp Gilda
  • Login: astrogilda
  • Kind: user
  • Location: Gainesville, FL

Machine Learning Engineer | Ph.D., Astronomy

GitHub Events

Total
Last Year