[!TIP] [Updates in May 2025] 🎉 Our survey paper Deep Learning for Multivariate Time Series Imputation: A Survey gets accepted by IJCAI 2025! We comprehensively review the literature of the state-of-the-art deep-learning imputation methods for time series, provide a new taxonomy based on uncertainty and model architecture for them, systematically compare multiple toolkits, and discuss the challenges and future directions in this field.

[Updates in Jun 2024] 😎 The 1st comprehensive time-seres imputation benchmark paper TSI-Bench: Benchmarking Time Series Imputation is now publicly available. The code is open source in the repo Awesome_Imputation. With nearly 35,000 experiments, we provide a comprehensive benchmarking study on 28 imputation methods, 3 missing patterns (points, sequences, blocks), various missing rates, and 8 real-world datasets.

[Updates in May 2024] 🔥 We applied SAITS embedding and training strategies to iTransformer, FiLM, FreTS, Crossformer, PatchTST, DLinear, ETSformer, FEDformer, Informer, Autoformer, Non-stationary Transformer, Pyraformer, Reformer, SCINet, RevIN, Koopa, MICN, TiDE, and StemGNN in PyPOTS to enable them to be applicable to the time-series imputation task.

‼️Kind reminder: This document can help you solve many common questions, please read it before you run the code.

The official code repository is for the paper SAITS: Self-Attention-based Imputation for Time Series (preprint on arXiv is here), which has been accepted by the journal Expert Systems with Applications (ESWA) [2022 IF 8.665, CiteScore 12.2, JCR-Q1, CAS-Q1, CCF-C]. You may never have heard of ESWA, while it was ranked 1st in Google Scholar under the top publications of Artificial Intelligence in 2016 (info source), and is still the top 1 AI journal according to Google Scholar metrics (here is the current ranking list FYI).

SAITS is the first work applying pure self-attention without any recursive design in the algorithm for general time series imputation. Basically you can take it as a validated framework for time series imputation, like we've integrated more than 2️⃣0️⃣ forecasting models into PyPOTS by adapting SAITS framework. More generally, you can use it for sequence imputation. Besides, the code here is open source under the MIT license. Therefore, you're welcome to modify the SAITS code for your own research purpose and domain applications. Of course, it probably needs a bit of modification in the model structure or loss functions for specific scenarios or data input. And this is an incomplete list of scientific research referencing SAITS in their papers.

🤗 Please cite SAITS in your publications if it helps with your work. Please star🌟 this repo to help others notice SAITS if you think it is useful. It really means a lot to our open-source research. Thank you! BTW, you may also like PyPOTS for easily modeling your partially-observed time-series datasets.

[!IMPORTANT] 📣 Attention please:

SAITS now is available in PyPOTS, a Python toolbox for data mining on POTS (Partially-Observed Time Series). An example of training SAITS for imputing dataset PhysioNet-2012 is shown below. With PyPOTS, easy peasy! 😉

❖ Motivation and Performance

⦿ Motivation: SAITS is developed primarily to help overcome the drawbacks (slow speed, memory constraints, and compounding error) of RNN-based imputation models and to obtain the state-of-the-art (SOTA) imputation accuracy on partially-observed time series.

⦿ Performance: SAITS outperforms BRITS by 12% ∼ 38% in MAE (mean absolute error) and achieves 2.0 ∼ 2.6 times faster training speed. Furthermore, SAITS outperforms Transformer (trained by our joint-optimization approach) by 2% ∼ 19% in MAE with a more efficient model structure (to obtain comparable performance, SAITS needs only 15% ∼ 30% parameters of Transformer). Compared to another SOTA self-attention imputation model NRTSI, SAITS achieves 7% ∼ 39% smaller mean squared error (above 20% in nine out of sixteen cases), meanwhile, needs much fewer parameters and less imputation time in practice. Please refer to our full paper for more details about SAITS' performance.

❖ Brief Graphical Illustration of Our Methodology

Here we only show the two main components of our method: the joint-optimization training approach and SAITS structure. For the detailed description and explanation, please read our full paper Paper_SAITS.pdf in this repo or on arXiv.

Fig. 1: Training approach Fig. 2: SAITS structure

❖ Citing SAITS

If you find SAITS is helpful to your work, please cite our paper as below, ⭐️star this repository, and recommend it to others who you think may need it. 🤗 Thank you!

bibtex @article{du2023saits, title = {{SAITS: Self-Attention-based Imputation for Time Series}}, journal = {Expert Systems with Applications}, volume = {219}, pages = {119619}, year = {2023}, issn = {0957-4174}, doi = {10.1016/j.eswa.2023.119619}, url = {https://arxiv.org/abs/2202.08516}, author = {Wenjie Du and David Cote and Yan Liu}, } or

Wenjie Du, David Cote, and Yan Liu. SAITS: Self-Attention-based Imputation for Time Series. Expert Systems with Applications, 219:119619, 2023.

😎 Our latest survey and benchmarking research on time-series imputation may also be useful to your work:

bibtex @article{wang2025survey, title={Deep Learning for Multivariate Time Series Imputation: A Survey}, author={Jun Wang and Wenjie Du and Yiyuan Yang and Linglong Qian and Wei Cao and Keli Zhang and Wenjia Wang and Yuxuan Liang and Qingsong Wen}, booktitle = {Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI)}, year={2025} }

bibtex @article{du2024tsibench, title={TSI-Bench: Benchmarking Time Series Imputation}, author={Wenjie Du and Jun Wang and Linglong Qian and Yiyuan Yang and Fanxing Liu and Zepu Wang and Zina Ibrahim and Haoxin Liu and Zhiyuan Zhao and Yingjie Zhou and Wenjia Wang and Kaize Ding and Yuxuan Liang and B. Aditya Prakash and Qingsong Wen}, journal={arXiv preprint arXiv:2406.12747}, year={2024} }

🔥 In case you use PyPOTS in your research, please also cite the following paper:

bibtex @article{du2023pypots, title={{PyPOTS: a Python toolbox for data mining on Partially-Observed Time Series}}, author={Wenjie Du}, journal={arXiv preprint arXiv:2305.18811}, year={2023}, } or

Wenjie Du. PyPOTS: a Python toolbox for data mining on Partially-Observed Time Series. arXiv, abs/2305.18811, 2023.

❖ Repository Structure

The implementation of SAITS is in dir modeling. We give configurations of our models in dir configs, provide the dataset links and preprocessing scripts in dir dataset_generating_scripts. Dir NNI_tuning contains the hyper-parameter searching configurations.

❖ Development Environment

All dependencies of our development environment are listed in file conda_env_dependencies.yml. You can quickly create a usable python environment with an anaconda command conda env create -f conda_env_dependencies.yml.

❖ Datasets

For datasets downloading and generating, please check out the scripts in dir dataset_generating_scripts.

❖ Quick Run

Generate the dataset you need first. To do so, please check out the generating scripts in dir dataset_generating_scripts.

After data generation, train and test your model, for example,

```shell

create a dir to save logs and results

mkdir NIPS_results

train a model

nohup python runmodels.py \ --configpath configs/PhysioNet2012SAITSbest.ini \ > NIPSresults/PhysioNet2012SAITS_best.out &

during training, you can run the blow command to read the training log

less NIPSresults/PhysioNet2012SAITS_best.out

after training, pick the best model and modify the path of the model for testing in the config file, then run the below command to test the model

python runmodels.py \ --configpath configs/PhysioNet2012SAITSbest.ini \ --test_mode ```

❗️Note that paths of datasets and saving dirs may be different on personal computers, please check them in the configuration files.

❖ Acknowledgments

Thanks to Ciena, Mitacs, and NSERC (Natural Sciences and Engineering Research Council of Canada) for funding support.
Thanks to all our reviewers for helping improve the quality of this paper.
Thanks to Ciena for providing computing resources.
And thank you all for your attention to this work.

✨Stars/forks/issues/PRs are all welcome!

❖ Last but Not Least

If you have any additional questions or have interests in collaboration, please take a look at my GitHub profile and feel free to contact me 😃.