Timer (Time-Series Transformer)

This repo provides official code, datasets, and checkpoints for Timer: Generative Pre-trained Transformers Are Large Time Series Models. [Poster] [Slides].

Updates

🚩 News (2025.5) Sundial, a family of generative time series foundation models has been accepted as ICML 2025 Spotlight (Top 2.6%). Get your zero-shot probabilistic predictions within milliseconds! [HuggingFace] [Quickstart].

🚩 News (2025.2) We release an open codebase OpenLTM, which contains the whole pipeline to pre-train customized large time-series models.

🚩 News (2024.12) Timer-XL for unified forecasting is accepted as ICLR 2025. We released a pre-trained model for zero-shot forecasting [HuggingFace] [Quickstart].

🚩 News (2024.10) We release the pre-training dataset UTSD on HuggingFace or you can use the numpy format UTSD and this dataloader.

🚩 News (2024.5) Accepted by ICML 2024, a camera-ready version of 31 pages.

🚩 News (2024.2) Releasing model checkpoints and code for fine-tuning on different tasks [README].

Introduction

Time Series Transformer (Timer) is a Generative Pre-trained Transformer for general time series analysis.

Quickstart

We provide out-of-the-box models to make predictions without training. See our HuggingFace for more information.

Example of Timer (Zero-Shot Forecasting)

import torch
from transformers import AutoModelForCausalLM

# load pretrain model
model = AutoModelForCausalLM.from_pretrained('thuml/timer-base-84m', trust_remote_code=True)

# prepare input
batch_size, lookback_length = 1, 2880
seqs = torch.randn(batch_size, lookback_length)

# generate forecast
prediction_length = 96
normed_output = model.generate(normed_seqs, max_new_tokens=prediction_length)

print(output.shape)

Example of Sundial (Multi-Prediction Generation)

import torch
from transformers import AutoModelForCausalLM

# load pretrain model
# supports different lookback/forecast lengths
model = AutoModelForCausalLM.from_pretrained('thuml/sundial-base-128m', trust_remote_code=True) 

# prepare input
batch_size, lookback_length = 1, 2880 
seqs = torch.randn(batch_size, lookback_length)

# Note that Sundial can generate multiple probable predictions
forecast_length = 96 
num_samples = 20

output = model.generate(seqs, max_new_tokens=forecast_length, num_samples=num_samples)

# use raw predictions for mean/quantiles/confidence-interval estimation
print(output.shape)

Model Adaption

• For developers interested in fine-tuning large time-series models or pre-training on customized datasets, please use OpenLTM, including code scripts and checkpoint of various models.

• For developers interested in applying large time-series models on other time series analysis tasks (e.g., imputation and anomaly detection), this repo contains scripts and checkpoints [README].

Datasets

We collect Unified Time Series Datasets (UTSD), which encompass well-curated time series to facilitate the research on large time-series models. Our dataset is released in HuggingFace.

Usage

You can access the data from HuggingFace and load the data in the style of TSLib:

# huggingface-cli login
# export HF_ENDPOINT=https://hf-mirror.com 

python ./scripts/UTSD/download_dataset.py

# dataloader
python ./scripts/UTSD/utsdataset.py

If you meet troubles when accessing the data, you can also download UTSD in numpy from [Tsinghua Cloud] and use UTSD_Npy dataloader [here].

Introduction

ICML 2024

We propose Timer, a decoder-only pre-trained time series Transformer. We propose single-series sequence (S3) format, converting diverse series into unified 1D sequences. The predictive model can also be adapted for forecasting, imputation, and anomaly detection [README].

ICLR 2025

We proposed Timer-XL for unified time series forecasting. It can be used for supervised training or large-scale pre-training, explicitly modeling multi-dimensional time series [GitHub].

ICML 2025 Spotlight

We proposed Sundial, a family of generative time series foundation models, which is pre-trained on a trillion (10^12) time points. The model can be applied for both point and probabilistic forecasting, making zero-shot forecasting within milliseconds [GitHub].

Citation

If you find this repo helpful, please cite our paper.

@inproceedings{liutimer,
  title={Timer: Generative Pre-trained Transformers Are Large Time Series Models},
  author={Liu, Yong and Zhang, Haoran and Li, Chenyu and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  booktitle={Forty-first International Conference on Machine Learning}
}

@article{liu2024timer,
  title={Timer-XL: Long-Context Transformers for Unified Time Series Forecasting},
  author={Liu, Yong and Qin, Guo and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  journal={arXiv preprint arXiv:2410.04803},
  year={2024}
}

@article{liu2025sundial,
  title={Sundial: A Family of Highly Capable Time Series Foundation Models},
  author={Liu, Yong and Qin, Guo and Shi, Zhiyuan and Chen, Zhi and Yang, Caiyin and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  journal={arXiv preprint arXiv:2502.00816},
  year={2025}
}

Acknowledgment

We appreciate the following GitHub repos a lot for their valuable code and datasets:

• Time-Series-Library (https://github.com/thuml/Time-Series-Library)
• AutoTimes (https://github.com/thuml/AutoTimes)
• LoTSA Data (https://huggingface.co/datasets/Salesforce/lotsa_data)
• UCR Anomaly Archive (https://arxiv.org/pdf/2009.13807)

Contributors

If you have any questions or want to use the code, feel free to contact:

• Yong Liu (liuyong21@mails.tsinghua.edu.cn)
• Haoran Zhang (zhang-hr24@mails.tsinghua.edu.cn)
• Chenyu Li (lichenyu20@mails.tsinghua.edu.cn)
• Guo Qin (qinguo24@mails.tsinghua.edu.cn)