This is the official repository of Sundial: A Family of Highly Capable Time Series Foundation Models.
🚩 News (2025.05) Get 1st MASE on the GIFT-Eval Benchmark.
🚩 News (2025.05) Released a trillion-scale pre-trained model on HuggingFace. A quickstart is provided here.
🚩 News (2025.05) Sundial has been accepted as ICML 2025 Spotlight (Top 2.6%). See you at Vancouver :)
🚩 News (2025.02) Get 1st MSE/MAE zero-shot performance on Time-Series-Library datasets.
Sundial is a family of generative time series foundation models, which is pre-trained on TimeBench (10^12 time points). The model can be applied for both point and probabilistic forecasting.
Not only the mean or quantiles, you can estimate anything about the predictive distribution with raw generated samples.
We propose TimeFlow Loss to predict next-patch’s distribution, allowing Transformers to be trained without discrete tokenization and make non-deterministic predictions.
We release a HuggingFace model, which can make zero-shot predictions on CPU within seconds! 🚀
Inference Time on Apple M1 Pro CPU (16 GB)
| Lookback | Forcast | # Generated | Wall-Clock Time | Accelerate By |
|---|---|---|---|---|
| 672 | 16 | 1 | 249ms | - |
| 2880 | 16 | 1 | 510ms | FlashAttention |
| 2880 | 720 | 1 | 510ms | Multi-Patch Prediction |
| 2880 | 1440 | 1 | 789ms | KV Cache |
| 2880 | 720 | 20 | 949ms | Shared Condition |
All you need is a network connection and the HuggingFace access!
pip install transformers==4.40.1
import torch
from transformers import AutoModelForCausalLM
# load pretrain model
# supports different lookback/forecast lengths
model = AutoModelForCausalLM.from_pretrained('thuml/sundial-base-128m', trust_remote_code=True)
# prepare input
batch_size, lookback_length = 1, 2880
seqs = torch.randn(batch_size, lookback_length)
# Note that Sundial can generate multiple probable predictions
forecast_length = 96
num_samples = 20
output = model.generate(seqs, max_new_tokens=forecast_length, num_samples=num_samples)
# use raw predictions for mean/quantiles/confidence-interval estimation
print(output.shape)
More examples of predicting quantiles or confidence intervals are provided in this notebook. Please raise your valuable suggestions here, we 'd like to solve it ASAP 🤗.
Intuitively, Sundial can be viewed as an ARMA model (Auto-Regression and Moving-Average). Transformer learns auto-regressive token representations. Conditioned on them, TimeFlow transforms random noises into non-deterministic predictions.
We have currently built three different sizes of Sundial. Model configurations are provided here:
We evaluate Sundial (Base) with advanced time series foundation models on well-recognized benchmarks:
Code for fine-tuning is on its way and will be available soon! Stay tuned for updates!
If you find this repo helpful, please cite our paper.
@article{liu2025sundial,
title={Sundial: A Family of Highly Capable Time Series Foundation Models},
author={Liu, Yong and Qin, Guo and Shi, Zhiyuan and Chen, Zhi and Yang, Caiyin and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
journal={arXiv preprint arXiv:2502.00816},
year={2025}
}
We appreciate the following resources a lot for their valuable code and datasets:
- Time-Series-Library (https://github.com/thuml/Time-Series-Library)
- Large-Time-Series-Model & UTSD (https://github.com/thuml/Large-Time-Series-Model)
- Timer-XL (https://github.com/thuml/Timer-XL)
- LoTSA Data (https://huggingface.co/datasets/Salesforce/lotsa_data)
- Chronos Datasets (https://huggingface.co/datasets/autogluon/chronos_datasets)
If you have any questions or want to use the code, feel free to contact:
- Yong Liu (liuyong21@mails.tsinghua.edu.cn)
- Guo Qin (qinguo24@mails.tsinghua.edu.cn)
This model is licensed under the Apache-2.0 License.