<!--

    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at
    
        http://www.apache.org/licenses/LICENSE-2.0
    
    Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.

-->

# TimeSeries Large Model

## Introduction

A time series large model is a foundational model specifically designed for time series analysis. The IoTDB team has independently developed time series large models, which are pre-trained on massive time series data using technologies such as transformer structures. These models can understand and generate time series data across various domains and are applicable to applications like time series forecasting, anomaly detection, and time series imputation. Unlike traditional time series analysis techniques, time series large models possess the capability to extract universal features and provide technical services based on zero-shot analysis and fine-tuning for a wide range of analytical tasks.

The team's related technologies of time series large models have been published in top international machine learning conferences.

## Application Scenarios

- **Time Series Forecasting**: Provides forecasting services for time series data in industrial production, natural environments, and other areas, helping users to understand future trends in advance.
- **Data Imputation**: For missing segments in time series, perform context imputation to enhance the continuity and completeness of the dataset.
- **Anomaly Detection**: Utilizing regression analysis technology, monitor time series data in real-time and provide timely warnings for potential anomalies.

![](/img/LargeModel10.png)

## Timer Model

The Timer<sup><a href="#appendix1" id="ref1" style="text-decoration: none;">[1]</a></sup> model(not built-in) not only demonstrates excellent few-shot generalization and multi-task adaptation capabilities but also gains a rich knowledge base through pre-training, endowing it with the universal capability to handle a variety of downstream tasks, featuring the following:

- **Generalization**: The model can be fine-tuned using a small number of samples to achieve leading predictive performance in the industry.
- **Versatility**: The model is designed flexibly to adapt to various task requirements and supports variable input and output lengths, enabling it to play a role in various application scenarios.
- **Scalability**: As the number of model parameters increases or the scale of pre-training data expands, the model's performance continues to improve, ensuring the model can optimize its predictive effects with the growth of time and data volume.

![](/img/model01.png)

## Timer-XL Model

Timer-XL<sup><a href="#appendix2" id="ref2" style="text-decoration: none;">[2]</a></sup> is an upgraded version of Timer that further extends the network structure and achieves comprehensive breakthroughs in multiple dimensions(available since V2.0.5.1):

- **Ultra-long Context Support**: This model breaks through the limitations of traditional time series forecasting models, supporting inputs of thousands of Tokens (equivalent to tens of thousands of time points), effectively solving the context length bottleneck problem.
- **Multi-variable Prediction Scenarios Coverage**: Supports various forecasting scenarios, including non-stationary time series forecasting, multi-variable forecasting tasks, and forecasting with covariates, meeting diverse business needs.
- **Large-scale Industrial Time Series Dataset**: Pre-trained on a trillion-scale industrial IoT time series dataset, which has the important characteristics of huge volume, excellent quality, and rich domains, covering energy, aerospace, steel, transportation, and other fields.

![](/img/model02.png)

## Timer-Sundial Model

Timer-Sundial<sup><a href="#appendix3" id="ref3" style="text-decoration: none;">[3]</a></sup> is a series of generative foundational models focused on time series forecasting(available since V2.0.5.1). The basic version has 128 million parameters and has undergone large-scale pre-training on 1 trillion time points. Its core features include:

- **Powerful Generalization Performance**: Possesses zero-shot forecasting capabilities, supporting both point forecasting and probabilistic forecasting simultaneously.
- **Flexible Forecasting Distribution Analysis**: Can not only forecast mean values or quantiles but also evaluate any statistical characteristics of the forecasting distribution through the original samples generated by the model.
- **Innovative Generative Architecture**: Adopts a "Transformer + TimeFlow" collaborative architecture—Transformer learns the autoregressive representation of time segments, and the TimeFlow module converts random noise into diversified forecasting trajectories based on the Flow-Matching framework, achieving efficient non-deterministic sample generation.

![](/img/model03.png)

## Effect Demonstration

Time series large models can adapt to real-time series data in various domains and scenarios, demonstrating excellent processing effects on various tasks. The following are real-world performances on different datasets:

**Time Series Forecasting:**

Utilizing the forecasting capability of time series large models, the future change trends of time series can be accurately predicted. In the figure below, the blue curve represents the forecast trend, and the red curve represents the actual trend, with a high degree of 吻合 (coincidence) between the two curves.

![](/img/LargeModel03.png)

**Data Imputation:**

Utilizing time series large models to perform predictive imputation on missing data segments.

![](/img/timeseries-large-model-data-imputation.png)


**Anomaly Detection:**

Utilizing time series large models to accurately identify outliers that deviate significantly from normal trends.

![](/img/LargeModel05.png)

## Deployment Usage

1. Open the IoTDB CLI console and verify that the ConfigNode, DataNode, and AINode statuses are all ​Running.

```sql
IoTDB> show cluster
+------+----------+-------+---------------+------------+--------------+-----------+
|NodeID|  NodeType| Status|InternalAddress|InternalPort|       Version|  BuildInfo|
+------+----------+-------+---------------+------------+--------------+-----------+
|     0|ConfigNode|Running|      127.0.0.1|       10710|       2.0.5.1|    069354f|
|     1|  DataNode|Running|      127.0.0.1|       10730|       2.0.5.1|    069354f|
|     2|    AINode|Running|      127.0.0.1|       10810|       2.0.5.1|069354f-dev|
+------+----------+-------+---------------+------------+--------------+-----------+
Total line number = 3
It costs 0.140s
```

2. When the AINode is started for the first time in a networked environment, the Timer-XL and Sundial models will be automatically pulled.

3. Verify model registration success


```sql
IoTDB:etth> show models
+---------------------+--------------------+--------+------+
|              ModelId|           ModelType|Category| State|
+---------------------+--------------------+--------+------+
|                arima|               Arima|BUILT-IN|ACTIVE|
|          holtwinters|         HoltWinters|BUILT-IN|ACTIVE|
|exponential_smoothing|ExponentialSmoothing|BUILT-IN|ACTIVE|
|     naive_forecaster|     NaiveForecaster|BUILT-IN|ACTIVE|
|       stl_forecaster|       StlForecaster|BUILT-IN|ACTIVE|
|         gaussian_hmm|         GaussianHmm|BUILT-IN|ACTIVE|
|              gmm_hmm|              GmmHmm|BUILT-IN|ACTIVE|
|                stray|               Stray|BUILT-IN|ACTIVE|
|              sundial|       Timer-Sundial|BUILT-IN|ACTIVE|
|             timer_xl|            Timer-XL|BUILT-IN|ACTIVE|
+---------------------+--------------------+--------+------+
Total line number = 10
It costs 0.004s
```

## Appendix

<a id="appendix1"></a>[1] Timer- Generative Pre-trained Transformers Are Large Time Series Models, Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, Mingsheng Long.
[↩ ](#ref1)

<a id="appendix2"></a>[2] TIMER-XL- LONG-CONTEXT TRANSFORMERS FOR UNIFIED TIME SERIES FORECASTING ,Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, Mingsheng Long.
[↩ ](#ref2)

<a id="appendix3"></a>[3] Sundial- A Family of Highly Capable Time Series Foundation Models, Yong Liu, Guo Qin, Zhiyuan Shi, Zhi Chen, Caiyin Yang, Xiangdong Huang, Jianmin Wang, Mingsheng Long, ICML 2025 spotlight.
[↩ ](#ref3)