In previous articles, I discussed how to prepare data and assess predictions. Now, it's time to return to machine learning and time series forecasting methods. There are many different techniques available. Some are quite straightforward (like moving averages) while others are more advanced (such as neural networks).
In this article, I will provide an overview of selected forecasting models and discuss their relevance.
Popular time series forecasting models
Below are ten of the most popular time series forecasting models, along with their basic characteristics. I aim to present them in a way that highlights their most effective applications. Let’s break them down and learn how to use each model optimally:
Auto-Regressive Moving Average (abb. ARIMA)
This model is based on the concept of moving averages, which typically smooths out predictions. The basic version of the model does not account for seasonality, but the alternative model, SARIMA, does consider this factor. The SARIMAX variant allows for the inclusion of exogenous variables.
This model is effective for:
- short-term forecasting,
- data that exhibit autocorrelation.
Exponential Smoothing
This approach is based on weighted averages. The basic version does not account for trends or seasonality; however, its extensions do consider those factors:
- Holt's method handles time series data with a trend,
- Holt-Winters method addresses time series data with both trend and seasonality.
Similar to ARIMA, these methods are effective for short-term forecasting.
Prophet
This time series forecasting approach was developed by Meta. It includes the management of pervasive trends and seasonality, even when those factors change over time. It is an additive model that utilizes the decomposition of time series. First, it analyzes different components separately. Then, it adds their predictions to obtain the final one.
I recommend this model for two instances:
- for time series with well-distinguished seasonal effects,
- for short-term forecasting.
Theta
Similar to Prophet, Theta is a decomposition time series forecasting method. However, the former is much more sophisticated. Theta, despite its simple structure, performs very well on both seasonal and non-seasonal data.
Croston
This model is based on exponential smoothing and works on two components:
- average values of time series,
- average interval between occurrences.
By design, Croston applies the best to intermittent time series. In such a case, classical time series forecasting methods often fail because the high number of zero values disrupts the detection of stable trends and seasonality.
XGBoost
XGBoost belongs to the gradient boosting time series models that are based on decision trees. To grasp this idea, imagine a core, strong model resembling a trunk that connects many weaker models. A distinctive feature of this model is the way trees are constructed — they grow depth-wise, which means that at each level, all leaves are split, ultimately creating a complete, balanced tree.
This model is especially versatile and can handle most time series, apart from some sparse examples. It can also include external variables, which may enhance the model's accuracy.
Light gradient boosting machine (LGBM)
This time series forecasting model is another example of one based on gradient boosting. Its key distinction from XgBoost lies in its approach to growing decision trees leaf-wise. This means that it only splits the leaf with the highest gain, resulting in unbalanced trees that tend to be deeper. Consequently, the algorithm operates much faster and requires less memory, making it well-suited for large datasets.
Recurrent Neural Networks (RNN)
This type of neural network learns through backpropagation through time, meaning it takes the output of a neuron as an input for neurons at the next time step. This enables handling sequential data as a time series. It can also include exogenous variables.
However, the classic RNN is not the strongest suit in terms of time series prediction, mostly due to vanishing gradients.
This problem causes the model to "forget" information from the distant past because the signal used to update the weights becomes exponentially smaller over time. This makes it impossible for the network to learn long-term dependencies, which are crucial for accurate time series forecasting.
Therefore, the most suitable model for time series forecasting is LSTM (Long Short-Term Memory), which addresses this problem by ensuring the preservation of information throughout the layers.
Box-Cox, ARMA, Trend, and Seasonal Components (BATS)
This highly advanced model is capable of automated modeling of complex patterns in time series. Its name is an acronym that reflects its vital components:
- the Box-Cox transformation to adjust the data to be closer to the normal distribution,
- The ARMA components, along with exponential smoothing, effectively address both long-term and short-term data dependencies,
- Trend and seasonality identify and model complex seasonal trends, including annual cycles.
N-BEATS
This deep learning-driven model utilizes a sophisticated architecture comprising multiple interconnected feed-forward neural networks, commonly referred to as multilayer perceptrons (MLPs). Its primary function is to decompose time series data into projections for various components, such as trends or seasonality. These components are then aggregated to produce the final output.
N-BEATS is also capable of incorporating exogenous variables, which contributes to its high accuracy rate. Additionally, this model is notable for its enhanced interpretability.
Understanding time series: Which time series model is the best?
Every forecasting method has specific features, but none is universal—that is, no method will consistently yield the best results for every time series. Each time series is unique, and a model with the same parameters can perform exceptionally well on one series and poorly on another. Relying on a single time series forecasting model can be risky.
While there are pre-trained models available, their performance can be unpredictable; they may sometimes perform better or worse depending on the context. These models can be useful, but they should not be regarded as a universal solution—rather, they should be considered as one option among many.
Time series and language learning models (LLMs): Can ChatGPT successfully predict?
This is a question I hear quite often, and to which I always answer, starting with a digression.
All the described models can be classified as based on machine learning algorithms – all of them predict based on what model was allowed to learn from the data. All machine learning models are artificial intelligence methods, even the simplest ones. We tend to forget about this fact.
AI is more often identified only with chats like GPT, which are built on LLMs – large language models. Such chats can also serve as a tool for time series forecasting.
However, their performance is poor. LLMs are not designed to build extensive tools by themselves. I cannot say in good faith that it is possible to do so with chat's help.
But – maybe both of the last two sentences should end with the word ‘yet’.
Analyzing prediction: What if the prediction is not accurate enough?
All of the methods mentioned above can be used for univariate forecasting, which focuses solely on the historical data of a single variable. This is the most basic type of forecasting. If the accuracy of this approach is not satisfactory, one can consider multivariate forecasting. Some of the models presented are suitable for this purpose.
In multivariate forecasting, both the primary variable and the exogenous variables—those that influence the primary variable—are predicted. The process of selecting appropriate exogenous variables and assessing their actual impact on the forecast is a topic for another article.
Another issue is lumpy demand, which refers to unpredictable (unforecastable) data. No model will yield accurate predictions in such cases. What can be done in such a case? One approach is to aggregate data, possibly within the framework of a business hierarchy or by identifying correlations between products. First, predict the overall total and then, reconcile that with individual products. This process is known as hierarchical forecasting..
The conclusion: Overview of time series forecasting
There is no such thing as an ultimate model that fits all existing time series. Every time series should be treated separately, and a model should be chosen individually. For some series, simple models will perform better than the more complicated ones.
Therefore, it is important to test many models on time series and not limit the solution to only 2 or 3 models. Chats can be included in this, as a method, but not as an individual tool.
Predictions made using BiModal Forecasting are reliable; each process step is clearly explained, and every decision is backed by statistical analysis or business insight.