Author Archives: Nikos

Improving forecasting by estimating time series structural components across multiple frequencies

N. Kourentzes, F. Petropoulos and J.  R. Trapero, 2014, International Journal of Forecasting, 30: 291-302. http://dx.doi.org/10.1016/j.ijforecast.2013.09.006

Identifying the most appropriate time series model to achieve a good forecasting accuracy is a challenging task. We propose a novel algorithm that aims to mitigate the importance of model selection, while increasing the accuracy. Multiple time series are constructed from the original time series, using temporal aggregation. These derivative series highlight different aspects of the original data, as temporal aggregation helps in strengthening or attenuating the signals of different time series components. In each series the appropriate exponential smoothing method is fitted and its respective time series components are forecasted. Subsequently, the time series components from each aggregation level are combined, and then used to construct the final forecast. This approach achieves a better estimation of the different time series components, through temporal aggregation, and reduces the importance of model selection through forecast combination. An empirical evaluation of the proposed framework demonstrates significant improvements in forecasting accuracy, especially for long-term forecasts.

Download paper. Paper summary slides available.
R code to replicate paper results can be found here.
R code to run MAPA on your data can be found here.
An online demo can be found here.
A simplified discussion of the article can be found here.

Errata: p. 296, Table 1, Md Trend case should be: b_{i+h} =(b_i^{\sum_{j=1}^{h}{\phi^{j}}}-1)l_{i+h}.

Related Posts

On the identification of sales forecasting models in the presence of promotions

J.  R. Trapero, N. Kourentzes and R. Fildes, 2014, Journal of the Operantional Research Society, 66: 299-307. http://dx.doi.org/10.1057/jors.2013.174

Sales forecasting is of paramount importance to reduce inventory investment, enhance customer satisfaction and improve distribution operations. Shorter product life cycles, more competitive markets and more aggressive marketing, among other factors, have increased the complexity of sales forecasting. Forecasts are often produced using a Forecasting Support System that integrates univariate statistical forecasting with managerial judgment. One of the main reasons to use expert judgment is to forecast sales under promotional activity. An alternative approach to promotional forecasting is to replace expert adjustments by multivariate statistical models that use past promotions information, resulting in regression models whose exogenous inputs are promotion features (price discounts, type of display, type of advertising, etc.). Nonetheless, these regression type models may have a large dimension as well as multicollinearity issues. This work proposes a multivariate method that reduces the dimensionality of the problem by using Principal Component Analysis and models the error term as a transfer function identified by minimizing the Schwartz Information Criteria. To provide promotional forecasts for items with limited history we pool information across products. The performance of the model is compared against forecasts provided by experts and statistical benchmarks, on weekly data from a manufacturing company; outperforming both substantially. We find that the proposed multivariate model, developed on the basis of past promotional information, outperforms expert promotional adjustments.

Download paper. Paper summary slides available.

Related Posts

ANOM and Nemenyi tests

Code for the ANOM and Nemenyi tests for MatLab. Download here.

For a discussion of the two tests and the various ways to visualise the results look at this post.

Here are some examples, using the M3 results:

>> anom(X,0.05,labels);

nemM.fig1
The models in red are significantly better than the average (solid line).

For the Nemenyi test:

>> nemenyi(X,1,'labels',labels,'colormap','hsv');

nemM.fig2
There is no evidence of significant differences for models joined by the vertical lines. Depending on the model we are considering, different groups can be formed. The groups are identified using the mean rank of a model ± the critical distance.
There are several plotting options coded for the the nemenyi.m. I have also put an option for changing the colourmap to better visualise the comparisons. A potentially useful option is to set:

>> nemenyi(X,1,'labels',labels,'colormap','omcb');

which gives you the MCB test with ordered models. These two tests use exactly the same statistic, although MCB compares only with the best model.
nemM.fig3

You may also be interested in this very nice visualisation of Nemenyi post-hoc test by Farshid Sepehrband available here.

Related Posts

Intermittent Demand Forecasts with Neural Networks

N. Kourentzes, 2013, International Journal of Production Economics, 143: 198-206. http://dx.doi.org/10.1016/j.ijpe.2013.01.009

Intermittent demand appears when demand events occur only sporadically. Typically such time series have few observations making intermittent demand forecasting challenging. Forecast errors can be costly in terms of unmet demand or obsolescent stock. Intermittent demand forecasting has been addressed using established forecasting methods, including simple moving averages, exponential smoothing and Croston’s method with its variants. This study proposes a neural network (NN) methodology to forecast intermittent time series. These NNs are used to provide dynamic demand rate forecasts, which do not assume constant demand rate in the future and can capture interactions between the non-zero demand and the inter-arrival rate of demand events, overcoming the limitations of Croston’s method. In order to mitigate the issue of limited fitting sample, which is common in intermittent demand, the proposed models use regularised training and median ensembles over multiple training initialisations to produce robust forecasts. The NNs are evaluated against established benchmarks using both forecasting accuracy and inventory metrics. The findings of forecasting and inventory metrics are conflicting. While NNs achieved poor forecasting accuracy and bias, all NN variants achieved higher service levels than the best performing Croston’s method variant, without requiring analogous increases in stock holding volume. Therefore, NNs are found to be effective for intermittent demand applications. This study provides further arguments and evidence against the use of conventional forecasting accuracy metrics to evaluate forecasting methods for intermittent demand, concluding that attention to inventory metrics is desirable.

Download paper.

Related Posts

Analysis of judgmental adjustments in the presence of promotions

J.  R. Trapero, D. J. Pedregal, R. Fildes and  N. Kourentzes, 2013, International Journal of Forecasting, 29: 234-243. http://dx.doi.org/10.1016/j.ijforecast.2012.10.002

Sales forecasting is increasingly complex due to many factors, such as product life cycles that have become shorter, more competitive markets and aggressive marketing. Often, forecasts are produced using a Forecasting Support System that integrates univariate statistical forecasts with judgment from experts in the organization. Managers add information to the forecast, like future promotions, potentially improving accuracy. Despite the importance of judgment and promotions, the literature devoted to study their relationship on forecasting performance is scarce. We analyze managerial adjustments accuracy under periods of promotions, based on weekly data from a manufacturing company. Intervention analysis is used to establish whether judgmental adjustments can be replaced by multivariate statistical models when responding to promotional information. We show that judgmental adjustments can enhance baseline forecasts during promotions, but not systematically. Statistical models based on past promotions information achieved lower overall forecasting errors. Finally, a hybrid model illustrates the fact that human experts still added value to the statistical models.

Download paper.
Paper summary slides available.
A discussion of this paper can be found here.

Related Posts

New Product Forecasting and Inventory Planning Using Time Series Clustering

M. Hibon, N. Kourentzes and S. F. Crone, 2013, The 33rd Annual international Symposium on Forecasting, Seoul.

New product forecasting is a prerequisite for operational decisions in production and inventory management. With no historic demand data, traditional statistical forecasting methods cannot be employed and new product forecasting is left to the judgment of human experts. With some industries introducing thousands of new products multiple times per year, analytical methods to forecast new products are needed. This paper proposes a methodology of time series clustering and similarity search for analytical, data driven and fully automatic new product forecasting by analogies; designed to construct launch profiles from past product launches data, it utilitizes increasing sources of information including product features before launch, recalibrated by using initial orders during launch and early sales observations past launch. The method provides forecasts for new products and empirical quantiles which are used to derive safety stocks. Its promising performance is illustrated in an empirical evaluation using real data from the textile industry.

Download presentation.

Related Posts

Impact of Information Exchange on Supplier Forecasting Performance

J.  R. Trapero, N. Kourentzes and R. Fildes,  2012,  OMEGA,  40: 738-747. http://dx.doi.org/10.1016/j.omega.2011.08.009

Forecasts of demand are crucial to drive supply chains and enterprise resource planning systems.Usually, well-known univariate methods that work automatically such as exponential smoothing are employed to accomplish such forecasts. The traditional Supply Chain relies on a decentralized system where each member feeds its own Forecasting Support System (FSS) with incoming orders from direct customers. Nevertheless, other collaboration schemes are also possible, for instance, the Information Exchange framework allows demand information to be shared between the supplier and the retailer. Current theoretical models have shown the limited circumstances where retailer information is valuable to the supplier. However, there has been very little empirical work carried out. Considering a serially linked two-level supply chain, this work assesses the role of sharing market sales information obtained by the retailer on the supplier forecasting accuracy. Weekly data from a manufacturer and a major UK grocery retailer have been analyzed to show the circumstances where information sharing leads to improved forecasting accuracy. Without resorting to unrealistic assumptions, we find significant evidence of benefits through information sharing with substantial improvements in forecast accuracy.

Download paper.
Paper summary slides available.

Related Posts

Statistical Significance of Forecasting Methods – an empirical evaluation of the robustness and interpretability of the MCB, ANOM and Friedman-Nemenyi Test

M. Hibon, S. F. Crone and N. Kourentzes, 2012, The 32nd Annual international Symposium on Forecasting, Boston.

The series of M-competitions (1982, 1993,2000) have firmly established forecasting competitions as an objective approach to assess the empirical ex ante accuracy of competing forecasting methods. While the original M-competitions merely ranked algorithms by performance , Koning et al. (2005) assessed the results of M3 for the significance of the error differences using two non-parametric statistical tests: multiple comparisons with the mean (ANOM) and multiple comparisons with the best (MCB). As a result, they identified few methods which significantly outperformed all others.

However, MCB and ANOM have been subject to criticism due to their sensitivity to sample size of methods assessed and their limited interpretability, as both provide only a binary classification whether a model is significantly “better” or “worse” to the average (ANOM) or the best (MCB) performance of all models. This does not provide information on how individual models differ within each of these classes, and whether such differences are significant. To overcome these limitations, the Friedman and Nemenyi test has been proposed (Demsar, 2006), which is frequently employed in model comparisons in data mining ad machine learning.

We empirically assess (a) the robustness of ANOM and MCB in comparison to the Friedman and Nemenyi test, and (b) the interpretability of results of the significantly different subgroups. As a dataset, this study revisits the results of the recent NN3 competition by Crone et al. (2011), which extended the M3 competition to 59 new algorithms of computational intelligence in comparison to a subset of the 5 best models originally submitted to M3, evaluated on two masked subsets of 111 and 11 empirical time series of monthly M3 industry data. We assess the robustness of the statistical tests (1) on the error measure that is used to derive model ranks and provide suggestions, (2) their the sensitivity to the number of included models by including all contenders of the M3, and (3) their sensitivity to the number of time series compared. As a result we derive the robustness of each test for performance evaluations in competitions as well as empirical simulation studies of forecasting.

Download presentation.

Related Posts

Optimum parameters for Croston intermittent demand methods

N. Kourentzes, 2012, The 32nd Annual international Symposium on Forecasting, Boston.

Intermittent demand time series involve items that are requested infrequently, resulting in sporadic demand. That makes intermittent demand forecasting challenging and forecast errors can be costly in terms of unmet demand or obsolescent stock. In the literature such forecasting problems have been addressed using Croston’s method and its variants, which have a single smoothing parameter alpha. Although the literature provides suggestions on the effective range of the parameter, it does not provide guidelines how to select it. This is crucial, particularly since growing evidence in the literature points against the use of accuracy error metrics for model evaluation and hence parameter selection in intermittent demand time series. This leaves no valid methods how to best set the smoothing parameter of Croston’s method. This study proposes a novel optimisation framework that is based directly on inventory metrics instead of accuracy measures. Models optimised this way are found to outperform Croston’s models with fixed or conventionally optimised model parameters. Furthermore, this work finds that employing different parameters for smoothing the non-zero demand and the inter-demand intervals of Croston’s, instead of a single parameter as the literature suggests, provides further performance improvements.

Download presentation.

A paper on intermittent demand optimisation and model selection, following a simpler approach, can be found here.

Related Posts

Data Driven Fitting Sample Selection For Time Series Forecasting With Neural Networks

N. Kourentzes, 2012, International Joint Conference on Neural Networks, Brisbane, 10-15 June 2012.

In this paper we propose a data driven method to select the fitting sample of neural networks for time series forecasting. In spite of the fundamental importance of sample selection for model building there has been limited research in the forecasting literature, mostly concluding in vague recommendations on how much time series history should be used and stored. This research addresses this issue in a data driven framework. The proposed method allows the neural networks to iteratively adjust the fitting sample, penalizing the time series history for age and inconsistent behavior. The resulting selected sample helps the networks to produce accurate out-of-sample forecasts, focusing on the recent history of the time series. The performance of the method is demonstrated using time series from different domains, exhibiting substantial improvements in accuracy.

Download paper.

Related Posts