Author Archives: Nikos

Incorporating macroeconomic leading indicators in tactical capacity planning

Yves R. Sagaert, Nikolaos Kourentzes, Stijn Du Vuyst, El-Houssaine Aghezzaf and Bram Desmet, 2018, International Journal of Production Economics.

Tactical capacity planning relies on future estimates of demand for the mid- to long-term. On these forecast horizons there is increased uncertainty that the analysts face. To this purpose, we incorporate macroeconomic variables into microeconomic demand forecasting. Forecast accuracy metrics, which are typically used to assess improvements in predictions, are proxies of the real decision associated costs. However, measuring the direct impact on decisions is preferable. In this paper, we examine the capacity planning decision at plant level of a manufacturer. Through an inventory simulation setup, we evaluate the gains of incorporating external macroeconomic information in the forecasts, directly, in terms of achieving target service levels and inventory performance. Furthermore, we provide an approach to indicate capacity alerts, which can serve as input for global capacity pooling decisions. Our work has two main contributions. First, we demonstrate the added value of leading indicator information in forecasting models, when evaluated directly on capacity planning. Second, we provide additional evidence that traditional metrics of forecast accuracy exhibit weak connection with the real decision costs, in particular for capacity planning. We propose a more realistic assessment of the forecast quality by evaluating both the first and second moment of the forecast distribution. We discuss implications for practice, in particular given the typical over-reliance on forecast accuracy metrics for choosing the appropriate forecasting model.

Download paper.

Judgmental Selection of Forecasting Models

Fotios Petropoulos, Nikolaos Kourentzes, Konstantinos Nikolopoulos and Enno Siemsen, 2018, Journal of Operations Management. https://doi.org/10.1016/j.jom.2018.05.005

In this paper, we explored how judgment can be used to improve the selection of a forecasting model. We compared the performance of judgmental model selection against a standard algorithm based on information criteria. We also examined the efficacy of a judgmental model-build approach, in which experts were asked to decide on the existence of the structural components (trend and seasonality) of the time series instead of directly selecting a model from a choice set. Our behavioral study used data from almost 700 participants, including forecasting practitioners. The results from our experiment suggest that selecting models judgmentally results in performance that is on par, if not better, to that of algorithmic selection. Further, judgmental model selection helps to avoid the worst models more frequently compared to algorithmic selection. Finally, a simple combination of the statistical and judgmental selections and judgmental aggregation significantly outperform both statistical and judgmental selections.

Download paper.

Quantile forecast optimal combination to enhance safety stock estimation

Juan R. Trapero, Manuel Cardos and Nikolaos Kourentzes, 2018, International Journal of Forecasting.

The safety stock calculation requires a measure of the forecast error uncertainty. Such errors are usually assumed Gaussian iid (independent, identically distributed). However, deviations from iid deteriorate the supply chain performance. Recent research has shown that, alternatively to theoretical approaches, empirical techniques that do not rely on the aforementioned assumptions, can enhance the safety stock calculation. Particularly, GARCH models cope with time-varying heterocedastic forecast error, and Kernel Density Estimation do not need to rely on a determined distribution. However, if forecast errors are both time-varying heterocedastic and do not follow a determined distribution, the previous approaches are inadequate. To overcome this, we propose an optimal combination of the empirical methods that minimizes the asymmetric piecewise linear loss function, also known as tick loss. The results show that combining quantile forecasts yields safety stocks with a lower cost. The methodology is illustrated with simulations and real data experiments for different lead times.

Download paper.

Another look at forecast selection and combination: evidence from forecast pooling

Nikolaos Kourentzes, Devon K. Barrow and Fotios Petropoulos, 2018, International Journal of Production Economics. https://doi.org/10.1016/j.ijpe.2018.05.019

Forecast selection and combination are regarded as two competing alternatives. In the literature there is substantial evidence that forecast combination is beneficial, in terms of reducing the forecast errors, as well as mitigating modelling uncertainty as we are not forced to choose a single model. However, whether all forecasts to be combined are appropriate, or not, is typically overlooked and various weighting schemes have been proposed to lessen the impact of inappropriate forecasts. We argue that selecting a reasonable pool of forecasts is fundamental in the modelling process and in this context both forecast selection and combination can be seen as two extreme pools of forecasts. We evaluate forecast pooling approaches and find them beneficial in terms of forecast accuracy. We propose a heuristic to automatically identify forecast pools, irrespective of their source or the performance criteria, and demonstrate that in various conditions it performs at least as good as alternative pools that require additional modelling decisions and better than selection or combination.

Download paper.

Empirical safety stock estimation based on kernel and GARCH models

Juan R. Trapero, Manuel Cardos and Nikolaos Kourentzes, 2018, Omega. https://doi.org/10.1016/j.omega.2018.05.004

Supply chain risk management has drawn the attention of practitioners and academics alike. One source of risk is demand uncertainty. Demand forecasting and safety stock levels are employed to address this risk. Most previous work has focused on point demand forecasting, given that the forecast errors satisfy the typical normal i.i.d. assumption. However, the real demand for products is difficult to forecast accurately, which means that—at minimum—the i.i.d. assumption should be questioned. This work analyzes the effects of possible deviations from the i.i.d. assumption and proposes empirical methods based on kernel density estimation (non-parametric) and GARCH(1,1) models (parametric), among others, for computing the safety stock levels. The results suggest that for shorter lead times, the normality deviation is more important, and kernel density estimation is most suitable. By contrast, for longer lead times, GARCH models are more appropriate because the autocorrelation of the variance of the forecast errors is the most important deviation. In fact, even when no autocorrelation is present in the original demand, such autocorrelation can be present as a consequence of the overlapping process used to compute the lead time forecasts and the uncertainties arising in the estimation of the parameters of the forecasting model. Improvements are shown in terms of cycle service level, inventory investment and backorder volume. Simulations and real demand data from a manufacturer are used to illustrate our methodology.

Download paper.

ISF2018 Presentation: Beyond summary performance metrics for forecast selection and combination

Nikolaos Kourentzes, Ivan Svetunkov and Stephan Kolassa, ISF2018, 20th June 2018.

In doing forecast selection or combination we typically rely on some performance metric. For example, that could be Akaike Information Criterion or some cross-validated accuracy measure. From these we can either pick the top performer, or construct combination weights. There is ample empirical evidence demonstrating the appropriateness of such metrics, both in terms of resulting forecast accuracy and automation of the forecasting process. Yet, these performance metrics are summary statistics, that do not reflect higher moments of the metrics. This poses similar issues to analysing only point forecasts to assess the risks associated with a prediction, instead of looking at prediction intervals as well. Looking at summary statistics does not reflect the uncertainty in the ranking of alternative forecasts, and therefore the uncertainty in selection and combination of forecasts. We propose a modification in the use of the AIC and an associated procedure for selecting a single forecast or constructing combination weights that aims to go beyond the use of summary statistics to characterise each forecast. We demonstrate that our approach does not require an arbitrary dichotomy between forecast selection, combination or pooling, and switches appropriately depending on the time series on hand and the pool of forecasts considered. The performance of the approach is evaluated empirically on a large number of real time series from various sources.

Download slides.

ISF2018 Presentation: Information Sharing in the Presence of Promotions in a Supply Chain

Patrick Saoud, Nikolaos Kourentzes and John Boylan, ISF2018, 20th June 2018

Many supply chains experience the Bullwhip effect, defined as the upstream amplification of demand variability. This information distortion results in a misalignment of forecasts, generating expensive business costs. A proposed remedy in the literature is the sharing of Point of Sales information data among the members of the supply chain. The theoretical and empirical results have pointed in different directions, with the empirical evidence suggesting that information sharing helps achieve better forecasting accuracy. A less studied facet of the Bullwhip is the effect of promotions on it, which was highlighted as one of its four original sources. This research is dedicated to examining the effect of promotions and other demand shocks on the performance of the different tiers of the supply chain. In particular, it will study the impact of promotions on forecasting accuracy, Bullwhip propagation and safety stocks for the participants of the supply chain. Furthermore, it will also investigate the impact of different types of information sharing in this context on the Supply Chain, and compare their performance in terms of gains in forecasting accuracy.

Download slides.

Demand forecasting with user-generated online information

O. Schaer, N. Kourentzes and R. Fildes, 2018, International Journal of Forecasting. https://doi.org/10.1016/j.ijforecast.2018.03.005

Recently, there has been substantial research on augmenting aggregate forecasts with individual consumer data from internet platforms, such as search traffic or social network shares. Although the majority of studies report increased accuracy, many exhibit design weaknesses including lack of adequate benchmarks or rigorous evaluation. Furthermore, their usefulness over the product life-cycle has not been investigated, which may change, as initially, consumers may search for pre-purchase information, but later for after-sales support. In this study, we first review the relevant literature and then attempt to support the key findings using two forecasting case studies. Our findings are in stark contrast to the literature, and we find that established univariate forecasting benchmarks, such as exponential smoothing, consistently perform better than when online information is included. Our research underlines the need for thorough forecast evaluation and argues that online platform data may be of limited use for supporting operational decisions.

Download paper and online supplement.

ISF 2018 Presentation: The inventory impact of including macroeconomic leading indicators in global supply chain management

Yves R. Sagaert, Nikolaos Kourentzes, Stijn De Vuyst and El-houssaine Aghezzaf, ISf2018, 19th June 2018

Supply chain management is increasingly performed at a global level. The decision process is often based on tactical sales forecasts, which has been shown to benefit from including relevant exogenous information. Leading indicators that cover different aspects of macroeconomic dynamics are appealing in this context, as macroeconomic dynamics in target countries can affect companies end markets. Even though this information can be beneficial on a tactical level, it remains unclear how this information can impact sales forecasts at Stock-Keeping-Unit (SKU) product level, due to increased levels of noise and products having differing demand patterns and dynamics, masking macro-effects. Nonetheless, hierarchical forecasting can be used to reconcile macroeconomic leading indicators from tactical level forecasts to detailed SKU levels, and vice versa. In this paper, we evaluate the feasibility and benefits of merging tactical and operational forecasting, where higher level forecasts include leading indicators, in contrast to univariate SKU operational predications. We present a framework that identifies automatically the most relevant leading indicators on global sales level, and by exploiting the hierarchical product structure, carries this information to sales forecasts at SKU product level. For our evaluation we rely on inventory metrics obtained from simulation experiments, reflecting the associated supply chain risk.

Download slides.

ISF2018 Presentation: Interpreting algorithmic and qualitative information when making judgmental forecast adjustments

Anna Sroginis, Robert Fildes and Nikolaos Kourentzes, ISF2018, 19th June 2018

Despite the continuous improvements in statistical forecasting, human judgment remains essential in business forecasting and demand planning. Typically, forecasters do not solely rely on statistical forecasts, which are obtained from various Forecasting Support Systems (FSS); they also adjust forecasts according to their knowledge, experience and information that is not available to the statistical models. However, we do not have adequate understanding of the adjustment mechanisms, particularly how people use additional information (e.g. special events, promotions, strikes, holidays etc.) and under which conditions this is beneficial. To investigate this, we conduct experiments that simulate a typical supply chain forecasting process that additionally provides qualitative and model-based information about past and future promotional periods for retail products. Using laboratory experiments, we find that when making adjustments people tend to focus on several anchors: the last promotional uplift, current statistical forecast and contextual statements for the forecasting period. At the same time, participants ignore the past baseline promotional uplifts and domain knowledge about the past promotions. They also discount statistical models with incorporated promotional effects, hence showing lack of trust in algorithms. These results highlight the need for more fundamental understanding of processes behind human adjustments and the reasons for them since it can help to guide forecasters in their tasks and to increase forecast accuracy.

Download slides.