Measuring the behaviour of experts on demand forecasting: a complex task

N. Kourentzes, J. R. Trapero and I. Svetunkov, 2014.

Forecasting plays a crucial role in decision making and accurate forecasts can bring important benefits for organizations. Human judgement is a significant element when preparing these forecasts. Judgemental forecasts made by experts may influence accuracy, since experts can incorporate information difficult to structure and include in statistical models. Typically, such judgemental forecasts may enhance the accuracy under certain circumstances, although they are biased given the nature of human behaviour. Although researchers has been actively looking into possible causes of human bias, there has been limited research devoted to empirically measuring it, to the extent that conclusions can be totally divergent depending on the error metric chosen. Furthermore, most of the error metrics are focused on quantifying the magnitude of the error, where the bias measure has remained relatively overlooked. Therefore, in order to assess human behaviour and performance, an error metric able to measure both the magnitude and bias of the error should be designed. This paper presents a novel metric that overcomes the aforementioned limitations by using an innovative application of the complex numbers theory. The methodology is successfully applied to analyse the judgemental forecasts of a household products manufacturer. This new point of view is also utilized to revisit related problems as the mechanistic integration of judgemental forecasts and the bias-accuracy trade-off.

Download paper.