This is joint work with Fotios Petropoulos and Kostantinos Nikolopoulos and discusses the performance of experts selecting forecasting models, against automatic statistical model selection, as well as providing guidelines how to maximise the benefits. This is very exciting research, demonstrating the both some limitations of statistical model selection (and avenues for new research), as well as the advantages and weaknesses of human experts performing this task.<\/p>\n
Abstract<\/strong><\/p>\n In this paper we explore how judgment can be used to improve model selection for forecasting.We benchmark\u00a0the performance of judgmental model selection against the statistical one, based on information criteria. Apart\u00a0from the simple model choice approach, we also examine the efficacy of a judgmental model build approach,\u00a0where experts are asked to decide on the existence of the structural components (trend and seasonality) of the\u00a0time series. The sample consists of almost 700 participants that contributed in a custom-designed laboratory\u00a0experiment. The results suggest that humans perform model selection differently than statistics. When\u00a0forecasting performance is assessed, individual judgmental model selection performs equally if not better to\u00a0statistical model selection. Simple combination of the statistical and judgmental selections and judgmental\u00a0aggregation significantly outperform both statistical and judgmental selection.<\/p>\n