{"id":1417,"date":"2017-09-19T14:16:59","date_gmt":"2017-09-19T14:16:59","guid":{"rendered":"http:\/\/kourentzes.com\/forecasting\/?p=1417"},"modified":"2017-09-19T14:16:59","modified_gmt":"2017-09-19T14:16:59","slug":"or59-keynote-uncertainty-in-predictive-modelling","status":"publish","type":"post","link":"https:\/\/kourentzes.com\/forecasting\/2017\/09\/19\/or59-keynote-uncertainty-in-predictive-modelling\/","title":{"rendered":"OR59 Keynote: Uncertainty in predictive modelling"},"content":{"rendered":"

I recently presented at the OR59<\/a> conference my views and current work (with colleagues) on uncertainty in predictive modelling. I think this is a topic that deserves quite a bit of research attention, as it has substnatial implications for estimation, model selection and eventually decision making.<\/p>\n

The talk has three parts:<\/p>\n

\n

Argue (as others before me!) that model based uncertainty (the sigma we get from our models) is not the full story, and estimation\/model uncertainty should be accounted in prediction intervals and decision making. Key point: most model outputs assume that the model itself is true, which is… not true!<\/li>\n

Provide initial results from an approach to directly account for model selection uncertainty that leads is improvements in model selection, but also leads naturally to model combination, aswering what and when to combine.<\/li>\n

Demonstrate how work in Multiple Temporal Aggregation<\/a> is an effective way at addressing modelling uncertainty, summarising research up to this point.<\/li>\n<\/ul>\n
You can download the talk here<\/a>.<\/p>\n
Abstract:<\/p>\n
Forecasts are central to decision making. Over the last decades there have been substantial innovations in business forecasting, resulting in increased accuracy of forecasts. Models and modelling principles have matured to address company problems in a realistic sense, i.e. they are aware of the requirements and limitations of practice; and tested empirically to demonstrate their effectiveness. Furthermore, there has been a shift in recognising the importance of having models instead of methods to facilitate parameterisation, model selection and the generation of prediction intervals. The latter has been instrumental in refocusing from point forecasts to prediction intervals, which reflect the relevant risk for the decisions supported by the forecasts. At the same time the quality and quantity of potential model inputs has increased exponentially, permitting models to use more information sources and support higher frequency of decision making, such as daily and weekly planning cycles. All these have facilitated and made necessary an increase in automation of the forecasting process, bringing to the forefront a new dimension of uncertainty: the model selection and specification uncertainty. The uncertainty captured in the prediction intervals assumes that the selected model is `true\u2019. This is hardly the case in practice and we should account for that additional uncertainty. First, we discuss the uncertainties implied in model selection and specification. Then we proceed to develop a way to measure this uncertainty and derive a new way to perform model selection. We demonstrate that that this not only leads to superior selection, but also provides a natural link to model combination and specifying the relevant pool of models. Last, we demonstrate that once we recognise the uncertainty in model specification, we can extract more information from our data by using the multiple temporal aggregation frameworks, and empirically show the achieved increase in forecast accuracy and reliability.<\/p>\n
Related Posts<\/H3>