Together with Devon Barrow and Sven Crone, we gave a talk at the recent OR 62 conference, moderated by Christina Phillips. The topic was: “The quest for greater forecasting accuracy: Perspectives from Statistics & Machine Learning”. I have worked with both Devon and Sven in the past years and the three of us share quite a few perspectives on what are the promising avenues for forecasting, but also have our diverging views, influenced by our research interests and interactions with the industry. The discussion reflects that, and I think that there are a few helpful points about the future of the various disciplines in forecasting. Of course, we dutifully avoid making too many forecasts about forecasting!
Last week we run the first workshop of the Forecasting Forum Scandinavia, hoping to start an ongoing discussion between academia and practice around forecasting and predictive analytics. The vision is for this to be the catalyst in:
- providing innovative solutions to real business problems, at a rigorous scientific standard;
- shorten the path to implementing innovative and impactful research to practice;
- create consortia between and within industry and academia to facilitate ambitious research by sharing know-how, resources, and risk.
The topic of the workshop the use of information from the business and market environment to enhance forecasts. You can find slides and recordings of all the talks in the forum’s LinkedIn group.
My talk focused on the academic perspective, and a gave a non-technical overview of:
- What are the elements of a “good” forecast? (I keep the quotation, as I did not touch upon loss functions and objectives.)
- Limitations of extrapolative forecasting and some motivations for using external predictors. (You won’t get me saying causal models! We are still so far away from being able to claim causality!)
- Potential variables to enhance your forecasts and relevant considerations.
You can find the slides here and a recording of the talk below.
A few words on the “we”. With David Fagersand, who is the CEO of Indicio Technologies, we share the view that there is a substantial gap in the interaction between academia and industry on forecasting and predictive analytics, at least in Sweden and neighbouring Scandinavia – although my experience is that this is a wider challenge (more on that below!). We both recognise that there is strength in putting different perspectives and objectives together, to keep some balance between academia and practice. I do not think it is contested that academia can be “too academic” at times, and practice “too practical” (see a previous opinion piece co-authored with Fotios Petropoulos here). Obviously, Indicio is a company and therefore for-profit. I nowadays work at a Skövde university in Sweden, that is a public university, which in line with my ethos for freely accessible knowledge and open-source. My personal view is that bringing these two sides together can only be beneficial! My view for the ideal evolution of initiative is to be less driven by individuals, and more by the interaction in the community. It would be great if organisations would openly speak about the challenges they face and provide the means to universities to help them solve them. It would also be great if more academics would get their hands dirty! And obviously outstanding if it is widely acknowledged that such an initiative to run requires both resources and speakers! So, a call for action for current and future members!
I will expand a bit on this. Over my academic career I had the luck to work with many great colleagues and some of the biggest companies internationally. Naturally, at every country the business culture and the academic attitude differs. It will come as no surprise than some foster impactful research and innovation more than others. I find Sweden to be a great place for this, with both companies and universities focusing more into how to get exciting work done, rather than how to split the pie – necessary, but let’s get the priorities right when you involve academia: we are not consultants (at that point!). I find that organisations (and that includes universities ironically!) often do not understand how resource intensive research has become. It needs time, very skilled people, computing resources, data and time. Did I mention time? More importantly, training new academics is critical, and that requires the investment by all industry, academia and state. Let me also add that the skill often does not come solely from academia. We are all smart people, so if we ask someone/a team to outsmart us all and solve a very difficult problem, at least let’s give then the resources!
I would not expect from academics to always get the economics right (unless they are economists? – of course we have the responsibility to get it right!), but if companies are into money making, they surely understand that there is no free lunch! We face great societal challenges, and we all need to play our part. Improving forecasts is not just fun (for academics), or impacting the bottom line (for companies), it is also important for a more sustainable society and environment in the large scheme of things, but also for meeting the needs of societies. The initiative is called Forecasting Forum Scandinavia, but I would so much like to see the name proving to be wrong and becoming an international community of people eager to solve problems, meet challenges, and contribute!
I am delighted to receive the news that my recent paper with George Athanasopoulos at the European Journal of Operational Research has been selected as the EJOR editor’s choice article for June 2020. My thanks to the editor and the reviewers for their help with their comments and recommendations in improving the paper and bringing it to its current form.
I plan to write a post about the gist of the idea.
Devon Barrow, Nikolaos Kourentzes, Rickard Sandberg, and Jacek Niklewski, 2020. Expert Systems with Applications.
A major challenge in automating the production of a large number of forecasts, as often required in many business applications, is the need for robust and reliable predictions. Increased noise, outliers and structural changes in the series, all too common in practice, can severely affect the quality of forecasting. We investigate ways to increase the reliability of exponential smoothing forecasts, the most widely used family of forecasting models in business forecasting. We consider two alternative sets of approaches, one stemming from statistics and one from machine learning. To this end, we adapt M-estimators, boosting and inverse boosting to parameter estimation for exponential smoothing. We propose appropriate modifications that are necessary for time series forecasting while aiming to obtain scalable algorithms. We evaluate the various estimation methods using multiple real datasets and find that several approaches outperform the widely used maximum likelihood estimation. The novelty of this work lies in (1) demonstrating the usefulness of M-estimators, (2) and of inverse boosting, which outperforms standard boosting approaches, and (3) a comparative look at statistics versus machine learning inspired approaches.
Nikolaos Kourentzes and George Athanasopoulos, 2020. European Journal of Operational Research. https://doi.org/10.1016/j.ejor.2020.05.046
Intermittent demand forecasting has been widely researched in the context of spare parts management. However, it is becoming increasingly relevant to many other areas, such as retailing, where at the very disaggregate level time series may be highly intermittent, but at more aggregate levels are likely to exhibit trends and seasonal patterns. The vast majority of intermittent demand forecasting methods are inappropriate for producing forecasts with such features. We propose using temporal hierarchies to produce forecasts that demonstrate these traits at the various aggregation levels, effectively informing the resulting intermittent forecasts of these patterns that are identifiable only at higher levels. We conduct an empirical evaluation on real data and demonstrate statistically significant gains for both point and quantile forecasts.
Nikolaos Kourentzesa, Juan R. Trapero, and Devon K. Barrow, 2020. International Journal of Production Economics. https://doi.org/10.1016/j.ijpe.2019.107597
Inaccurate forecasts can be costly for company operations, in terms of stock-outs and lost sales, or over-stocking, while not meeting service level targets. The forecasting literature, often disjoint from the needs of the forecast users, has focused on providing optimal models in terms of likelihood and various accuracy metrics. However, there is evidence that this does not always lead to better inventory performance, as often the translation between forecast errors and inventory results is not linear. In this study, we consider an approach to parametrising forecasting models by directly considering appropriate inventory metrics and the current inventory policy. We propose a way to combine the competing multiple inventory objectives, i.e. meeting demand, while eliminating excessive stock, and use the resulting cost function to identify inventory optimal parameters for forecasting models. We evaluate the proposed parametrisation against established alternatives and demonstrate its performance on real data. Furthermore, we explore the connection between forecast accuracy and inventory performance and discuss the extent to which the former is an appropriate proxy of the latter.
Evangelos Spiliotis, Fotios Petropoulos, Nikolaos Kourentzes, and Vassilios Assimakopoulos, 2020. Applied Energy 261: 114339. https://doi.org/10.1016/j.apenergy.2019.114339
Achieving high accuracy in energy consumption forecasting is critical for improving energy management and planning. However, this requires the selection of appropriate forecasting models, able to capture the individual characteristics of the series to be predicted, which is a task that involves a lot of uncertainty. When hierarchies of load from different sources are considered together, the uncertainty and complexity increase further. For example, when forecasting both at system and region level, not only the model selection problem is expanded to multiple time series, but we also require aggregation consistency of the forecasts across levels. Although hierarchical forecasting, such as the bottom-up, the top-down, and the optimal reconciliation methods, can address the aggregation consistency concerns, it does not resolve the model selection uncertainty. To address this issue, we rely on Multiple Temporal Aggregation (MTA), which has been shown to mitigate the model selection problem for low-frequency time series. We propose a modification of the Multiple Aggregation Prediction Algorithm, a special implementation of MTA, for high-frequency time series to better handle the undesirable effect of seasonality shrinkage that MTA implies and combine it with conventional cross-sectional hierarchical forecasting. The impact of incorporating temporal aggregation in hierarchical forecasting is empirically assessed using a real data set from five bank branches. We show that the proposed MTA approach, combined with the optimal reconciliation method, demonstrates superior accuracy, aggregation consistency, and reliable automatic forecasting.
A few weeks ago I gave a talk at Amazon’s 2019 AMLC in Seattle. The talk was focused on current research in temporal and cross-temporal hierarchies. People who have been following my blog will be familiar with the topic and recent advances. This talk is different in the sense that it does not go in the technicalities, but rather looks at the benefits of temporal and cross-temporal hierarchies for the forecasting process of companies. The last few slides outline current research and my view for some interesting upcoming applications.
You can find the slides here.
Many thanks for the invitation to give the forecasting keynote!
I was recently invited to give a talk at AWS in Berlin. I presented the current work on temporal and cross-temporal hierarchical forecasting. My view is that there is a lot of potential for these approaches to augment existing forecasting processes with relative ease.
Considering the wider forecasting problem, we do not forecast for the sake of forecasting, but to support decisions, with different planning horizons, objectives and information base. These approaches permit merging all these views to achieve aligned decision making, while at the same time improving forecast accuracy. On a personal note, it is nice to see that industry is nowadays very fast at considering/adopting new research. Creating in-house teams with forecasting expertise offers tremendous opportunities to companies to capitalise on innovations in research and the open source community that most researches contribute to. If you are not familiar with the research follow the links above to find out more about them.
On another note, if you are not familiar with the forecasting work of AWS, I will point you to their new open source library for forecasting with deep learning: gluonts. I had the chance to discuss with the team some of the internal workings of the library and they have put together a very interesting and useful tool. I hope to find the time to try it some more myself and I will post my results and thoughts here. A word of caution (this is something that the team at AWS also repeat themselves quite often): deep learning is not the solution for all the problems, but has a lot of potential when the data permit. If you deal with limited sales data and only a few time series, perhaps the humble exponential smoothing is still a very good contender. But otherwise, there are a lot of innovations in neural networks to make them a worthy contender for forecasting. Nonetheless, irrespectively of your views on deep learning and forecasting hats off to AWS for contributing back to the research and open source community.
Finally, many thanks to Tim Januschowski for the invitation and hosting me!
I was recently invited to a workshop focused on forecasting and supply chain management at Valencia Polytechnic University. Many thanks to Ester Guijarro for organising the workshop and helping to bring together forecasters and supply chain experts!
I presented on optimising forecasting model parameters for inventory management. You can find the presentation here, and a working version of the paper here. The paper is currently under review, so I would expect quite a few changes in the final version! Whether we are critical of the review process or not, in the vast majority of cases the it improves papers substantially and this will certainly be the case here. The view we take on this work with my co-authors is that we can integrate forecasting and inventory management more closely, and instead of optimising forecasting models to maximise fit on past sales, hoping that this will result in good inventory performance (and there are many good reasons for this to hold!), we can directly optimise so as to minimise deviations from the desired inventory performance. This seems to work quite well in our empirical evaluation.
Juan R Trapero presented a paper we have worked together with Manuel Cardos on calculating empirical safety stocks. You can find the presentation here. It looks at using kernel density estimation and GARCH models to address different deficiencies of standard approaches. Namely, kernels are particularly good at handling asymmetries in the forecast error distributions (promotional forecasts I am looking at you) and GARCH for handling residual autocorrelations. Both cases are quite common in practice, as often our forecasting models are far from the underlying demand generating process. You can find the relevant published paper here, as well as a follow up work here.