Author Archives: Nikos

Solar irradiation forecasting based on dynamic harmonic regression

J.R. Trapero, N. Kourentzes, A. Martin, 2015, 27th European Conference on operational Research, Glasgow.

Solar power generation is a crucial research area for countries that have high dependency on fossil energy sources and count on high solar resource potential. In order to integrate the electricity generated by solar power plants into the grid, solar irradiation must be reasonably well forecasted, where deviations of the forecasted value from the actual measured value involve significant costs. The present paper proposes a univariate Dynamic Harmonic Regression model set up in a State Space framework for short-term (1 to 24 hours) solar irradiation forecasting. The DHR is a type of Unobserved Components model that can be considered as an extension of the typical harmonic regression, where the coefficients are time-varying. This method provides a fast automatic identification and estimation procedure based on the frequency domain. Furthermore, the recursive algorithm as the Kalman Filter is employed to yield adaptive predictions. Time series hourly aggregated as the Global Horizontal Irradiation and the Direct Normal Irradiation will be used to illustrate the proposed approach. The good forecasting performance is illustrated with solar irradiance measurements collected from ground-based weather stations located in Spain. The results show that the Dynamic Harmonic Regression achieves a relative Root Mean Squared Error about 30% and 47% for the Global and Direct irradiation components, respectively, for a forecast horizon of 24 hours ahead.

Download presentation.
Related paper can be found here.

Related Posts

Do sales depend on search traffic? The case of video games

O. Schaer, N. Kourentzes, 2015, 27th European Conference on operational Research, Glasgow.

Public available search engine (SE) data such as from Google Trend can offer insights on the search interest for a specific product. Nowadays, customers often use search engines to inform themselves about a product prior to the purchase. This permits new modelling approaches that can improve forecasting accuracy by using it as a leading indicator. Several studies already focused on the correlation of SE data and actual sales but rather focused on brand-level or initial phase of a new product only. This study focuses on describing the relationship between search engine and sales data considering the entire product life span using weekly video games sales data. At first, we evaluate the suitability of search volume data as a leading indicator and focus especially on the detection of causal dependency, overcoming a common limitation in many studies using SE data. In particular, exploring the direction of causality is interesting, as during the mature phase of a product, search traffic can also be caused by users of already sold entities and therefore may no longer be causal of new sales. According to those findings we propose appropriate forecasting models that incorporate SE data for short and long-term forecasting and evaluate their accuracy using real sales data from the gaming industry.

Download presentation.

Related Posts

Causal leading indicators detection for demand forecasting

Y.R. Sagaert, N. Kourentzes, E-H. Aghezzaf, B. Desmet, 2015, 27th European Conference on operational Research, Glasgow.

Demand forecasts are often univariate, or include only limited causal promotional information on a short-term horizon, which do not capture changing long-term global markets. More especially, including causal exogenous information in the forecasting models could enrich the long-term forecast. The limited historical data is typically used to both identify the current univariate structure and select the appropriate causal leading indicators from a large set of exogenous variables. A key challenge is to be able to distinguish between correlated and causal variables. The resulting variable selection problem is well studied in literature, but far from resolved. Furthermore, the problem gets harder by the limited available historical data in this context of business forecasting. The amount of historical sales observations is far less than the size of the pool of potential causal leading indicators. Methodologies from heuristics to shrinkage estimators, such as LASSO, are examined to overcome the variable selection problem. In a case study, we use real demand data from a global manufacturer and potential causal leading macro-economic indicators from the different global markets the manufacturer trades in.

Download presentation.

Related Posts

Forecasting intraday arrivals at a call centre using neural networks: forecasting anomalous days

D. Barrow, N. Kourentzes, 2015, 27th European Conference on operational Research, Glasgow.

A key challenge for call centres remains the forecasting of high frequency call arrivals collected in hourly or shorter time buckets. These forecasts are required for decisions concerning the scheduling, hiring and training of staff. In addition to the high frequency nature of call arrival series and the complex seasonal patterns, including the multiple seasonal cycles, call arrival data often contain a large number of anomalies, driven by holidays, special events, promotional activities and system failures. This study presents an approach based on artificial neural networks (ANNs) for forecasting intraday call arrivals. In so doing, we empirically evaluate alternative methodologies for modelling and forecasting outliers in high frequency data, which span over several periods, addressing a gap in research of practical significance considering the difficulty and the cost associated with manual exploration and treatment of such data. We assess the performance of different ANN modelling methodologies in terms of the accuracy with which normal and outlying periods are modelled. Multi-period outliers are modelled using alternative encodings ranging from binary dummy variables to functional profiles, as well as segmenting the series to separate it into outlying and normal days. Results show that ANNs outperform conventional benchmarks and are capable of modelling high frequency outliers using relatively simple outlier modelling approaches.

Download presentation.

Related Posts

Trace likelihood and shrinkage in estimation of the forecasting models

I. Svetunkov, N. Kourentzes, 2015, 27th European Conference on operational Research, Glasgow.

The standard approach for estimating forecasting models is usually based on the likelihood function using one-step-ahead forecasting error. But several empirical studies have shown over the years that minimizing the sum of squared errors for multiple steps-ahead forecasts may lead to more accurate parameters estimation. Although it is intuitive that aligning the forecasting objective with the optimisation cost function is beneficial, we propose a statistically justified theoretical rationale to do this. We also demonstrate the difference between the traditional multi-steps-ahead objective function and the objective function obtained from the proposed trace likelihood function. In addition it has been argued in the literature that in the absence of a true model, it is preferable to use an extended likelihood, using multiple steps-ahead forecast error. We extend this finding by proving that maximising multi-steps-ahead likelihood is equivalent to single-step-ahead optimisation with parameter shrinkage. Therefore, maximising the proposed likelihood both incorporates the forecasting objective in the estimation and overcomes estimation limitations due to sampling or model form. We validate our theoretical findings by conducting experiments on real data and showing the advantage of the proposed approach in comparison to the standard likelihood function maximization.

Download presentation.

Related Posts

Re-evaluating the bullwhip effect measurement: what are we capturing?

P. Saoud, N. Kourentzes, J. Boylan, 2015, 27th European Conference on operational Research, Glasgow.

A major problem that supply chains face is the Bullwhip effect, which manifests itself by an upstream increase in the variability of demand. This phenomenon bears costly implications on the firms in the supply chain, and thus has been at the topic of numerous studies. Even though a vast body of literature has been dedicated to alleviating it, a much smaller effort has been placed in quantifying it. We proposes a new measurement to gauge the bullwhip effect, after highlighting the possible flaws of the current ones. Indeed, establishing adequate measures proves to be crucial in evaluating any contribution to dampen the Bullwhip Effect. The most pervasive metric employed is the ratio of variance, which consists of computing the ratio of variance of order placed over the variance of the demand at each node. Despite its ubiquity, this measure fails to serve its purpose on several occasions, such as in the case of seasonal demand, which is frequently encountered in real life. It also penalizes promotions, which appear as outliers, another driver of the Bullwhip Effect. An additional issue associated with this measurement is the nature of the costs that the metric ought to reflect. The fluctuations of the demand variability does not translate directly into the possible costs that can be incurred, nor does it assess the performance of the supply chain as a whole. This paper will investigate potential caveats and costs of the ratio of variance metric before introducing a new measure.

Download presentation.

Related Posts

ISF 2015 and invited session on “Forecasting with Combinations and Hierarchies”

The International Symposium on Forecasting (ISF 2015) was held this week in Riverside, CA. It was a very interesting conference, with stimulating talks and a wide variety of forecasting related topics, both for academics and practitioners. It is a highly recommended conference to attend.

I organised the invited session on the topic of “Forecasting with Combinations and Hierarchies”. In my opinion these two are highly related topics, as hierarchical and grouped forecasting can be seen as a special case of forecast combinations. There were four talks in the this session, which you can find here:

  1. Wickramasuriya S., Athanasopoulos G., Hyndman R. J., Forecasting Hierarchical Time Series Through Trace Minimization.
  2. Athanasopoulos G., Hyndman R. J., Kourentzes N., Petropoulos F., Forecasting With Temporal Hierarchies.
  3. Barrow D., Kourentzes N., To Combine Forecasts Or To Combine Forecast Models?
  4. Athanasopoulos G., Hyndman R. J., Kourentzes N., Petropoulos F., Using Temporal Hierarchies In Practice.

In boldface are the presenters. Presentations 2. and 4. are on the same topic, with George’s presentation discussing the theory behind temporal hierarchies and mine looking at case studies and implications for practice. Temporal hierarchies is the next step from the Multiple Aggregation Prediction Algorithm (MAPA). In contrast to MAPA, temporal hierarchies are forecasting method independent and more powerful in terms of flexibility and combination options.

The session attracted interesting comments and discussions on the various presentations and I hope the audience enjoyed both the topics and the discussions.

Related Posts

Holt Winters method example

From time to time people have asked me how to implement Holt Winters (trend-seasonal exponential smoothing) in Excel. Let me start by saying that although Excel is probably the most common forecasting tool in business, it is also not a good one. It does not provide many useful options and tools and there is plenty of space for mistakes.

I have produced a small example of Holt Winters that you can download. It comes with two options, depending on how the initial values are calculated. The first option is using a simple heuristic, while the second requires finding optimal initial values with solver.

Two words of caution: i) Excel’s optimiser can easily get stuck to local minima, so try to start with reasonable starting values; ii) Do not use Holt-Winters method when you do not have trend-seasonal data. Instead prefer the simpler Holt or Single Exponential Smoothing methods.

You can download the example spreadsheet here.

My personal view is that organisations should invest in expertise in forecasting and appropriate systems. There are multiple benefits to be gained by accurate and robust forecasts. A relatively inexpensive solution is using R. Several forecasting related packages exist. Prof. Rob Hyndman maintains the excellent forecast package that includes a state-of-the art exponential smoothing implementation. Alternatively you can consider the TStools package that includes a very flexible exponential smoothing implementation by Ivan Svetunkov, amongst other useful forecasting tools and methods (like the Theta method) or try the MAPA package, which implements the Multiple Aggregation Prediction Algorithm that has demonstrated very good performance relatively to exponential smoothing.

Related Posts

Short-term Solar Irradiation forecasting based on Dynamic Harmonic Regression

J.R. Trapero, N. Kourentzes and A. Martin, 2015, Energy, 84: 289-295. http://dx.doi.org/10.1016/j.energy.2015.02.100

Solar power generation is a crucial research area for countries that have high dependency on fossil energy sources and is gaining prominence with the current shift to renewable sources of energy. In order to integrate the electricity generated by solar energy into the grid, solar irradiation must be reasonably well forecasted, where deviations of the forecasted value from the actual measured value involve significant costs. The present paper proposes a univariate Dynamic Harmonic Regression model set up in a State Space framework for short-term (1 to 24 hours) solar irradiation forecasting. Time series hourly aggregated as the Global Horizontal Irradiation and the Direct Normal Irradiation will be used to illustrate the proposed approach. This method provides a fast automatic identification and estimation procedure based on the frequency domain. Furthermore, the recursive algorithms applied offer adaptive predictions. The good forecasting performance is illustrated with solar irradiance measurements collected from ground-based weather stations located in Spain. The results show that the Dynamic Harmonic Regression achieves the lowest relative Root Mean Squared Error; about 30% and 47% for the Global and Direct irradiation components, respectively, for a forecast horizon of 24 hours ahead.

Download paper.
EURO2015 presentation slides.

Related Posts

EURO 2015 special section update

An update with regards to the special session on Energy Forecasting in the upcoming EURO conference. The abstract submission code for this session is: 4e0e6c1f

Information about this special session of the Forecasting & Time Series Prediction stream can be found in the original post.

Related Posts