New R package nnfor: time series forecasting with neural networks

By | October 25, 2017

My new R packageĀ nnfor is available on CRAN. This collects the various neural network functions that appeared in TStools. See this post for demo of these functions. In summary the package includes:

  • Automatic, semi-automatic or fully manual specification of MLP neural networks for time series modelling, that helps in specifying inputs with lags of the target and exogenous variables. It can automatically deal with pre-processing (differencing and scaling) and identify the number of hidden nodes. The user can control which of these settings are left on automatic or not.
  • A few options for building network ensembles.
  • Plotting functions of the network topology, fit and forecast.
  • All the above for ELMs (Extreme Learning Machines).
  • Support for Temporal Hierarchies Forecasting, with the thief package for R.

This builds on the neuralnet package for R, and provides the code to make the networks capable of handling time series data automatically. Although that package is quite flexible, it is computationally expensive and does not permit for deep learning. The plan is to eventually implement such capabilities in the package.

There are numerous papers that support the ideas used to put together this package:

  • In my new book, Ord et al., 2017, Principles of Business Forecasting, 2e, Wessex Press Publishing. Chapter 10 describes the basic logic in building MLP networks for time series forecasting. This package implements the logic described there.
  • This paper demonstartes the performance of the input variable selection algorithm: Crone and Kourentzes, 2010, Feature selection for time series prediction – a combined filter and wrapper approach for neural networks. Neurocmputing, 73, 1923-1936. There is some influence from this proceedings paper as well. (These feel like really old papers!)
  • This paper looks at the combination operator for the ensembles. Please move away from the average! Kourenztes et al., 2014, Neural network ensembles operators for time series forecasting. Expert Systems with Applications, 41, 4235-4244.

The neural network functions in TStools will be removed, initially pointing towards this package and latter removed completely.

There is a github repository for this, where I will be posting updates and fixes till they go on CRAN: https://github.com/trnnick/nnfor

Happy (nonlinear) forecasting!

6 thoughts on “New R package nnfor: time series forecasting with neural networks

  1. Alex Raboin

    I have been following your work and was excited to see you made progress on getting this on cran.

    I am having two issues with switching from nnetar to mlp…

    One, I am getting an error for non conforming arguments….

    and two, how do you implement forward regressors?

    In nnetar i would cbind the time series objects of forward regressors into freg, then use xreg = freg in the forecast argument.

    I am looking forward to upgrading to using this and figuring how to use thief too, seems impact in what you have been working on.

    and I am hopeful I will see similar results in the short term natural gas load forecasting I am doing.

    Thanks for the help. Cheers!

    Reply
    1. Nikos Post author

      First, I found a small bug that only appears when there are no univariate lags and the modeller wants to turn off automatic input selection. I put up a github repository here: https://github.com/trnnick/nnfor. The bug is fixed there.

      I wrote a R example to show how to handle xreg inputs with mlp(). There are two options. Either input the dynamics in the xreg.lags or create an xreg where each column is the appropriate dynamic manually and set xreg.lags to zeros. However, I have not implemented leads (forward regressors) yet, so you will have to go with option two. The following example should be helpful.

      library(nnfor)
      
      # The objective is to forecast the Airline Passengers series with only deterministic trend and seasonality
      # mlp does the deterministic seasonality internally, when needed, but not the trend.
      
      # Let us prepare some data
      y <- AirPassengers
      h <- 2*frequency(y)
      tt <- cbind(c(1:(length(y)+h),rep(0,2*h)))
      # Observe that the deterministic trend ends with zeros
      print(tt)
      
      # Fit a network with no differencing, no univariate lags, and fixed deterministic trend
      fit1 <- mlp(y,difforder=0,lags=0,xreg=tt,xreg.lags=list(0),xreg.keep=TRUE)
      print(fit1)
      plot(fit1)
      plot(forecast(fit1,h=h,xreg=tt))
      # The forecast is reasonable
      
      # Now let us shift the input so that the zeros are in the forecast period
      tt2 <- tt[-(1:h),,drop=FALSE]
      plot(forecast(fit1,h=h,xreg=tt2))
      # The seasonality is there, but there is zero trend, as the inputs suggest. 
      # Also note that the mlp modelled multiplicative seasonality on its own. NNs are cool. 
      
      # Now let us fit a network on the shifted inputs
      # I will ask for outplot=1 to see the model fit
      fit2 <- mlp(y,difforder=0,lags=0,xreg=tt2,xreg.lags=list(0),xreg.keep=TRUE,outplot=1)
      plot(forecast(fit2,h=h,xreg=tt2))
      # Same as before
      
      # Now lets fit with two inputs, the shifted (lead of 24 periods) and the original trend
      fit3 <- mlp(y,difforder=0,lags=0,xreg=cbind(tt[1:192,,drop=FALSE],tt2),xreg.lags=list(0,0),xreg.keep=list(TRUE,TRUE),outplot=1)
      print(fit3)
      plot(forecast(fit3,h=h,xreg=cbind(tt[1:192,,drop=FALSE],tt2)))
      # The network gets a bit confused with one of the trend vectors stopping!
      

      Hope this helps!

      Reply
      1. Alex Raboin

        Thanks! Very helpful. I’m training away. What are your thoughts on lead inputs (forward regressors) ?

        Computational time is killing me, With so many networks to train for an ensemble model (84) I’ve set to train all day then save models, load then forecast the next morning. I have a rolling different dates to look at….
        I should be able to utilize a gpu to speed this up right? Ive been looking at a few R packages with CUDA. I’m uncertain if I can actually utilize the hardware. I have been looking through some documentation but nothing definitive yet.

        I appreciate your input you an are expert, I am still working to perfect my forecasting application with the latest and greatest.

        Thank you!

        Reply
        1. Nikos Post author

          The computational time is a big issue. At the core of the nnfor, for now, I am calling the neuralnets package, which unfortunately does not use the GPU, and relies on CPU. This makes training rather slow. A potential way forward might be to use ELMs with the elm() function. The argument barebone=TRUE for elm() may help as well. On my todo list for the package is to implement an alternative core that will make use of GPU to speed up processing and also allow experimenting with deep learning (though I am currently very skeptical if it is needed for time series forecasting, at least I am skeptical for now!) – when I run large neural network experiments I use the neural networks toolbox in MatLab, which is very fast, but unfortunately not free or open source.

          Using regressors with leads can be an useful at times. The typical restriction is that we often do not have information about a regressor for the future periods, so that is one of the reasons you do not see this happening often in examples. I have used leads quite a bit in promotional modelling, where you know the promotional plan in advance for the future periods, and therefore this information is available and known with certainty. In that particular case leads are very useful. I would suspect that if you know your regressors with some confidence then they should be helpful as well. If your regressors are forecasted, then you carry any forecast errors in your new forecast that uses them as inputs – this may be a mixed bag, depending on how good is the forecast of the regressors.

          Reply
          1. Josh

            Hi Nikos,

            Thanks for your work here. I’m very interested in neural network applications in time series. I was curious if you’ve seen the work being done with LSTMs and time series? Here’s a link to some work being done at Uber regarding utilizing autoencoders to capture time series properties:
            https://arxiv.org/abs/1709.01907

            Also this recent Kaggle winner applied similar techniques to win a forecasting competition:
            https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39367

            I’m interested to see what you discover for deep learning applications.

          2. Nikos Post author

            Hi Josh,
            Many thanks for the links. I will read them in detail. Quickly skimming through the first paper I noticed that there are no simple statistical benchmarks (something as simple as exponential smoothing!) in the empirical evaluation. It is a shame, as this would help make NNs and deep learning more mainstream in the forecasting literature, but also help highlight whether the gains are over NNs or more general. These two literatures need to talk more to each other šŸ™‚ When I get the time, I will investigate more and post my findings!

Leave a Reply

Your email address will not be published. Required fields are marked *