Forecasting by Neural Network

Hello everyone ! I am entirely new to the "Neural Network Toolbox". However I have explored many example data sets provided in NN toolbox , but i am having trouble in fitting my own case study into it . i am frustrated and completely drained out after following many webinars and searching here. Your help will be highly appreciated . My problem is as follows .

 I have the data set which contains monthly demand of 30  vehicle models of a heavy commercial automobile manufacturer for the past 3 years. In short , my datasheet is of size [ 30 X 36 ] , ( 30  vehicle models are represented   row wise) .

I want to forecast the future demand of all vehicles by using Neural Network . I have tried using Curve Fitting , Time series (both NAR and NARX) by the default setting and various training algorithms , But still the MSE is not acceptable . Is it just because of my "SMALL data set" ? or anything else ?

Kindly suggest me some method (or some way ) , So that i can work with the same dataset and predict the future values by using Neural Network .

Your coperation will be higly praised . Thank You all in advance ( Please ignore my bad english)

[Greg Heath]

Insufficient information. Need quantitative details of what you did.

[pradeep kumar]

@ Greg Heath , Thanks for your response . I have tried to perform a NAR by considering only one vehicle model output (dataset of 37x1) , I have generated the following code for that ,but I am not satisfied with the accuracy and the MSE . Kindly help me

% Solve an Autoregression Time-Series Problem with a NAR Neural Network
% Script generated by Neural Time Series app
% Created Fri Dec 19 11:43:35 IST 2014
%
% This script assumes this variable is defined:
%
%   myoutput2518 - feedback time series.

T = tonndata(myoutput2518,false,false);

% Choose a Training Function
% For a list of all training functions type: help nntrain
% 'trainlm' is usually fastest.
% 'trainbr' takes longer but may be better for challenging problems.
% 'trainscg' uses less memory. NTSTOOL falls back to this in low memory situations.
trainFcn = 'trainbr';  % Bayesian Regularization

% Create a Nonlinear Autoregressive Network
feedbackDelays = 1:2;
hiddenLayerSize = 10;
net = narnet(feedbackDelays,hiddenLayerSize,'open',trainFcn);

% Prepare the Data for Training and Simulation
% The function PREPARETS prepares timeseries data for a particular network,
% shifting time by the minimum amount to fill input states and layer states.
% Using PREPARETS allows you to keep your original time series data unchanged, while
% easily customizing it for networks with differing numbers of delays, with
% open loop or closed loop feedback modes.
[x,xi,ai,t] = preparets(net,{},{},T);

% Setup Division of Data for Training, Validation, Testing
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

% Train the Network
[net,tr] = train(net,x,t,xi,ai);

% Test the Network
y = net(x,xi,ai);
e = gsubtract(t,y);
performance = perform(net,t,y)

% View the Network
view(net)

% Plots
% Uncomment these lines to enable various plots.
%figure, plotperform(tr)
%figure, plottrainstate(tr)
%figure, plotresponse(t,y)
%figure, ploterrcorr(e)
%figure, plotinerrcorr(x,e)

% Closed Loop Network
% Use this network to do multi-step prediction.
% The function CLOSELOOP replaces the feedback input with a direct
% connection from the outout layer.
netc = closeloop(net);
[xc,xic,aic,tc] = preparets(netc,{},{},T);
yc = netc(xc,xic,aic);
perfc = perform(net,tc,yc)

% Step-Ahead Prediction Network
% For some applications it helps to get the prediction a timestep early.
% The original network returns predicted y(t+1) at the same time it is given y(t+1).
% For some applications such as decision making, it would help to have predicted
% y(t+1) once y(t) is available, but before the actual y(t+1) occurs.
% The network can be made to return its output a timestep early by removing one delay
% so that its minimal tap delay is now 0 instead of 1.  The new network returns the
% same outputs as the original network, but outputs are shifted left one timestep.
nets = removedelay(net);
[xs,xis,ais,ts] = preparets(nets,{},{},T);
ys = nets(xs,xis,ais);
stepAheadPerformance = perform(net,ts,ys)

[Greg Heath]

Why are you dissatisfied with the performance?

Using the target variance

 MSE00 = var(cell2mat(t,1))

as a reference, what are the following normalized, scale free measures of performance

 nperf    = performance/MSE00
 ntrnperf = tr.best_perf/MSE00
 ntstperf = tr.best_tperf/MSE00
 nperfc   = perfc/MSE00

[pradeep kumar]

@ Greg Heath , Respected Sir , Thanks once again for your response . I am not satisfied because the MSE is very large (not even close to zero) . How can i trust this network to predict future values . additionally , How can i able to predict the demand for 37th , 38th , 39th month and so on . Please explain me in detail as i am entirely new to this field . Thank you in advance

[Greg Heath]

           You did not answer my question!!!???

[pradeep kumar]

@ Greg heath , Respected Sir , I am not getting , what exactly you are trying to say . If you can just explain briefly , that will be very benificial . I Thank you for your coperation .

[Greg Heath]

You say the net doesn't yield a low enough error. However you do not prove it by normalizing the four mse performance values I listed above by the average target variance.

I consider a design acceptable if it can model at least 99% of the average target variance, i.e.,

 NMSE = mse(error)/MSE00 <= 0.01  % "N" stands for normalized.

In fact, if you search NEWSGROUP and ANSWERS using

 greg MSEgoal

you will see what I mean. In some of those posts you will see the ratio Ndof/Ntrneq used as a correction factor that takes into account the optimistic bias of evaluating performance with the same data that is used to obtain the weight values.

So, what are those four values I asked for?

[pradeep kumar]

@ Greg Heath , Respected Sir , please find the following values ypu have asked for . Now please tell me how can i modify the network to predict future values. Thank you.

nperf = 0.2774

ntrnperf = 0.0602

ntstperf = 1.1145

nperfc = 3.1906

NMSE = mse(e)/MSE00 = 0.2774

[Greg Heath]

This performance is not good enough for closeloop prediction. Consider

1. Estimating better values for feedback delays by obtaining the significant delays of the autocorrelation function

2. Determining, by trial and error, the smallest good value for the number of hidden nodes

3. Taking a closer look at my previous posts

 greg nncorr narnet
 greg narnet Hub
 greg narnet Ntrials
 greg narnet closeloop

HTH

Greg

[pradeep kumar]

@ Greg Heath , Respected Sir . As per your instructions i have tried to rectify the network , and the values are as follows .

MSE00 = var(cell2mat(t)) % since my inputs are taken row wise it should not be MSE00 = var(cell2mat(t,1)) .

MSE00 =

    1.0021

>> performance

performance =

    0.0084

>> nperf = performance/MSE00

nperf =

    0.0084

>> ntrnperf = tr.best_perf/MSE00

ntrnperf =

    0.0059

>> ntstperf = tr.best_tperf/MSE00

ntstperf =

    0.0200

>> nperfc = perfc/MSE00

nperfc =

    1.6429

>> NMSE = mse(e)/MSE00

NMSE =

    0.0084

I think these values are now upto your expectations . now please tell me how am i able to PREDICT(FORECAST) the next values which are not included in my histrical database . Thanks in advance and have a nice weekend . -Pradeep

Answer 1

[Greg Heath]

It is straightforward. The basic logic is

0. Standardize T using cell2mat and zscore

1. Minimize H subject to NMSEo <= 0.005

neto                   = narnet(FD,H);
[ Xo Xoi Aoi To ]      = preparets( neto,{},{}, T );
neto                   = configure( neto, Xoi, Aoi);
[ neto Yo Eo Xof Aof ] = train( neto, Xo, To, Xoi, Aoi );
% [ Yo Xof Aof ]         = net( Xo, Xoi ,Aoi );
% Eo                     = gsubtract(To,Yo);
NMSEo                  = mse(Eo)/mean(var(cell2mat(To}',1))

2. Close the loop on OL neto to obtain CL netc

[ netc Xci Aci ] = closeloop( neto, Xoi, Aoi );
[ Xc Xci Aci Tc ]= preparets( netc,{},{}, T );
[ Yc Xcf Acf ]   = netc( Xc, Xci ,Aci );
Ec               = gsubtract(Tc,Yc);
NMSEc            = mse(Ec)/mean(var(cell2mat(Tc}',1))

3. If NMSEc is not sufficiently small (<= 0.01, 0.05 ???) then train netc initialized with the final weights of neto

[ netc Yc Ec Xcf Acf ] = train( netc, Xc, Tc, Xci, Aci );
% [ Yc Xcf Acf ]         = netc( Xc, Xci ,Aci );
% Ec                     = gsubtract(Tc,Yc);
NMSEc                  = mse(Ec)/mean(var(cell2mat(Tc}',1))

4. Predict performance Np steps beyond the currently known data

[ Yc2 Xcf2 Acf2 ] = netc( cell(1,Np), Xcf ,Acf );

Hope this helps.

Thank you for formally accepting my answer

Greg

PS I'm sure this is covered in one of my previous posts in the NEWSGROUP or ANSWERS. Search using

 greg narnet closeloop

[pradeep kumar]

@ Greg Heath : Respected Sir , i got this error , in the 3rd line of your code . kindly help me .

neto = configure( neto, Xoi, Aoi);
Error using network/configure (line 116)
The number of target signals and network outputs do not match.

[pradeep kumar]

@ Greg Heath , Respected Sir , apart from the above i am getting the following error at the final step . point 4 .Predict performance Np steps beyond the currently known data .

[ Yc2 Xcf2 Acf2 ] = netc( cell(1,Np), Xcf ,Acf );

Error using network/sim (line 267)
Number of input states does not match net.numInputs.

Error in network/subsref (line 14)
     case 3, [v,out2,out3] = sim(vin,subs{:});

I want to forecast (predict ) the future value from the modeled Network . Please do me a last favour . Thank You.

Answer 2

[Greg Heath]

MSE00 = mean(var(cell2mat(T)',1)) % For T cell

MSE00 = mean(var(t',1)) % For t double, series are rows

nperfc is unsatisfactory.

Try training CL netc using the original data but initialized by the final weights of OL neto

If unsatisfactory, design another OL neto. Designing a CL netc from scratch will take too much time AND it will not be guaranteed to be a satisfactory design because the initial weights are random.

Hope this helps.

Greg

[pradeep kumar]

@ GReg : how to try training CL netc using the original data but initialized by the final weights of OL neto ? please help me . thankx