Neural Network Fitnet Output Is Not Realistic

I'm attempting to run an ANN with the goal of determining which of 34 climate variables contributes the least error to the target data (daily precipitation). In other words, which of the 34 input variables in the *best predictor variables. Some of my issues have been resolved by reading the boards. However, I'm having an issue with the output which is causing poor performance. My target data has a lower bound of 0 (can't have less than 0 precipitation). Yet, no matter how I normalize the data [0,1] or [-1,1] and the output transfer function used [e.g. logsig, softmax), I get negative values out of the NN. When testing the model, it performs poorly and thus I'm not getting an accurate representation of the best predictor variable. I'm assuming it's a simple fix, but can't seem to get a handle on it after multiple days of trying. Thanks in advance for any help.

 %%%CODE IS BELOW %%%
%Solve an Input-Output Fitting problem with a Neural Network
% Script generated by Neural Fitting app
% Created Tue Sep 30 15:17:44 EDT 2014
%
% This script assumes these variables are defined:
%
%   all_ecmwf - input data.
%   precip_shaped - target data.
%
%
% x = all_ecmwf;
% t = precip_shaped;
%
% [ I N ] = size( x ) % [ 34 8353 ]
% [ O N ] = size( t ) % [ 1  8353 ]
%%Analyze Outliers in target data
% mu = mean(t)
% sigma = std(t)
% MeanMat = repmat(mu,O,1);
% SigmaMat = repmat(sigma,O,1);
% outliers = abs(t - MeanMat) > 3*SigmaMat;
%
%%Replace Outliers with mean of t
% t(outliers) = MeanMat;
%
%%Normalize Input, Target Data
% [x1, xs1] = mapminmax(x,-1,1);
% [t1, ts1] = mapminmax(t,0,1);
%
% Choose a Training Function
%    trainFcn = 'trainbr';  % Bayesian Regularization
%
% %Create a Fitting Network
%    hiddenLayerSize = 15;
%    net = fitnet(hiddenLayerSize,trainFcn);
%    net.layers{1}.transferFcn = 'tansig';
%    net.layers{2}.transferFcn = 'logsig';
%
% %Setup Division of Data for Training, Validation, Testing
%    net.divideFcn = 'dividerand';  % Divide data randomly
%    net.divideMode = 'sample';  % Divide up every sample
%    net.divideParam.trainRatio = 70/100;
%    net.divideParam.valRatio = 15/100;
%    net.divideParam.testRatio = 15/100;
%
%%Choose a Performance Function
%    net.performFcn = 'mse';  % Mean squared error
%
%%Choose Plot Functions
%    net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
%  'plotregression', 'plotfit'};
%
%%Train the Network
%    [net,tr] = train(net,x1,t);
%
%%Test the Network
%    y = net(x1);
%    e = gsubtract(t1,y);
%    performance = perform(net,t1,y);
%    
%%Reverse Transform target, input data
%realt = mapminmax('reverse',y,ts1);
%realx = mapminmax('reverse',y,xs1);
%
%%Recalculate Training, Validation and Test Performance
%    trainTargets = t1 .* tr.trainMask{1};
%    valTargets = t1  .* tr.valMask{1};
%    testTargets = t1  .* tr.testMask{1};
%    trainPerformance = perform(net,trainTargets,y)
%    valPerformance = perform(net,valTargets,y)
%    testPerformance = perform(net,testTargets,y)
%
%%Create new arrays for further testing of variables 
% perf=zeros(34,1);
% error=zeros(34,1);
% e=zeros(34,8353);
% testtarg=zeros(34,8353);
%
%%Determine MSE contribution of each variable
% for i = 1:size(x,1)
%    x1(i,:) = repmat((mean(x1(i,:),2)),1,1);
%    test=sim(net,x1);
%    e=(t-test);
%    err=mse(test);
%    perf(i)=mse(e);
%    error(i)=err;
%    ei(i,:)=e;
%    % Look at output values
%    realvar = mapminmax('reverse',test,ts1);
%    testtarg(i,:)=realvar;
%end

Answer 1

[Greg Heath]

 % x = all_ecmwf;
% t = precip_shaped;
%
% [ I N ] = size( x ) % [ 34 8353 ]
% [ O N ] = size( t ) % [ 1  8353 ]
%%Analyze Outliers in target data
% mu = mean(t)
% sigma = std(t)
% MeanMat = repmat(mu,O,1);
% SigmaMat = repmat(sigma,O,1);
% outliers = abs(t - MeanMat) > 3*SigmaMat;

1. Incorrect. Doesn't yield indices.

%% Replace Outliers with mean of t % t(outliers) = MeanMat;

2. Incorrect: t(outliers)= MeanMat + or - 3*SigMat

3. Similarly for 34-dimensional x

 %%Normalize Input, Target Data
% [x1, xs1] = mapminmax(x,-1,1);
% [t1, ts1] = mapminmax(t,0,1);

4. Delete mapminmax(x,-1,1) is a default

 % Choose a Training Function
%    trainFcn = 'trainbr';  % Bayesian Regularization
%
% %Create a Fitting Network
%    hiddenLayerSize = 15;
%    net = fitnet(hiddenLayerSize,trainFcn);
%    net.layers{1}.transferFcn = 'tansig';

5. Delete. 'tansig' is a default

 %    net.layers{2}.transferFcn = 'logsig';

6. net.outputs{2}.processFcns = {'removeconstantrows'}; % No default mapminmax

 % %Setup Division of Data for Training, Validation, Testing
%    net.divideFcn = 'dividerand';  % Divide data randomly
%    net.divideMode = 'sample';  % Divide up every sample
%    net.divideParam.trainRatio = 70/100;
%    net.divideParam.valRatio = 15/100;
%    net.divideParam.testRatio = 15/100;
%
%%Choose a Performance Function
%    net.performFcn = 'mse';  % Mean squared error
%
%%Choose Plot Functions
%    net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
%  'plotregression', 'plotfit'};

7. Delete last 7 statements (defaults)

 %%Train the Network
%    [net,tr] = train(net,x1,t);

8. [net,tr] = train(net,x1,t1);

 %%Test the Network
%    y = net(x1);
%    e = gsubtract(t1,y);
%    performance = perform(net,t1,y);

9-10 Replace y with y1.

 %%Reverse Transform target, input data
%realt = mapminmax('reverse',y,ts1);
%realx = mapminmax('reverse',y,xs1);

11. y = mapminmax('reverse',y1,ts1);

12. delete realx statement

 %%Recalculate Training, Validation and Test Performance
%    trainTargets = t1 .* tr.trainMask{1};
%    valTargets = t1  .* tr.valMask{1};
%    testTargets = t1  .* tr.testMask{1};

13:15 replace t1 with t

 %    trainPerformance = perform(net,trainTargets,y)
%    valPerformance = perform(net,valTargets,y)
%    testPerformance = perform(net,testTargets,y)
%
%%Create new arrays for further testing of variables 
% perf=zeros(34,1);
% error=zeros(34,1);
% e=zeros(34,8353);
% testtarg=zeros(34,8353);
%
%%Determine MSE contribution of each variable
% for i = 1:size(x,1)
%    x1(i,:) = repmat((mean(x1(i,:),2)),1,1);

16. delete above

17. x2=x1;

18. x2(i,:) = repmat((mean(x1(i,:),2)),1,I); %I = 8353

 %    test=sim(net,x1);

19. test=sim(net,x2);

 %    e=(t-test);
%    err=mse(test);

20. delete above

 %    perf(i)=mse(e);

21. Delete below. perf can be used to rank inputs.

 %    error(i)=err;
%    ei(i,:)=e;
%    % Look at output values
%    realvar = mapminmax('reverse',test,ts1);
%    testtarg(i,:)=realvar;
%end

[Craig]

Greg,

Thank you for the help. I will check and see if any of these errors still exist in my code. Do you have any advice on improving model performance for target data that are significantly skewed? My target dataset (precipitation) has a large quantity of zeros and small magnitude values with very few target data points of large magnitude. It seems that the model is having difficulty in handling the large quanitity of small values. Thanks again for the advice.

[Greg Heath]

Have you tried a log transform?