Neural network with softmax output function giving sum(output)~=1

Hi, I have an issue with training neural network with softmax output transferFcn. I have trained NN using code appended below:

      net = feedforwardnet(6,'trainscg');
        net.layers{2}.transferFcn='softmax';
        net.trainParam.epochs = 100;
        net.divideParam.trainRatio = 1.0;
        net.divideParam.testRatio = 0.0;
        net.divideParam.valRatio = 0.0;
        net.trainParam.lr = 0.1;
        net.trainParam.goal = 0.001;
        net.trainParam.showWindow=0;
       net = train(net,normX',outputLabels); %
where  normX = 26,000 X 7 and outputLabels = 2 X 26,000. Output can be 0 or 1.

when I simulate network with same input data, I am getting output for 2 output units in the range of [0,1] but their sum is not equal to one. I have read and have also searched on web that using softmax one can get sum(output activation) = 1.

Please help me with this issue. Let me know if you need further information.

Thanks Regards Anshul

Answer 1

[Greg Heath]

There appears to be a bug in MATLAB's softmax. Before MATLAB introduced their version I coded my own. I lost it when my computer crashed and do not remember if I ever replaced it. If interested, You can search in COMP.AI.NEURAL-NETS, COMP.SOFT-SYS.MATLAB and ANSWERS and GOOGLE using

 greg softmax

OR you can try to debug the current version.

I do not have time to chase it down. However, I do not see that you used the crossentropy performance function instead of mean-square-error.

[Anshul Gupta]

Thanks Greg. I tried searching for crossentropy for performance function but couldn't find it in Matlab. I will try searching your implementation of softmax.

Thanks Anshul

[Anshul Gupta]

Hey Greg,

I found the problem.

The implementation for softmax is correct but Matlab by default puts a processing function in the output layer (mapminmax), which was actually giving the problem. When I removed it, as my output was already between 0 and 1, I am getting output activation as desired.

Anyway Thanks for your help.

Regards

Anshul

[Antonio Mendes]

Anshul Gupta thanks for reporting what the problem is. I was experiencing the same problem today. Thank you.

Answer 2

[wang]

how to remove it?

Answer 3

[reza]

Hi,

You may use the followwing command line:

net.outputs{end}.processFcns={};