fitnet Why test results are unbiased?

I've read form Greg Heath:

Data = Design + Non-design

Design = Train(estimate weights) + Validation(Stop training when MSEval goes thru a minimum).

Non-design = Test(Obtain UNBIASED generalization estimate of performance on unseen non-design data).

I like to know why and how test results are calculated on unbiased net despite the fact that training is made on biased network and how to obtain the same results using :

    output=net(input)

I must use :

    net.biasConnect = [0; 0];

after training and before using net for new outputs?

There is a contradiction between training net and the changing made after?

What do you mean when you say in:

http://www.mathworks.com/matlabcentral/answers/58761-nn-validation-and-data-partition

"The test set error is unbiased because it is completely independent of design (training and validation)." ?

http://it.mathworks.com/matlabcentral/answers/195077-good-test-error-and-wrong-relative-output

Answer 1

[Greg Heath]

This discussion has absolutely nothing to do with bias weights.

Theoretically, training, validation and test data are all assumed to be random samples from the same probability distribution function.

Both training and validation data are used to design the net with the goal of minimizing a performance function (such as mse (fitnet) or crossentropy (patternnet) for net performance on ANY data (seen and unseen) obtained from the same probability distribution.

Therefore, their performance cannot be used to obtain an unbiased (i.e., honest) estimate of performance on unseen data.

On the other hand, the test data is in no way involved in the design of the net. Therefore it is a valid representative of "unseen" data and it's performance is considered an unbiased estimate of net performance on unseen data.

Hope this helps.

Greg

PS Not sure why I never saw this post before.