Deep learning softmax classification posterior probability calculated from validation/test set

Hi All

Im working on LSTM classification for a physical system in applied opics and using this example

https://www.mathworks.com/help/deeplearning/examples/classify-sequence-data-using-lstm-networks.html

However, for our particular system/data we always have slightly overfit on training data before accuracy on validation data is optimum (due to the nature of the data). Hence our probability given by the trained model always overestimates our certainty in prediction (on both validation and independent test set). We like to correct for this by recalculating the posterior probability. Can any advise on how it is possible to calculate confidence in prediction (or posterior probability) from a LSTM on the validation or test data instead of the training data that is given by the trained model?

The problem is that unfortunately the classify functions does not allow this

[YPred,scores] = classify(net,sequences) or

[updatedNet,YPred,scores] = classifyAndUpdateState(___)

as they outputs posterior probablity (scores) calculated from training data

Any advise it highly appreciated

Mads

[Image Analyst]

Original question, in case user edits it away:

Hi All

Im working on LSTM classification for a physical system in applied opics and using this example

https://www.mathworks.com/help/deeplearning/examples/classify-sequence-data-using-lstm-networks.html

However, for our particular system/data we always have slightly overfit on training data before accuracy on validation data is optimum (due to the nature of the data). Hence our probability given by the trained model always overestimates our certainty in prediction (on both validation and independent test set). We like to correct for this by recalculating the posterior probability. Can any advise on how it is possible to calculate confidence in prediction (or posterior probability) from a LSTM on the validation or test data instead of the training data that is given by the trained model?

The problem is that unfortunately the classify functions does not allow this

[YPred,scores] = classify(net,sequences) or
[updatedNet,YPred,scores] = classifyAndUpdateState(___)

as they outputs posterior probablity (scores) calculated from training data

Any advise it highly appreciated

Mads