How to implement Bayesian Neural Network to get error bars in model predictions

Hello World,

I have written a customized neural network code. I am able to run it and was also able to do model predictions.

Now, I am looking for how can we implement Bayesian method in Neural Networks. By this I mean, how to give inputs to the network with its standard deviation so that when we check model's Regression plots after training, we would get error bars on each fitted data point in those Regression plots. What I know of Bayesian Networks is that it actually trains several models and with probabilistic weights making more robust way of getting best models. This makes more sense as claiming that only one single neural network model cannot be the best, so various committees of model will make us reach more generalized one.

I am attaching my code, Input data file and the output using Bayesian Neural Network. I got this Bayesian Neural Network output from the literature. I am expecting the same to get using powerful MATLAB provision.

Filename Data_3.xlsx is the Input file

Filename trial_1.m is the Customized Neural Network Code

Filename Predicted_vs_Target.PNG is the Bayesian output from the literature, and I am looking to get such same output through MATLAB.

I will be very much grateful and hope for getting solution.

Thank you

Answer 1

[Vineet Joshi]

Hi

Currently we are not supporting Bayesian Neural Networks. I have brought this issue to the notice of the concerned people and it might be considered for a future release.

Meanwhile, you can use an ensemble of Neural Networks to achive a similar result and get error bars in model predictions. Not exactly same to a BNN but will also results in a robust model which can give the error in predictions.

Hope this helps.

Regards

[Onkar Khadke]

Sure vineet, that helps. Will be looking forward to use Bayesian optimization in MATLAB.

Thank you are updating regarding Bayesian neural networks.

[Franck Tancret]

This was in 2021. What about now, in 2024? I see that BNN are implemented in the "Deep Learning Toolbox", and an example is given here:

https://fr.mathworks.com/help/deeplearning/ug/train-bayesian-neural-network.html

However, the example is on image regression, and it is complex to implement. Would there be an example of a "simple" regression (i.e. a one-column output vector as a function of a set of input variables), using for instance a NN with one hidden layer, with intructions on how to make predictions for new inputs and also get the corresponding predictive error (e.g. standard deviation of the distribution of the predicted output). In other words, something similar to the case of Gaussian process regression exposed here (using the "Statistics and Machine Learning Toolbox"):

https://fr.mathworks.com/help/stats/gaussian-process-regression-models.html

What I mean by "simple" is typically what is found in the GP example above: training of the model is one instruction ("fitrgp") and making predictions with error as well ("predict").