MATLAB Answers

Using Matlab Neural Network Toolbox in real-time application

26 ビュー (過去 30 日間)
Enrico Anderlini
Enrico Anderlini 2016 年 3 月 11 日
コメント済み: Greg Heath 2016 年 3 月 16 日
I am trying to implement Neural Fitted Q iteration as described by Riedmiller (2005): http://ml.informatik.uni-freiburg.de/_media/publications/rieecml05.pdf This is a real-time application, where I update the weights of the neural network using the samples stored over a number of iterations (so, a small batch). I have tried to use the Neural Network Toolbox, but I have found out that I cannot update the weights of the neural network when I want to do it.
Trying to be a bit clearer, I use a function to initialize the neural network, which contains the following code:
net = feedforwardnet(5,'trainlm'); % say I use 5 hidden nodes to start with
net.trainParam.showWindow = false;
net.numInputs = 2; % say I have 2 inputs
net.inputs{1}.range = [-1 1;-1 1]; % any range in fact
% Network weight and bias initalization:
net = init(net);
Now, with the code above I have initialized the weight of the neural network, but if I try to run
y = net([x1;x2]);
the network would return an empty matrix. This is a problem for the first iteration of the loop in the Fitted Q-learning program, since I have to call this function before I have any actual data with which to update the neural network. So, a solution could be:
net = configure(net,[0;0],0); % configuring neural network to 2 inputs, 1 output
net = init(net);
or otherwise:
net = train(net,[7;0],1);
But by doing this, I then observe that the neural network will return 0 from now on indefinitely, i.e. I CANNOT update the weights of the neural network anymore despite training it again multiple times in the main function whenever I have a new batch of data (at first 20 items, then 40, then 60 until I stabilize with 100 values).
So, is there a way I can use the neural network in a pseudo-real-time application? Should I store the weights in external variables, then use them as input before the next training step? Any other solutions?
Thank you in advance for your help!

  0 件のコメント

サインイン to comment.

採用された回答

Greg Heath
Greg Heath 2016 年 3 月 12 日
net = init(net)
> Now, with the code above I have initialized the weight of the neural network,
No. All weights are either 0 or [] !
See (no semicolon)
wb = getwb(net)
IW = net.IW
b = net.b
LW = net.LW
Hope this helps.
Thank you for formally accepting my answer
Greg

  2 件のコメント

Enrico Anderlini
Enrico Anderlini 2016 年 3 月 14 日
Thank you, Greg. You are right. However, my problem is still there.
How can I initialize the weights of the neural network to small random values, and then update them in real time whenever I have a new batch of data?
Thanks!
Greg Heath
Greg Heath 2016 年 3 月 16 日
help configure
doc configure

サインイン to comment.

その他の回答 (0 件)

サインイン してこの質問に回答します。


Translated by