Reinforcement Learning Error with two scalar inputs
1 回表示 (過去 30 日間)
古いコメントを表示
I have a strange error from a critic network that has 3 inputs, image, and two scalars. But I see the following error:
Error using rl.internal.validate.mapFunctionObservationInput
Unable to automatically specify deep neural network observation input layer names because some specifications have similar dimension. Specify "ObservationInputNames" name-value pair when
creating function object.
Error in rlContinuousGaussianActor (line 95)
modelInputMap = rl.internal.validate.mapFunctionObservationInput(model,observationInfo,nameValueArgs.ObservationInputNames);
Do you have any idea? It is really long time I am working on this error. My network is as follows:
type=3;
w1=2;
w2=1;
% obsMat = [1 1];
obsMat = [4 3; 5 3; 6 3; 7 3; 8 3; 9 3; 5 11; 6 11; 7 11; 8 11; 6 12; 7 12; 10 12; ];
sA0 = [2 5];
sB0 = [11 5];
sC0 = [3 2];
sD0 = [6 5];
sE0 = [12 5];
sF0 = [6 12];
sG0 = [11 5];
sH0 = [3 11];
sI0 = [6 3];
sJ0 = [1 11];
s0 = [sA0; sB0; sC0];
% s0 = [sA0; sB0; sC0; sD0];
% s0 = [sA0; sB0; sC0];
Ts = 0.1;
Tf = 100;
maxsteps = ceil(Tf/Ts);
mdl = "rlAreaCoverage32024";
open_system(mdl)
% Define observation specifications.
scalarObs1Info = rlNumericSpec([1 1]);
scalarObs1Info.Name ="scalarObservation1";
scalarObs2Info = rlNumericSpec([1 1]);
scalarObs2Info.Name ="scalarObservation2";
obsSize = [12 12 4];
oinfo = rlNumericSpec(obsSize);
oinfo.Name ="image";
% oinfo.Name = "observations";
allObsInfo = [ oinfo, scalarObs1Info, scalarObs2Info];%, scalarObs3Info];
actionSpace = {1,2,3,4};
ActionInfo = rlNumericSpec([1, 2], 'Lowerlimit', -1, 'Upperlimit', 1); ainfo = ActionInfo;
ainfo.Name = "actions";
actInfo.UpperLimit=1;
actInfo.Lowerlimit=-1;
blks = mdl + ["/Agent A (Red)","/Agent B (Green)","/Agent C (Blue)"];
env = rlSimulinkEnv(mdl,blks,{allObsInfo,allObsInfo,allObsInfo},{ainfo,ainfo,ainfo});
env.ResetFcn = @(in) resetMap(in, obsMat);
rng(0)
for idx = 1:type
lgraph = layerGraph();
tempLayers = [
featureInputLayer(w1,"Name","scalarInput1")
reluLayer("Name","relu_3")
fullyConnectedLayer(1,"Name","fc_4")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
featureInputLayer(w3,"Name","scalarInput3")
reluLayer("Name","relu_1")
fullyConnectedLayer(1,"Name","fc_2")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
imageInputLayer(obsSize,Normalization="none")
convolution2dLayer(8,16, ...
Stride=1,Padding=1,WeightsInitializer="he")
reluLayer
convolution2dLayer(4,8, ...
Stride=1,Padding="same",WeightsInitializer="he")
reluLayer
fullyConnectedLayer(256,WeightsInitializer="he")
reluLayer
fullyConnectedLayer(128,WeightsInitializer="he")
% Hidden units (default = 128) are the number of units in the hidden layer of the neural network. Its size depends on the complexity of the problem, and should be set larger when there is a complex relationship between agent actions and observed variables.
reluLayer
fullyConnectedLayer(1,"Name","fc_1")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
concatenationLayer(1,3,"Name","concat")
softmaxLayer("Name","softmax")];
lgraph = addLayers(lgraph,tempLayers);
% clean up helper variable
clear tempLayers;
lgraph = connectLayers(lgraph,"fc_2","concat/in3");
% lgraph = connectLayers(lgraph,"fc_3","concat/in2");
lgraph = connectLayers(lgraph,"fc_1","concat/in4");
lgraph = connectLayers(lgraph,"fc_4","concat/in1");
plot(lgraph);
actorNetwork=lgraph;
% dlnetwork(actorNetwork);
actorOptions = rlOptimizerOptions('LearnRate',0.1,'GradientThreshold',inf);
actor(idx) = rlContinuousGaussianActor(actorNetwork,allObsInfo,ainfo);
1 件のコメント
Emmanouil Tzorakoleftherakis
2024 年 2 月 12 日
The code you sent cannot be executed as a standalone. Also, I am getting different errors that what you sent. Please modify the code to reproduce the error you mentioned above.
回答 (0 件)
参考
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!