Reinforcement Learning Error with two scalar inputs

1 回表示 (過去 30 日間)
ali farid
ali farid 2024 年 1 月 31 日
I have a strange error from a critic network that has 3 inputs, image, and two scalars. But I see the following error:
Error using rl.internal.validate.mapFunctionObservationInput
Unable to automatically specify deep neural network observation input layer names because some specifications have similar dimension. Specify "ObservationInputNames" name-value pair when
creating function object.
Error in rlContinuousGaussianActor (line 95)
modelInputMap = rl.internal.validate.mapFunctionObservationInput(model,observationInfo,nameValueArgs.ObservationInputNames);
Do you have any idea? It is really long time I am working on this error. My network is as follows:
type=3;
w1=2;
w2=1;
% obsMat = [1 1];
obsMat = [4 3; 5 3; 6 3; 7 3; 8 3; 9 3; 5 11; 6 11; 7 11; 8 11; 6 12; 7 12; 10 12; ];
sA0 = [2 5];
sB0 = [11 5];
sC0 = [3 2];
sD0 = [6 5];
sE0 = [12 5];
sF0 = [6 12];
sG0 = [11 5];
sH0 = [3 11];
sI0 = [6 3];
sJ0 = [1 11];
s0 = [sA0; sB0; sC0];
% s0 = [sA0; sB0; sC0; sD0];
% s0 = [sA0; sB0; sC0];
Ts = 0.1;
Tf = 100;
maxsteps = ceil(Tf/Ts);
mdl = "rlAreaCoverage32024";
open_system(mdl)
% Define observation specifications.
scalarObs1Info = rlNumericSpec([1 1]);
scalarObs1Info.Name ="scalarObservation1";
scalarObs2Info = rlNumericSpec([1 1]);
scalarObs2Info.Name ="scalarObservation2";
obsSize = [12 12 4];
oinfo = rlNumericSpec(obsSize);
oinfo.Name ="image";
% oinfo.Name = "observations";
allObsInfo = [ oinfo, scalarObs1Info, scalarObs2Info];%, scalarObs3Info];
actionSpace = {1,2,3,4};
ActionInfo = rlNumericSpec([1, 2], 'Lowerlimit', -1, 'Upperlimit', 1); ainfo = ActionInfo;
ainfo.Name = "actions";
actInfo.UpperLimit=1;
actInfo.Lowerlimit=-1;
blks = mdl + ["/Agent A (Red)","/Agent B (Green)","/Agent C (Blue)"];
env = rlSimulinkEnv(mdl,blks,{allObsInfo,allObsInfo,allObsInfo},{ainfo,ainfo,ainfo});
env.ResetFcn = @(in) resetMap(in, obsMat);
rng(0)
for idx = 1:type
lgraph = layerGraph();
tempLayers = [
featureInputLayer(w1,"Name","scalarInput1")
reluLayer("Name","relu_3")
fullyConnectedLayer(1,"Name","fc_4")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
featureInputLayer(w3,"Name","scalarInput3")
reluLayer("Name","relu_1")
fullyConnectedLayer(1,"Name","fc_2")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
imageInputLayer(obsSize,Normalization="none")
convolution2dLayer(8,16, ...
Stride=1,Padding=1,WeightsInitializer="he")
reluLayer
convolution2dLayer(4,8, ...
Stride=1,Padding="same",WeightsInitializer="he")
reluLayer
fullyConnectedLayer(256,WeightsInitializer="he")
reluLayer
fullyConnectedLayer(128,WeightsInitializer="he")
% Hidden units (default = 128) are the number of units in the hidden layer of the neural network. Its size depends on the complexity of the problem, and should be set larger when there is a complex relationship between agent actions and observed variables.
reluLayer
fullyConnectedLayer(1,"Name","fc_1")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
concatenationLayer(1,3,"Name","concat")
softmaxLayer("Name","softmax")];
lgraph = addLayers(lgraph,tempLayers);
% clean up helper variable
clear tempLayers;
lgraph = connectLayers(lgraph,"fc_2","concat/in3");
% lgraph = connectLayers(lgraph,"fc_3","concat/in2");
lgraph = connectLayers(lgraph,"fc_1","concat/in4");
lgraph = connectLayers(lgraph,"fc_4","concat/in1");
plot(lgraph);
actorNetwork=lgraph;
% dlnetwork(actorNetwork);
actorOptions = rlOptimizerOptions('LearnRate',0.1,'GradientThreshold',inf);
actor(idx) = rlContinuousGaussianActor(actorNetwork,allObsInfo,ainfo);
  1 件のコメント
Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2024 年 2 月 12 日
The code you sent cannot be executed as a standalone. Also, I am getting different errors that what you sent. Please modify the code to reproduce the error you mentioned above.

サインインしてコメントする。

回答 (0 件)

カテゴリ

Help Center および File ExchangeReinforcement Learning についてさらに検索

製品


リリース

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by