Dear friends, can I ask how to add an attention mechanism layer to TD3's critic network?

nObs=7；

% Define the attention layer

attentionSize=nObs;

attentionLayer = attentionLayer(attentionSize, 'AttentionLayer');

% Main path

mainPath = [

featureInputLayer(nObs,'Name',"StateInLyr")

attentionLayer

fullyConnectedLayer(128)

fullyConnectedLayer(128)

concatenationLayer(1,2,'Name',"concat")

reluLayer

fullyConnectedLayer(128)

reluLayer

fullyConnectedLayer(1,'Name',"CriticOutLyr")

];

% Action path

actionPath = [

featureInputLayer(nAct,'Name',"ActionInLyr")

fullyConnectedLayer(128,'Name',"fc2")

];

% Convert to layergraph object and connect layers

criticNet = layerGraph(mainPath);

criticNet = addLayers(criticNet, actionPath);

criticNet = connectLayers(criticNet,"fc2","concat/in2");

critic1 = rlQValueFunction(criticNet,obsInfo,actInfo,...

'ObservationInputNames',"StateInLyr",'ActionInputNames',"ActionInLyr");

critic2 = rlQValueFunction(criticNet,obsInfo,actInfo,...

'ObservationInputNames',"StateInLyr",'ActionInputNames',"ActionInLyr");

%%%%%%%%%%%%%%%%%%%%%%%%%

The code runs with the following error：

Error using string

Conversion from element failed. Element 1 must be convertible to a string scalar.

error: deep.internal.sdk.dag2dlnetwork_legacy>iCanApplyFix (line 221 )

layersWithIssues = string(analyzer.Issues.Layers);

error: deep.internal.sdk.dag2dlnetwork_legacy (line 69 )

canApplyFix = iCanApplyFix(analyzer);

error: rl.internal.model.createInternalModel (line 12 )

model = deep.internal.sdk.dag2dlnetwork_legacy(model);

error: rlQValueFunction (line 122 )

model = rl.internal.model.createInternalModel(model, nameValueArgs.UseDevice, inputSize, outputSize);

error: cfkz (line 114)

critic1 = rlQValueFunction(criticNet,obsInfo,actInfo,...

Answer 1

[Lokesh]

Hello SJ,

It seems that you are encountering an issue with your code due to an unconnected input to the 'attentionLayer'.

I faced a similar error when running the setup in MATLAB R2022b. You can observe this issue by using the following command:

analyzeNetwork(criticNet)

Using the 'analyzeNetwork' command can help you visualize the network structure and identify any disconnected layers or inputs, making it easier to debug and configure your network correctly.

To resolve this error, note that the 'attentionLayer' requires two inputs: 'hidden' and 'encoder' as per R2022b. Currently, only one input is connected, which is causing the problem. Ensure that both inputs are properly connected to the 'attentionLayer'.

Hope this helps!