Modify layer output in custom neural network

I want to modify the output of one layer before sending it to the next layer. As an example you can think of output a2(k) in the custom neural network in https://se.mathworks.com/help/nnet/ug/create-and-train-custom-neural-network-architectures.html. Is it possible to apply some function on it before sending to layer 3. Or else is it possible to apply a new input p3(k) at layer 3 whose elements depend on the value of a2(k)?

Answer 1

[Shantanu Dixit]

Hi Ilias,

If I understood the query correctly, you're trying to apply a custom function between layers in a neural network. You can achieve this using MATLAB's 'functionLayer': https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.functionlayer.html which can be used to apply specific function to the layer input.

You can refer to the below example script which defines a simple CNN with an intermediate 'functionLayer' that applies the softsign operation "f(x) = x / (1 + |x|)".

% Network with a custom softsign activation
layers = [
    imageInputLayer([28 28 1], 'Name', 'input')
    convolution2dLayer(5, 20, 'Name', 'conv')
    %% 
    % custom function
    functionLayer(@(X) X ./ (1 + abs(X)), 'Name', 'softsign', 'Description', 'softsign')
    %%
    maxPooling2dLayer(2, 'Stride', 2, 'Name', 'maxpool')
    fullyConnectedLayer(10, 'Name', 'fc')
    softmaxLayer('Name', 'softmax')
];

net = dlnetwork(layers);
sampleInput = rand(28, 28, 1, 1);
dlX = dlarray(sampleInput, 'SSCB');

dlY = predict(net, dlX);
disp("Output of the network:");

Output of the network:

disp(extractdata(dlY));

    0.1185
    0.0762
    0.0980
    0.1176
    0.1115
    0.0760
    0.0982
    0.0896
    0.1035
    0.1108

Hope this helps!