Matlab activation function list

Hello,

where can I find an overview about functions to be used as activation functions in NN, e.g. radbas, tanh, sigmoid, tansig.... available in Matlab.

Thank you.

Answer 1

[Simon Chan]

Check this ink: MATLAB Functions

[Simon Chan]

And more specific: Deep Learning Toolbox - Functions

Answer 2

[CSCh]

Thank you Simon,

I used the model function example below. From your link I can see some activationlayerfunction but can't derive if the "standalone" function is available. For Example I can not replace the tanh (I used in the model function) with a swish function, because it does not exists in Matlab, even there is a swishlayer. And the otherway around, there are no Transig- or radbas-layer , but the functions exits, and I can use it instead of tanh.

So I'a looking for a complete list of "Standalone" function which I can try for the model function instead of tanh.

function dlU = model(parameters,dlX,dlT)

dlXT = [dlX;dlT];

numLayers = numel(fieldnames(parameters))/2;

% First fully connect operation.

weights = parameters.fc1_Weights;

bias = parameters.fc1_Bias;

dlU = fullyconnect(dlXT,weights,bias);

% tanh and fully connect operations for remaining layers.

for i=2:numLayers

name = "fc" + i;

dlU = tanh(dlU);

weights = parameters.(name + "_Weights");

bias = parameters.(name + "_Bias");

dlU = fullyconnect(dlU, weights, bias);

end

end

Answer 3

[CSCh]

Found here:

How to use a Leaky Relu/Softmax function in a hidden layer in a Feedforward Neural Network? - MATLAB Answers - MATLAB Central (mathworks.com)