Custom Neural Network with learnable weights in activation function

2 ビュー (過去 30 日間)
Ahmed Rehan
Ahmed Rehan 2022 年 9 月 16 日
回答済み: Kausthub 2023 年 9 月 7 日
Hi,
I want to create a Custom Neural Network with learnable parameters in the activation function. The learnable parameters are the weights of layers, but the weights are also used inside the activation function in a certain nonlinear relation. Normally, the activation function takes the inputs multiplied with the weights ( e,g, y1=f1(X*W1 + b1)) but here I want to use weights directly inside in a certain nonlinear relation i.e. y1=g1(X*W1 + b1,W1).
Custom layers can be created as metioned here:
But the way I want to create a custom layer with a specific activation function is what i need to know. Any help will be highly appreciated.
AR

回答 (1 件)

Kausthub
Kausthub 2023 年 9 月 7 日
Hi Ahmed Rehan,
I understand that you would like to have a custom activation function that has weights as an additional input i.e., instead of y1= f1(X*W1 + b1) you would like to have y1=g1(X*W1 + b1, W1).
You could leverage the “Define Custom Deep Learning Layer with Formatted Inputs” article which explains how to provide inputs to a custom layer.
I have attached a simple working snippet regarding the same for your reference. Major changes from the article you mentioned (https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layer.html) include:
  • Adding weights as a parameter in the layer definition
function layer = customActivationLayer(weights, args)
  • Passing weights when using the layer
customActivationLayer(Weights);
Please have a look at the attached code sample for better understanding.
Hope this helps and clarifies your queries regarding how to provide weights to activation functions using custom layers!

カテゴリ

Help Center および File ExchangeImage Data Workflows についてさらに検索

製品


リリース

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by