How to access gradients in Deep Learning Toolbox

Is it possible to get access to gradients in Deep Learning Toolbox at all layers (custom and built-in)?

Thx

Answer 1

[Snehal]

Hello @John Smith,

My understanding is that you want to access a model’s gradients at each component layer (custom as well as built-in layers).

You can achieve this by implementing a custom training loop with all these layers and then using ‘dlgradient’ function to compute the gradients of the loss with respect to all learnable parameters.

You can refer to the below given code snippet on the implementation of ‘dlgradient’ function:

% Assuming ‘loss’ is the computed loss from your model 
% ‘modelParameters’ is a dlarray containing all learnable parameters 
gradients = dlgradient(loss, modelParameters); 

The result ‘gradients’ is a structure or array containing gradients for each of the learnable parameters, which you may then access layer by layer.

Below are some documentation links for more details:

• https://www.mathworks.com/help/deeplearning/ref/dlarray.dlgradient.html
• https://www.mathworks.com/help/deeplearning/ug/train-network-using-custom-training-loop.html

Hope this helps.