Custom deep learning layer: function with learnables

A custom deep learning layer, similar to functionLayer() but with learnable parameters.

Summary

This submission defines learnableFcnLayer, a custom deep learning layer, compatible with the Deep Learning Toolbox, that evaluates a user-supplied function. It is analgous to the toolbox's native native functionLayer, except that it allows an array of learnable parameters to be attached to the function. The learnable parameters are stored in the layer property theta and are optimized during network training.

layer = learnableFcnLayer(fun,theta0)
layer = learnableFcnLayer(fun,theta0, Name=name)

The supplied function must have the form

Y = fun(theta,X1,X2,...,XN)

where:

theta - Learnable parameter array.

X1...XN - Network inputs.

Y - Layer output.

Example

Linear combination of two inputs:

fun = @(theta,X1,X2) ...
    theta(1)*X1 + theta(2)*X2 + theta(3);

layer = learnableFcnLayer( fun, [1;1;0], Name="blend");

This layer computes

Y = theta(1)*X1 + theta(2)*X2 + theta(3)

where theta is learned during training.