How to implement PyTorch's Linear layer in Matlab?
16 ビュー (過去 30 日間)
John Smith 2023 年 2 月 11 日
How can I implement PyTorch's Linear layer in Matlab?
The problem is that Linear does not flatten its inputs whereas Matlab's fullyConnectedLayer does, so the two are not equivalent.
回答 (4 件)
Matt J 2023 年 2 月 11 日
編集済み: Matt J 2023 年 2 月 11 日
One possibility might be to express the linear layer as a cascade of fullyConnectedLayer followed by a functionLayer. The functionLayer can reshape the flattened input back to the form you want,
layer = functionLayer(@(X)reshape(X,[h,w,c]));
Matt J 2023 年 2 月 13 日
編集済み: Matt J 2023 年 2 月 13 日
Another possible way to interpret your question is that you are trying to apply pagemtimes to the input X with a non-learnable matrix A, where the different channels of X are the pages. That can also be done with a functionLayer, as illustrated below both with normal arrays and with dlarrays,
A=rand(4,3); %non-learnable matrix A
xdata=rand(3,3,2); %input layer data with 2 channels
multLayer=functionLayer(@(X) dlarray( pagemtimes(A,stripdims(X)) ,dims(X)) );
%%Verify agreement with normal pagemtimes
Matt J 2023 年 2 月 14 日
編集済み: Matt J 2023 年 2 月 14 日
Another approach is to write your own custom layer for channel-wise matrix multiplication. I have attached a possible version of this,
L=pagemtimesLayer(4); %Custom layer - premultiplies channels by 4-row learnable matrix A
Ycheck=pagemtimes(L.A,X) %Check agreement with a direct call to pagemtimes()
Matt J 2023 年 2 月 15 日
That sounds right.
Although, part of me questions whether it was the best design for TMW to make the the user responsible for summing over batched input in the backward() method, since that dimension should always be handled the same way.
Find more on Image Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!