Does DeepNetworkDesigner support Transformer Layers?

I want to add Transofmrer Layers while designing the network for deep learning using MATLAB DeepNetworkDesigner, but it doesn't show me such option. How can I add Transofmer Layers to the network designed using DeepNetworkDesigner?

Answer 1

[Matt J]

If the transformer layer is a custom layer that you have created for yourself, you would import it to deepNetworkDesigner from the workspace, as explained here,

https://www.mathworks.com/help/deeplearning/ref/deepnetworkdesigner-app.html

Answer 2

[David Ho]

Hi Subhadip,

In R2023b, the following layers are supported in Deep Network Designer that are commonly used in transformers:

• selfAttentionLayer
• wordEmbeddingLayer (requires Text Analytics Toolbox)
• sinusoidalPositionEncodingLayer
• positionEmbeddingLayer
• embeddingConcatenationLayer
• indexing1dLayer
• patchEmbeddingLayer (requires Computer Vision Toolbox)

For more information you can refer to the list of deep learning layers page and the links in it.

Best regards,

David