wordEmbeddingLayers() available in Deep Learning Toolbox?

Hello,

trying to run the "Deep Beer Designer", I got stuck on the use of wordEmbeddingLayer() which is flagged as an unknown function.

The reason is probably that I am still on Matlab and Deep Learning Toolbox (DLT) R2018b, but it is not quite clear if wordEmbeddingLayers should or should not be available in this release.

There are several Examples in the R2018b DLT Help which use it, but they do say that the Text Analytics Toolbox (TAT) has also been used (which is a little bit uncool, IMO :-)) .

The "List of Deep Learning Layers" in the R2019 DLT Help says "wordEmbeddingLayer (Text Analytics Toolbox™)"

(so why is it listed in the DLT at all?)

Moreover: the Release Notes for the R2018b DLT (both in the Help and online) under "Deep Learning: New network layers" state, somewhat confusingly, that

"You can now use the following layers in deep learning networks:

wordEmbeddingLayer

..."

but with no mention of the need for the TAT. Clicking the link of this layer then results in "Page not found"...

Was this layer type perhaps originally available in the DLT but later moved to the TAT?

Finally, if this wordEmbeddingLayer is really available only in the TAT, are there any "work-arounds" or replacements which would work in the R2018b DLT, without the need for the TAT?

Thanks in advance for all pointers.

Answer 1

[Steven Lord]

The wordEmbeddingLayer class is part of Text Analytics Toolbox and was introduced in release R2018b.

Perhaps that blog post should have mentioned that both Text Analytics Toolbox and Deep Learning Toolbox were required to run the examples it contains. IMO one of the purposes of some of the posts on the MathWorks blogs and some of the examples included as part of the documentation is to show how two or more of our products can work together to solve a problem that doesn't necessarily squarely fall into one or the other of those products' areas of focus.

There are several Examples in the R2018b DLT Help which use it, but they do say that the Text Analytics Toolbox (TAT) has also been used (which is a little bit uncool, IMO :-)) .

So should every layer suitable for use with Deep Learning Toolbox ship as part of that toolbox? If the wordEmbeddingLayer class uses functionality from Text Analytics Toolbox that is not directly related to the network architecture, should those functions also be included in Deep Learning Toolbox instead of or in addition to Text Analytics Toolbox? How about (looking at the list from release R2018b) the dozen or so layers that are part of Computer Vision System Toolbox? How much of that toolbox should be shipped as part of Deep Learning Toolbox in addition to being shipped as part of Computer Vision System Toolbox?

If you look at the list in the most recent release even more products are shipping layers that are compatible with the infrastructure in Deep Learning Toolbox: Lidar Toolbox, Computer Vision Toolbox, Text Analytics Toolbox, Reinforcement Learning Toolbox, Signal Processing Toolbox, Wavelet Toolbox, and Image Processing Toolbox.

The "List of Deep Learning Layers" in the R2019 DLT Help says "wordEmbeddingLayer (Text Analytics Toolbox™)"

(so why is it listed in the DLT at all?)

Because it is a layer that can be used with the infrastructure that is part of Deep Learning Toolbox to create deep learning models.

Finally, if this wordEmbeddingLayer is really available only in the TAT, are there any "work-arounds" or replacements which would work in the R2018b DLT, without the need for the TAT?

It is possible to define custom deep learning layers. I'm not sure how involved it would be to create a layer that's the equivalent of wordEmbeddingLayer.

[xitram]

Thanks for the replies.

> So should every layer suitable for use with Deep Learning Toolbox ship as part of that toolbox?

Of course not. But the comment was not about what layer is shipped with which toolbox, or that the wordEmbeddingLayer "ought to be" part of the Deep Learning Toolbox. And of course there is no problem with the DLT Help documentation referring to functions or layers from other toolboxes.

The point was that the expectation (mine anyway, rightly or wrongly ;-)) for the Examples in the Help of a given specific toolbox would rather be that they can be run within this toolbox without external dependencies. E.g. in the DLT Help chapter on "Deep Learning with Time Series, Sequences, and Text", section "Text Data", there are five examples, all interesting and instructive (as always in Matlab), but only one of which does not require the Text Analysis Toolbox, which provoked my remark. It can of course be argued that if I am that interested in text analysis, then I should procure the respective toolbox, too, but that is another discussion.

And obviously, a blog post or demo not linked to any specific toolbox is free to use whatever it requires.

> It is possible to define custom deep learning layers. I'm not sure how involved it would be to create a layer that's the equivalent of wordEmbeddingLayer.

Thanks for the tip, I'll look into it.

Answer 2

[Walter Roberson]

No, it has always been part of Text Analytics, https://www.mathworks.com/help/releases/R2018b/textanalytics/ref/nnet.cnn.layer.wordembeddinglayer.html since it was introduced in R2018b.