How do I import Hugging Face models into MATLAB?

Hugging Face (https://huggingface.co/) is a popular open-source repository of deep learning models and data for natural language processing (NLP). There are numerous pre-trained NLP models (e.g. BERT, RoBERTa, etc.), including their weights, available for use in downstream tasks through this repository. 

However, these Hugging Face models have typically been trained using PyTorch or TensorFlow, and there does not seem to be any MATLAB implementations of these models available. Is there an easy function or an example demonstrating how to import these models into MATLAB?

Answer 1

[MathWorks Support Team]

There is currently a GitHub repository (https://github.com/matlab-deep-learning/transformer-models) which implements some deep learning transformer models for NLP tasks in MATLAB. This repository includes popular models, such as BERT, FinBERT, and GPT-2. However, other models will need to be manually imported into MATLAB.

Unfortunately, there is no simple function or example detailing how to do this as of right now. However, this is a workflow our development team is very much interested in developing and is actively investigating.

In the meantime, the currently recommended workflow is to export the model and weights to ONNX format. Hugging Face provides a detailed explanation as to how this can be done in their documentation linked below.

https://huggingface.co/docs/transformers/serialization

Once you have the model and weights in ONNX format, you can use either the "importONNXNetwork" or the "importONNXFunction", depending on whether you want a layer graph or custom function (the documentation for both of these functions are linked below) to import the model into MATLAB.

https://www.mathworks.com/help/releases/R2021b/deeplearning/ref/importonnxnetwork.html

https://www.mathworks.com/help/releases/R2021b/deeplearning/ref/importonnxfunction.html

Unfortunately, due to the wide variety of transformers and self-attention layers found in Hugging Face models, there may be a number of custom layers that are not cleanly imported into the corresponding MATLAB layer. This may lead to custom layers in the imported network/function which contain multiple single layers. It may then be necessary to break these conglomerated layers into their individual parts manually. 

Furthermore, when using "importONNXFunction", it is likely that some of the layers will be replaced by placeholder layers. You will need to manually replace these placeholder layers with their correct layers in order to use this imported model for downstream tasks.

[xingxingcui]

The function importONNXNetwork is no longer the recommended choice and has been deprecated.