Missing layers in Deep Learning Toolboox results in failing to run EfficientNetV2S

I saved EfficientNetV2S in Python as follows

import tensorflow as tf

model = tf.keras.applications.efficientnet_v2.EfficientNetV2S(

include_top =True,

weights =None,

input_tensor=None,

input_shape =None,

pooling =None,

classes =2,

classifier_activation='softmax',

include_preprocessing=True

)

model.save('EfficientNetV2S')

model.save('EfficientNetV2S_.h5')

Then I also converted the created above folder EfficientNetV2S to ONNX format as follows

python -m tf2onnx.convert --saved-model ./EfficientNetV2S --output EfficientNetV2S.onnx

Now I have three ways to import this model to Matlab

First:

net = importTensorFlowNetwork('EfficientNetV2S');

Warning: The SavedModel 'EfficientNetV2S' was saved in TensorFlow version '2.8.0'. Import of SavedModel versions newer than '2.6.0' is not supported. The imported model may not
exactly match the model saved in the SavedModel. 
Error using nnet.internal.cnn.tensorflow.savedmodel.TFSavedModel (line 35)
'OutputLayerType' is missing. Please specify an output layer type or set the 'TargetNetwork' to be 'dlnetwork'.

Error in nnet.internal.cnn.tensorflow.importTensorFlowNetwork (line 20)
    sm = savedmodel.TFSavedModel(path, options, true);

Error in importTensorFlowNetwork (line 107)
Network = nnet.internal.cnn.tensorflow.importTensorFlowNetwork(modelFolder, varargin{:});

"'OutputLayerType' is missing" not sure what I should do about this.

Second:

net = importKerasNetwork('EfficientNetV2S_.h5');

Warning: File 'EfficientNetV2S_.h5' was saved in Keras version '2.8.0'. Import of Keras versions newer than '2.2.4' is not supported. The imported model may not exactly match the
model saved in the Keras file. 
Error using nnet.internal.cnn.keras.importKerasNetwork (line 21)
Keras network does not include loss information specifying the output layer type. Specify the type using the 'OutputLayerType' argument.

Error in importKerasNetwork (line 76)
Network = nnet.internal.cnn.keras.importKerasNetwork(modelfile, varargin{:});

"Keras network does not include loss information specifying the output layer type. Specify the type using the 'OutputLayerType' argument." will it help? How should I do that?

Thrid:

net = importONNXNetwork('EfficientNetV2S.onnx')

This one doesn't generate the errors! But, then I attempt training the model, it returns NaNs. Based on the deepNetworkDesigner, I believe this is caused by the missing layers:

• ElementWiseAffineLayer
• IdentityLayer
• FlattenInto2DLayer
• ReshapeLayer

Am I doing something wrong? Any suggestions are welcome.

---------------------------------------------------------------------------------

UPDATE

---------------------------------------------------------------------------------

I've tried computing each layer separately one by one. The model fails on

activations(trainedNet,imgs,'StatefulPartiti_1327')

that returns NaN. I believe this due to missing implementation of nnet.onnx.layer.IdentityLayer

The list of the first layers is as follows

573×1 Layer array with layers:

1 'input_1' Image Input 384×384×3 images

2 'StatefulPartiti_1324' Elementwise Affine Applies an elementwise scaling followed by an addition to the input.

3 'StatefulPartiti_1323' Elementwise Affine Applies an elementwise scaling followed by an addition to the input.

4 'StatefulPartiti_1328' nnet.onnx.layer.IdentityLayer nnet.onnx.layer.IdentityLayer

5 'StatefulPartiti_1327' Convolution 24 3×3×3 convolutions with stride [2 2] and padding [0 1 0 1]

...

Answer 1

[Sivylla Paraskevopoulou]

It is recommended to use importTensorFlowNetwork over importKerasNetwork. For more details, see the Import and Export Networks (documentation) and Importing Models from TensorFlow, PyTorch, and ONNX (blog post).

So let's try your first option and specify the output layer type:

net = importTensorFlowNetwork("EfficientNetV2S_.h5",OutputLayerType="classification");

This should import your TensorFlow model as a DAGNetwork object. You can also specify the class names. EfficientNet is trained on the ImageNet data set. See how to get the class names in this example: https://github.com/matlab-deep-learning/Image-Classification-in-MATLAB-Using-Converted-TensorFlow-Model

After you define ClassNames, you can import your model with classes by running this code:

net = importTensorFlowNetwork("EfficientNetV2S_.h5",OutputLayerType="classification",Classes=ClassNames);

You can also choose to import your TensorFlow model as a dlnetwork object. A dlnetwork does not have an output layer.

net = importTensorFlowNetwork("EfficientNetV2S_.h5",TargetNetwork="dlnetwork");

[Artem Lenskiy]

Thank you very much for your answer! It resolved my problem.

To summarise, I had to add OutputLayerType="classification" as an argument i.e.:

net = importTensorFlowNetwork('EfficientNetV2S',OutputLayerType="classification")

[Arefin Shamsil]

Hi there,

The solution you've suggested looked promising. But when I applied it in my case, I am getting errors that I cannot resolve. I have trained a simple 1D-CNN network in Google Colab using the TensorFlow Keras API. While training, I saved the best model using ModelCheckPoint callbacks as following:

callbacks = [keras.callbacks.ModelCheckpoint("best_model.h5", save_best_only=True, monitor="val_loss"), keras.callbacks.ReduceLROnPlateau(monitor="val_loss", factor=0.5, patience=20, min_lr=0.0001), keras.callbacks.EarlyStopping(monitor="val_loss", patience=50, verbose=1),]

Then I have tried to import the .h5 model file to Matlab as following:

model = 'C:/Trained_Models/best_model.h5';

classNames = {'0','1'};

netModel = importTensorFlowNetwork(model, OutputLayerType = "classification", Classes=classNames);

And the followings are the errors I am getting:

Error using nnet.internal.cnn.tensorflow.importTensorFlowNetwork

Unable to find folder 'C:/Trained_Models/best_model.h5'. Import the Keras

network layers in the HDF5 or JSON format from the file 'C:/Trained_Models/best_model.h5' by using importKerasLayers.

Error in importTensorFlowNetwork (line 107)

Network = nnet.internal.cnn.tensorflow.importTensorFlowNetwork(modelFolder, varargin{:});

I followed the suggestion here exactly, but cannot figure out why I am getting this error. Any suggestion would be really appreciated.

[Artem Lensky]

Just a random suggestion that is worth trying: try converting your model to onnx model

python -m tf2onnx.convert --saved-model ./best_model --output best_model.onnx

and then import it as follows

net = importONNXNetwork('best_model.onnx')

[Sivylla Paraskevopoulou]

@Arefin Shamsil, you saved your TensorFlow model in .h5 format. You should save it in SavedModel format and then import it by using the importTensorFlowNetwork function. The importTensorFlowNetwork function expects a folder that contains these variables: saved_model.pb, variables, assets.

To save the model in SavedModel format in Python:

tf.saved_model.save(model,"best_model")

To import the model in MATLAB:

net = importTensorFlowNetwork("best_model");

[wu feng]

iit still no work