Math used for BiLSTMLayer in predict function

I am trying to convert a trained Bi-LSTM network to use on another platform, and wish to know the exact math used for the BiLSTM layer in Matlab predict function, just like the equations used in the LSTM layer Architecture section on this page.

The reason I am asking is that, the output given by the "predict" function is different from the output I calculate using my own code.

So I am trying to figure out the problem.

(pretty sure that I got the parameters like the input weights, recurrectweight, and bias correctly.)

The way I was calculating is as follows:

% forward
ft = sigmoid(wf*xt + uf*ht +bf);
it = sigmoid(wi*xt + ui*ht +bi);
ot = sigmoid(wo*xt + uo*ht +bo);
ctb = tanh(wc*xt + uc*ht +bc);
ct = ft.*ct + it.*ctb;
ht = ot.*tanh(ct);

% backward
xtb = flip(xt);
ftb = sigmoid(wfb*xtb + ufb*htb +bfb);
itb = sigmoid(wib*xtb + uib*htb +bib);
otb = sigmoid(wob*xtb + uob*htb +bob);
ctbb = tanh(wcb*xtb + ucb*htb +bcb);
ctb = ftb.*ctb + itb.*ctbb;
htb = otb.*tanh(ctb);

% output hidden states
Ht = [ht;htb];

Can anyone help? Thank you very much!

Answer 1

[David Willingham]

Hi James,

One option for exporting is to export the network to onnx format using exportONNXnetwork. Once exported you could either import the onnx version into the other framework as well as look at how the network wax exported.

[James He]

Hello David, thank you very much for your suggestions.

I did try the ONNX exporting function, however, the ONNX documentation does not have much description about the bidirectional LSTM case. The page I read is this.

I did tried the equations they suggested on that page (do not flip the input), but the output is still different from the predict function results.

As I am trying to implement my network to a relatively old platform, I do need to specify how the Matlab predict function got the value in math.

May I know if you knows about that? Thank you.

[David Willingham]

I checked with our developers and what you have looks correct. Without having both your full MATLAB code and python? code it may be difficult for us to find the reason why they maybe different.

[James He]

Hi David, I really appreciate your assistance.

Maybe I will have to look into other options.

Thanks.