Issue with Python interface to TensorFlow and other third-party libraries in Mac and Linux

When I try to run TensorFlow (2.3) commands from MATLAB in out-of-process execution mode with a Linux or Mac OS, the process gets terminated.

Using TensorFlow 2.2, 2.1, or 2.0, I see a different error ("Unable to resolve the name ____") but the process is not terminated. Instead, the following is displayed at the terminal where MATLAB was launched:

[libprotobuf FATAL] This program requires version 3.8.0 of the Protocol Buffer runtime library, but the
installed version is 3.6.1. Please update your library. If you compiled the program yourself, make
sure that your headers are from the same version of Protocol Buffers as your link-time library.
(Version verification failed in "bazel-out/k8-py2-
opt/bin/tensorflow/core/framework/tensor_shape.pb.cc".)

Answer 1

[MathWorks Support Team]

Python packages, like TensorFlow, often have dependencies on a number of third party libraries with specific version requirements. These may cause issues when MATLAB ships with the same libraries, but with different version requirements. Some of these conflicts may be avoided by running Python Interface in a separate process, but some conflicts may persist. One such conflict often occurs with the Protocol Buffer runtime library ('protobuf'), which is used by both TensorFlow and MATLAB, but with different versions. Other common examples include 'libexpat' and 'libcrypto'.

One workaround for this issue is to force TensorFlow to load the symbols from its own version of 'protobuf' instead of the version shipped with MATLAB. Taking the example of TensorFlow and 'protobuf', the code below demonstrates how to set the 'dlopen' flags and force TensorFlow to load its own version of 'protobuf'.

>> pyenv('Version', '<path to Python exectuable>', 'ExecutionMode', 'OutOfProcess');
>> RTLD_NOW=2;
>> RTLD_DEEPBIND=8;
>> flag=bitor(RTLD_NOW, RTLD_DEEPBIND);
>> py.sys.setdlopenflags(int32(flag));
>> c = py.tensorflow.constant(1);

The above example can be adapted for other packages where similar third-party library conflicts may be occurring.