Simulink: database toolbox connection fails in Parallel Computing parfor loop?

I am trying to use mutlithreaded database updates in Simulink using the update function inside parfor

My code is something like this:

parfor i = 1:numLoad

updateQuery = strcat("{""$set"":", jsonencode(document(i)), "}");

findQuery = strcat("{""name"": """, document(i).name, """}");

temp_conn = dbConnections{i};

update(temp_conn, "collection", findQuery, updateQuery);

end

where dBConnections is initialised like this:

for i = 1:5

dbConnections{i} = mongoc("localhost",27017,"database"); % Replace with your connection setup

end

However, this crashes.

It will not crash if I either (1) change parfor to for, or (2) remove the update function.

Is the mongoc connection just not usable in parfor?

The error messages are:

Warning: A worker aborted during execution of the parfor loop. The parfor loop will now run again on the remaining workers. > In distcomp/remoteparfor/handleIntervalErrorResult (line 246) In distcomp/remoteparfor/getCompleteIntervals (line 396) In parallel_function>distributed_execution (line 746) In parallel_function (line 578) In my_model>Outputs (line 118)

An error occurred while running the simulation and the simulation was terminated

Caused by:

• Error evaluating registered method 'Outputs' of MATLAB S-Function 'my_model' in 'my_model/Level-2 S-Function For MongoDB'. The following is the MATLAB call stack (file names and line numbers) that produced this error: ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [195] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [259] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [396] ['C:\Users\me\MATLAB\Projects\learning_simulink\work\my_model.m'] [118]
• All workers aborted during execution of the parfor loop

Any ideas for what the issue is?

Answer 1

[Raymond Norris]

To summarize:

• You're running a single Simulink model (my_model), in which you have a block (Level-2 S-Function for MongoDB) that's running MATLAB code
• In this MATLAB code, you have serial code that runs a for-loop to create an array of database connections (but really all the same db connection)
• And you then run a parfor-loop, such that each loop iteration updates a collection with a set of documents a Mongo DB

A couple of thoughts:

• I don't see the need to create an array of database connections, if they're all pointing to the same MongoDB. Just create one dbConnection.
• You might be better off having each worker create a db connection. You can try either of these options
• https://www.mathworks.com/help/parallel-computing/parallel.pool.constant.html
• https://www.mathworks.com/help/database/ug/createconnectionforpool.html

[Hua Kun Tan]

Hi, thank you, you have summarised it quite well.

The array of db connections was an attempt to figure out if the issue was caused by multiple threads using the same connection.

I attempted to use parallel.pool.Constant to solve my issue, however, the same errors persist.

conn = mongoc("localhost",27017,"database"); % ip, port, database name

constant_conn = parallel.pool.Constant(conn);

parfor i = 1:numLoad

updateQuery = strcat("{""$set"":", jsonencode(document(i)), "}");

findQuery = strcat("{""name"": """, document(i).name, """}");

update(constant_conn.Value, "collection", findQuery, updateQuery);

end

An error occurred while running the simulation and the simulation was terminated

Caused by:

• Error evaluating registered method 'Outputs' of MATLAB S-Function 'my_model' in 'my_model/Level-2 S-Function For MongoDB'. The following is the MATLAB call stack (file names and line numbers) that produced this error: ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [195] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [259] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [396] ['C:\Users\me\MATLAB\Projects\learning_simulink\work\my_model.m'] [122]
• All workers aborted during execution of the parfor loop.

The baffling thing is that replacing parfor with for runs perfectly.

Do you have any luck attempting to run mongodb updates in a parfor loop?

[Raymond Norris]

Change the creation, as such

constant_conn = parallel.pool.Constant(@() mongoc("localhost",27017,"database")); % ip, port, database name

This will create the connection on the workers, instead of creating it from the client and then passing it to the workers.