Problems in sharing mex codes with workers in a cluster

Hi fellows, I am playing with the matlab 2011a parallel computing (via MATLAB Distributed Computing Server) in a cluster consisting of three computer in my office. I successfully got a test problem running when there were no mex files. However, when there are mex files, matlab reports that the mex files can not be found. The reason for using mex files from C is that they are faster. Could anyone help me out? Many thanks.

Answer 1

[Edric Ellis]

You need to use

matlabpool('addfiledependencies', {'mex1', 'mex2'})

to make the MEX files available to the workers. See the matlabpool reference page for more information.

[Ding Liu]

Hi Edric, many thanks for your reply. I tried your suggestions and seemingly solved part of the problem. However, I still got the following error message:

>> matlabpool Starting matlabpool using the 'JobManagerTest3' configuration ... connected to 4 labs. >> matlabpool('addfiledependencies', {'chebbasx.mexw64', 'kron.mexw32'}) ---------------- RoundAppr 1, # gridpoints = [39 42]---------------- ??? Error using ==> parallel_function at 598 Error in ==> chebbas at 44 Invalid MEX-file 'C:\WINDOWS\TEMP\MDCE\Checkpoint\DELL-G8V0YX1_DELL-G8V0YX1_worker01_mlworker_log\matlab\filedependencies\4\chebbasx.mexw64': The specified module could not be found.

Error in ==> debt_time_iteration_main at 251 parfor iLoop = 1:numel(mGrid(:,1))

The structure of the shared folders are shown as in the attached figure. I put the mex files under the CompEcon_test folder. Could you please give me some more hints? Thanks.

[Edric Ellis]

Do your MEX files need additional DLLs? If so, you'll need to add those too. You can check using the "depends" utility.

[Ding Liu]

Hi Edric, thanks a lot for your professional advice. The feedback from the Dependency Walker suggests that some DLLs are missing, though I don't fully understand why. The CompEcon toolbox (<http://www4.ncsu.edu/~pfackler/compecon/toolbox.html>) can be run successfully on each computer (in the cluster) alone. I checked the files in the toolbox, and found no DLL files, though the Dependency Walker said otherwise. Please see the attached figure. How should I add these required DLLs? Thank you, Edric.

[Edric Ellis]

LIBMX.DLL and LIBMEX.DLL are part of the MATLAB installation - these will be present on the workers. One other possibility is that you might need to rebuild the MEX files for R2011a if they were built for a different version of MATLAB - as per this answer.

[Ding Liu]

Hi Edric, many thanks for your help. It works! Cheers.