error when running ray to parallelise a simple matlab function wrapped as python application
1 回表示 (過去 30 日間)
古いコメントを表示
I would like to parallelise a matlab function using python and ray.
In this example
- I used MathWorks to wrap a very simple matlab function as Python application (the function just returns as result the input integer - see below)
- The application (the function) is then called in python (see below). The function is called using the ray package to parallelise the call.
- The function is called four times using as inputs some integers: 0, 1, 2 and 3. The result should be a list: [0, 1, 2, 3]
- The call fails with error: ray/cloudpickle/cloudpickle_fast.py, TypeError: 'str' object is not callable
- The same function (test_function_for_python_1) works well in a simple for loop and returns [0, 1, 2, 3], so the problem is not in the MathWorks wrapping of the matlab function into python.
In short, ray needs to pickle the matlab/python application, and the pickling encounters a problem I can't solve. Any help would be much appreciated.
The operating system is MacOSx Catalina, the python (and mwpython) version is 3.7.9, the ray python module version is 1.0.0
== Matlab function ==============================================
function result = test_function_for_python_2(an_integer_input)
result = an_integer_input;
end
== Python script that parallelises the matlab function ========================
import TestTwo # a class with a simple function returning the input integer
import matlab
import ray # for parallelisation
# Initialise ========================================================
ray.shutdown() # shutdown any open CPU pool
ray.init() # create a pool of parallel threads
# initialise the matlab app containing the code of a simple test function (returns the same integer that is input)
my_test_function = TestTwo.initialize()
print(type(my_test_function))
print(type(my_test_function.test_function_for_python_2))
@ray.remote
def function_two(an_integer):
'''Here a single string is input and returned.
The function responsible for it has been written in Matlab and wrapped as a python application in Mathworks.'''
result = my_test_function.test_function_for_python_2(an_integer, 1) #, nargout=1)
return result
some_integers = [0, 1, 2, 3]
# run function and collect results with ray for parallelisation
ray_objects = [function_two.remote(an_integer=i) for i in some_integers]
future_results = ray.get(ray_objects)
print('Results: {}'.format(future_results))
# Terminate ====================================
my_test_function.terminate() # close the matlab app containing the code for the test function
ray.shutdown() # shutdown any open CPU pool
== Error from mwpython console =====================================
% /Applications/MATLAB/MATLAB_Runtime/v99/bin/mwpython ~/Documents/TestTwo/for_redistribution_files_only/run_test_2_matlab_wrapped_in_python_parallel.py
2020-11-06 21:09:00,142 INFO services.py:1166 -- View the Ray dashboard at http://127.0.0.1:8265
/Applications/MATLAB/MATLAB_Runtime/v99/bin/maci64/mwpython3.7.app/Contents/MacOS/mwpython3.7: can't find '__main__' module in ''
<class 'matlab_pysdk.runtime.deployablepackage.DeployablePackage'>
<class 'matlab_pysdk.runtime.deployablefunc.DeployableFunc'>
Traceback (most recent call last):
File "/Users/me/Documents/TestTwo/for_redistribution_files_only/run_test_2_matlab_wrapped_in_python_parallel.py", line 32, in <module>
ray_objects = [function_two.remote(an_integer=i) for i in some_integers]
File "/Users/me/Documents/TestTwo/for_redistribution_files_only/run_test_2_matlab_wrapped_in_python_parallel.py", line 32, in <listcomp>
ray_objects = [function_two.remote(an_integer=i) for i in some_integers]
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/remote_function.py", line 99, in _remote_proxy
return self._remote(args=args, kwargs=kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/remote_function.py", line 207, in _remote
self._pickled_function = pickle.dumps(self._function)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 70, in dumps
cp.dump(obj)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 656, in dump
return Pickler.dump(self, obj)
TypeError: 'str' object is not callable
1 件のコメント
回答 (0 件)
参考
カテゴリ
Help Center および File Exchange で Call Python from MATLAB についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!