tall array and data store problem
2 ビュー (過去 30 日間)
古いコメントを表示
I have problem with runnig an exmple from Matlab for tall arrays.
My question: I want to save a big cell array (12GB,20*35*5). I wanted to make a datastore (first tall array) to work efficiently in future but I am getting an error.
Do you have a suggestion to save this big array properly? I would appreciate if there is a script.
t = tall(rand(500,1))
location = 'C:\ExampleData';
write(location,t);
Error I get:>>>>
Writing tall data to folder C:\ExampleData
Evaluating tall expression using the Parallel Pool 'local':
Evaluation 8% complete
Error using parallel.internal.bigdata.MatArrayShuffleSortStrategy/createReduceInputSource (line 83)
Internal problem while evaluating tall expression. The problem was:
Error: File: C:\Program Files\MATLAB-R2020a\toolbox\matlab\datastoreio\+matlab\+io\+datastore\@TallDatastore\TallDatastore.m Line: 501 Column: 24
The import statement 'import matlab.io.internal.cellArrayDisp' cannot be found or cannot be imported. Imported names must end with '.*' or be fully qualified.
Error in parallel.internal.bigdata.ParallelPoolExecutor>@(partition)shuffleSortStrategy.createReduceInputSource(partition.PartitionIndex) (line 233)
readerFactory = @(partition) shuffleSortStrategy.createReduceInputSource(partition.PartitionIndex);
Error in matlab.bigdata.internal.executor.convertToIndependentTasks>@(partition)LocalReadProcessor(readerFactory(partition),numOutputs) (line 337)
readerExecution = CompositeDataProcessorBuilder([], @(partition) LocalReadProcessor(readerFactory(partition), numOutputs));
Error in ...
Error in tall/write (line 248)
iDoWrite(location, ta, writeFunction);
Caused by:
Error using matlab.io.datastore.TallDatastore
Error: File: C:\Program Files\MATLAB-R2020a\toolbox\matlab\datastoreio\+matlab\+io\+datastore\@TallDatastore\TallDatastore.m Line: 501 Column: 24
The import statement 'import matlab.io.internal.cellArrayDisp' cannot be found or cannot be imported. Imported names must end with '.*' or be fully
qualified.
0 件のコメント
回答 (1 件)
ag
2023 年 9 月 27 日
Hi Boby,
I understand that you are facing problem while using the "tall" function in MATLAB.
As I haven't been able to reproduce this issue, and the code works fine for me, I think this might be caused because of “shadowing”.
This problem arises if you have third-party toolboxes on your MATLAB path and they are earlier on the path than the built-in folders.
To resolve this, you can use the below code:
restoredefaultpath
rehash toolboxcache
savepath
The "savepath" command will remove the the third-party tools from the path. If you still use them, do not use the “savepath” command, instead try moving the third-party tools to the end of the path (“pathtool” is very useful for this purpose).
For more details on “pathtool” please refer to the following documentation:
I hope this helps!
Best Regards,
Aryan Gupta.
0 件のコメント
参考
カテゴリ
Help Center および File Exchange で File Operations についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!