Running on AWS server does not benefit from more cores in parallel pool?
1 回表示 (過去 30 日間)
古いコメントを表示
I am using the built in Genetic Algorithm functionality in the Optimization Toolbox and time is a serious constraint when running the code.
The problem in question is an integer (binary) problem with linear constraints.
The solutions are on the order of a 400x1 vector of ones and zeroes, and the population size is 500. All other (relevant) Genetic Algorithm options are left as default.
The main bottleneck on run time is within the fitness function (it does not receive the population in vectorized form because this conflicts with using a parallel pool). It has to perform numerous operations on a 3D matrix that is roughly 300000x4x10. These operations are vectorized as much as possible to take advantage of Matlab's speed.
I start a parallel processing pool before calling the algorithm as follows:
delete(gcp('nocreate'));
c = parcluster('local'); % build the 'local' cluster object
numberOfCores = 6;
parpool(numberOfCores);
With 6 cores it takes roughly five minutes for each generation to complete (when running as a compiled DLL called by .NET). I have tried to set up an AWS server with 20 cores in hopes of speeding up the run time of each generation, but instead the run time per generation was roughly the same or slightly worse. (Matlab verified that the 20 core pool was properly created).
Does anyone know why that would be the case? Is there some way to estimate if there is a "sweet spot" for number of cores in a parallel processing pool compared to any potential overhead associated with parallelizing operations?
1 件のコメント
Run Zhu
2017 年 11 月 9 日
Maybe it is off the topic. But the performance one would obtain is typically: optimization model > optimization algorithm > parallel computing. As in your case, any particular reason to choose GA? Since it is usually slower than intlinprog and patternsearch, consider other solver would be more beneficial.
回答 (2 件)
Walter Roberson
2017 年 11 月 9 日
patternsearch is not "particularly susceptible to local minima": it is designed to have a reasonable chance of escaping local minima (but will certainly get stuck if they are deep enough.)
Since all of your inputs are integer (not a mix of integer and non-integer) you can use patternsearch using the techniques described at https://www.mathworks.com/matlabcentral/answers/285753-pattern-search-with-integer-decision-variable#answer_223391
0 件のコメント
参考
カテゴリ
Help Center および File Exchange で Surrogate Optimization についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!