GPU out of memory

I used R2017b , R2108b , R2019b, and R2020b the same problem .

When running the training option code to calculate the accuracy of the images and the appearance of the graph to calculate it, this code will stop working, the mentioned problem is (GPU out of memory . Try reducing 'MinibatchAize' using the trainingOptions function) Is there a difference if RAM = 4GB or must be 8GB to run the code.

Although the code running in another laptop, But it doesn't run on my laptop

[Joss Knight]

Perhaps tell us what model you're training, and what your trainingOptions are, and the output of the gpuDevice function, and we can advise.

Answer 1

[Harsh Parikh]

Hello Shumukh,

Out-of-memory error occurs when MATLAB asks CUDA(or the GPU Device) to allocate memory and it returns an error due to insufficient space. For a big enough model, the issue will occur across differnet releases since the issue is with the GPU hardware.

As suggested, you can try reducing 'MiniBathSize' or other Min-Batch Options mentioned here.

If you are using CNN, you can refer this and this links for troubleshooting steps.

I am sharing some advanced level troubleshooting steps below as well:

You can also allocate a certain number of GPU resources to MATLAB exclusively.

• Depending on the cluster setup you can control access to resources through things like cgroups on Linux, or generic resource management in schedulers like Slurm (https://slurm.schedmd.com/gres.html). In this situation jobs submitted to the cluster request the resources, they need e.g. that you want access to a GPU and the scheduler when assigning a machine would take that into account and apply access permissions to the job so that it has specific access to the resource requested. Your cluster administrator may be able to help you more with how this is set up on your own cluster.
• Alternatively, if you are working on a single machine and there is no scheduling software involved. You can switch NVIDIA devices to 'exclusive-mode' in nvidia-smi to force that only 1 compute application at a time can use the GPU resources. This will require administrator or sudo privileges on the machine to change that setting.
• For more information on this, you can refer to the manual-page of nvidia-smi
• Please try these steps with the help/guidance of your machine administrator.

[Tong Zhao]

Hi Harsh, could you suggest how to partition the large input data sent to the GPU or cluster? Does MATLAB GPU Coder have similar functions to OpenACC / MPI directives for coordinating different PEs/workers to exchange data and coordinate work? Thanks! BTW this is my problem post regarding GPU Coder running into out of memory problem: https://www.mathworks.com/matlabcentral/answers/855805-gpu-coder-used-but-got-error-error-generated-while-running-cuda-enabled-program-700-cudaerrorill