Can MATLAB tell me which gpu device is the one connected to the display?

For example, with a GTX590, I have two gpu devices to choose from. Now my theory is that for CUDA kernel performance it might be best to select the device that is not also busy controlling the display. So I guess I am really asking two questions:

1) Is there in fact a performance hit when running a CUDA kernel on a device that is also connected to the main display and rendering my desktop?

2) If so, how can I tell which gpuDevice is which? Thanks, -nuun

Answer 1

[Konrad Malkowski]

1. Yes there will be limitations on what can run on GPU that is used for rendering. In particular the CUDA kernels cannot run for more than 2 to 5 seconds on such a card. So if you have a computation that takes longer, you will need to break it down into smaller chunks. As computing each smaller chunk will involve an individual kernel launch, there will be a performance degradation.

2. I think you might be able to determine if a device is powering a screen by checking the KernelExecutionTimeout property of the object returned by the call to gpuDevice.

[Naor Movshovitz]

Thanks Konrad, this helps a lot. Unfortunately the KernelExecutionTimeout is True for both devices of a GTX590 although only one is connected to a display. This is a read-only property.

[Jason Ross]

See if you can set the one for compute to TCC mode, this may change this property value. You can do this with nvidia-smi

Answer 2

[Jason Ross]

Ways I could think of to determine which device is connected to a monitor. Some of this is a bit of wild conjecture on my part, so take it with a bit of salt:

• The output of gpuDevice gives the total memory and free memory on the device. In the case of a system that has a card connected and one reserved for compute, the compute resource will have no memory consumed, and the display one will have some memory consumed. This isn't a bullet-proof strategy, though.
• If you have access to the nvidia-smi utility (I'm not sure if it works for the GTX series of cards -- it says it's only supported on Tesla and some Quadro cards), you can check the driver mode of each GPU. You can get all the syntax from "nvidia-smi -h", but the main command to use is likely "nvidia-smi --query". A card reserved solely for compute will be in "TCC" mode, and one for graphics rendering will be in "WDDM" and can be used for compute and compute. You could parse this output to find out the "best" one to use. This is valid for Windows only -- WDDM mode does not apply on Linux or Mac
• If you can't use nvidia-smi, in the CUDA SDK there is an example called "deviceQuery" that gets information about the device. It would be possible to look through these fields and build a small executable that returns the "best" one. There might also be something available at this level that tells you if the monitor is connected -- since it has to know to prevent you from turning off WDDM on a card with a monitor connected!

[Naor Movshovitz]

Thanks Jason.

I will look into the smi technique. The deviceQuery hack might work, but I am not sure how to relate the device index from the CUDA SDK to the device index returned by PCT. It is tempting to just assume that device 0 of the SDK corresponds to device 1 in MATLAB but...

Free memory reported by gpuDevice is the same for both devices, when selected, and NaN for an unselected device. This is slightly suspicious, I agree that the rendering device should be reserving some memory.

Answer 3

[Geoff]

If you are running Windows, you could write a little MEX script that returns a list of devices and their status from the API call EnumDisplayDevices:

http://msdn.microsoft.com/en-us/library/dd162609(v=vs.85).aspx

[Naor Movshovitz]

Thanks Geoff. This is a good API function to know, but I don't think it can help me select the correct gpuDevice inside MATLAB. Suppose I decide based on the return from EnumDisplayDevices that I want to work with device index 1. Is this the same as the device indexed 2 by MATLAB? It is tempting to think so but I don't know if we should rely on it. Especially in the somewhat confusing case of a dual-gpu card where both GPU devices cannot be distinguished by model name or hardware parameters.

[Geoff]

Are you talking about one card with two display outputs, or two cards? In the case of two cards, I would have thought that the DISPLAY_DEVICE structure returned from EnumDisplayDevices would contain enough information (device name, device context, state flags). I usually use EnumDisplayMonitors, but for a different purpose (positioning windows on available monitors), so I haven't tried EnumDisplayDevices. Tempted to try it now =)

[Naor Movshovitz]

Neither. The GTX590 is a dual-core card. That is, one "device", two gpus. And two gpuDevice in ML, but with the same name.

Answer 4

[Geoff]

Well, I couldn't help myself. I had to try it. Here's the result of running an EnumDisplayDevices test on my machine:

Device: 0
  Name = \\.\DISPLAY1
  String = NVIDIA GeForce 9800 GT
  Flags = active pruned primary

Device: 1
  Name = \\.\DISPLAY2
  String = NVIDIA GeForce 9800 GT
  Flags = active pruned

Device: 2
  Name = \\.\DISPLAY3
  String = NVIDIA GeForce 9400 GT
  Flags =

Device: 3
  Name = \\.\DISPLAYV1
  String = RDPDD Chained DD
  Flags = mirror

Device: 4
  Name = \\.\DISPLAYV2
  String = RDP Encoder Mirror Driver
  Flags = mirror

My machine has one graphics card (the 9800GT) with two outputs (both of which are attached to monitors), and presumably the 9400GT is the on-board graphics.

The code:

/* Output available display devices */
DISPLAY_DEVICEA dd;
dd.cb = sizeof(dd);

DWORD iDevNum = 0;

while( EnumDisplayDevicesA(NULL, iDevNum, &dd, 0) )
{
  printf( "Device: %d\n", iDevNum );
  printf( "  Name = %s\n", dd.DeviceName );
  printf( "  String = %s\n", dd.DeviceString );
  printf( "  Flags =" );
  if( dd.StateFlags & DISPLAY_DEVICE_ACTIVE ) printf(" active");
  if( dd.StateFlags & DISPLAY_DEVICE_MIRRORING_DRIVER ) printf(" mirror");
  if( dd.StateFlags & DISPLAY_DEVICE_MODESPRUNED ) printf(" pruned");
  if( dd.StateFlags & DISPLAY_DEVICE_PRIMARY_DEVICE ) printf(" primary");
  if( dd.StateFlags & DISPLAY_DEVICE_REMOVABLE ) printf(" removable");
  if( dd.StateFlags & DISPLAY_DEVICE_VGA_COMPATIBLE ) printf(" vga");
  printf( "\n\n" );
  iDevNum++;
}

I presume you are looking for the devices that are not mirrored or active.

[Geoff]

Oh, I see you accepted this answer. Did it work for you? Perhaps you could comment on how you used the resulting structure to select the appropriate device, and how that translated into MatLab? Might be useful to others who want to do the same thing. =)

[Naor Movshovitz]

Unfortunately that was an accidental 'Accept' and I couldn't find a way to 'unaccept' :(

That was a good idea, but I don't think there is a way to robustly invert the information from what GPU is controlling this monitor, to what monitor is attached to this GPU. I think the answer here would be to ask NVIDIA to include a utility function in the CUDA C API in the next version, then ask TMW to include a wrapper m-file in the next version of PCT.

[Naor Movshovitz]

Here is the output from my machine, with a single GTX590, which is a dual core card. One of the cores is connected to two monitors. I still don't see how I can translate this into a choice of GPU in either a CUDA runtime or the PCT.

From the nvidia-smi, I see that *both* cores are in WDDM mode. I am reluctant to mess with that for fear of breaking something on a machine that is also used for gaming :)

Device: 0

Name = \\.\DISPLAY1

String = NVIDIA GeForce GTX 590

Flags = active pruned primary

Device: 1

Name = \\.\DISPLAY2

String = NVIDIA GeForce GTX 590

Flags = active pruned

Device: 2

Name = \\.\DISPLAYV1

String = RDPDD Chained DD

Flags = mirror

Device: 3

Name = \\.\DISPLAYV2

String = RDP Encoder Mirror Driver

Flags = mirror

Device: 4

Name = \\.\DISPLAYV3

String = RDP Reflector Display Driver

Flags = mirror

[Geoff]

Hrmmm, I see what you're talking about. I am not up with the play regarding GPU cores and physical outputs. I'm reluctant to delete this answer, as it seems potentially useful... but I will if there's no way for an Editor to come along and 'unaccept' it.

[Walter Roberson]

The mortal editors cannot "unaccept", but the matlabcentral team can.

[Randy Souza]

Hi Naor & Geoff,

I unaccepted this answer, no need to delete it.