How can I get higher throughput using fread with tcpip?

I have a device that produces a binary data stream of approx 3MB/sec, i.e. roughly 30Mbps, on a tcpip port.

Tried a few different things with fread using the tcpip object of the Instrument Control Toolbox; but can only get around 2Mbps.

[Friedrich]

What is you code? How do you read the data? How did you configure your TCPIP object on the MATLAB side?

Answer 1

[Philipp S.]

Has the performance of the tcpip / icinstrument fread been improved since? I'm getting just the same weird behaviour here. And I am not quite happy implementing the suggested workaround.

The problem seems to be rooted in the underlying Java method called by icinstrument, as almost all of the time is spent in line 234 of icinstrument.fread:

out = fread(igetfield(obj, 'jobject'), totalSize, type, signed);

[Tom DeLonge]

Have you found a solution? Which workaround are you referring to? Reading data in chunks?

Answer 2

[Vinod]

There are a lot of details to look into -

• Are the two devices on the same network/subnet?
• Is there other networks traffic that could be slowing things on the network?
• Are you using the optimum input and output buffer sizes when creating your TCPIP object?
• Could you use a cross over cable between the device and your computer to cut out other factors?

[Uwe]

• The device and computer are on the same network, directly connected by an Etnernet cable, i.e. no other taffic is present. Have 1GB ports, i.e. no cross-over cable is required.
• Tried different buffer sizes - no appreciable effect on throughput.
• Initially checked bytesAvailable before using fread and (via the profiler) found that almost all the time is spent in the bytesAvailable if clause. Then removed that check and used fread directly, because it waits for the requested amount of data to be available - unless the timeout occurs first. Now almost all the time is spent in fread, leading to no appreciable effect on throughput.
• Also tried to fread different chunk sizes - thought some overhead could adversely affect speeds for smaller chunks. Again, no appreciable effect on throughput.
• Note that a simple .exe written in C easily keeps up with the data, i.e. it does not appear to be a network issue or an issue with the device; but some bottleneck in java or the tcpip object.

Answer 3

[Vinod]

Good to know you've eliminated other potential bottlenecks.

We've done performance tests that achieve in excess of 45 Megabytes/s for binary data using gigabit networks that had other data running through it at the same time. We did find that the number of bytes being FREAD in each read affected the throughput. The best throughput we achieved was with a large inputbuffersize (1e9) and reading in data in chunks of 1e6 (i.e., fread(tcpipReadObj,1e6)).

I suspect you may need to pick some optimum parameters for FREAD size to get the throughput you need. If you are doing any datatype conversions, they may add to your overheads on the FREAD.