Error when converting Matlab time (double) to datetime (raw data captured from RealTerm)

I am reading raw data from an instrument via RealTerm (3.0.1.44) which gives me the option to dump it in a txt file while also appending a Matlab time stamp. My main interest is to keep millisecond precision as I must get the actual sampling rate given by the instrument

The first 10 captured Matlab time rows are the following:

1. 738545.739572000
2. 738545.739575000
3. 738545.739578000
4. 738545.739581000
5. 738545.739583000
6. 738545.739586000
7. 738545.739589000
8. 738545.739591000
9. 738545.739594000
10. 738545.739597000

As you can see, they are sequential double-precision numbers, however, when converting them to their corresponding datetime values I get a mismatch on the sixth value:

1. '05:01:59.021000000'
2. '05:01:59.280000000'
3. '05:01:59.539000000'
4. '05:01:59.798000000'
5. '05:01:59.971000000'
6. '05:01:00.230000000'
7. '05:01:00.490000000'
8. '05:01:00.662000000'
9. '05:01:00.922000000'
10. '05:01:01.181000000'

Why the clock time jumps from '05:01:59.971000000' to '05:01:00.230000000' if the double type times are sequential?

These values are obtained applying the following line inside a for-loop to dataCell, a cell that contains the imported values in columns where the first corresponds to the times and pivot is just an index:

currentTime = datetime(datestr(dataCell{pivot,1}, 'hh:mm:ss.FFF'),'InputFormat','HH:mm:ss.SSSSSS');

currentTime.Format = 'hh:mm:ss.SSSSSSSSS'

Is this the wrong way to convert Matlab times to hours with millisecond precision?

It should be noted that this also happens in other data acquisitions where the numbers are sequential too.

I intend to use the first value as the start time then compute the time deltas in milliseconds using datevec then datenum, however, I clearly get negative deltas:

1. 0
2. 0.00431666666666667
3. 0.00863333333333333
4. 0.0129500000000000
5. 0.0158333333333333
6. -0.979850000000000
7. -0.975516666666667
8. -0.972650000000000
9. -0.968316666666667
10. -0.964000000000000

Thanks for yor help

Answer 1

[Stephen23]

"Why the clock time jumps from '05:01:59.971000000' to '05:01:00.230000000' if the double type times are sequential?"

Because your data are less sequential than you think they are. Let have a look:

N = [738545.739572000; 738545.739575000; 738545.739578000; 738545.739581000; 738545.739583000; 738545.739586000; 738545.739589000; 738545.739591000; 738545.739594000; 738545.739597000];
format long G
diff(N) % not that regular

ans = 9×1

1.0e+00 * 3.00002284348011e-06 3.00002284348011e-06 2.99990642815828e-06 2.00001522898674e-06 3.00002284348011e-06 3.00002284348011e-06 2.00001522898674e-06 3.00002284348011e-06 2.99990642815828e-06

"Is this the wrong way to convert Matlab times to hours with millisecond precision?"

Yes. Not only is this the wrong way to convert such values, it is also the wrong way to store such values: a serial date number is limited by the precision of the DOUBLE class, which means that it cannot store more accuracy than

P = eps(N(1)) % days

P = 
      1.16415321826935e-10

P*24*60*60 % seconds

ans = 
      1.00582838058472e-05

So what you are attempting is already beyond anything that can be stored in a DOUBLE serial date number: they cannot store micro-second precision for the dates that you are using. As the documentation clearly states, avoid the old, deprecated, low-precision functions DATESTR, DATENUM, and DATEVEC. Use DATETIME and DURATION objects instead.

And finally to answer your question, this is the best way (but of course it will not magically make information appear that is not stored due to the precision limit of the DOUBLE class):

T = datetime(N,'ConvertFrom','datenum','Format','HH:mm:ss.SSSSSSSSS')

T = 10×1 datetime array
   17:44:59.020799316
   17:44:59.280000000
   17:44:59.539203125
   17:44:59.798395019
   17:44:59.971196533
   17:45:00.230398437
   17:45:00.489600341
   17:45:00.662401611
   17:45:00.921603759
   17:45:01.180795654

D = diff(T);
D.Format = 'hh:mm:ss.SSSSSS'

D = 9×1 duration array
   00:00:00.259200
   00:00:00.259203
   00:00:00.259191
   00:00:00.172801
   00:00:00.259201
   00:00:00.259201
   00:00:00.172801
   00:00:00.259202
   00:00:00.259191

[Cris LaPierre]

Note that datetimes have a date associated with them, even if it is not displayed.

N = [738545.739572000; 738545.739575000; 738545.739578000; 738545.739581000; 738545.739583000; 738545.739586000; 738545.739589000; 738545.739591000; 738545.739594000; 738545.739597000];
T = datetime(N,'ConvertFrom','datenum','Format','hh:mm:ss.SSSSSSSSS')

T = 10×1 datetime array
   05:44:59.020799316
   05:44:59.280000000
   05:44:59.539203125
   05:44:59.798395019
   05:44:59.971196533
   05:45:00.230398437
   05:45:00.489600341
   05:45:00.662401611
   05:45:00.921603759
   05:45:01.180795654

T.Format = 'default'

T = 10×1 datetime array
   24-Jan-2022 17:44:59
   24-Jan-2022 17:44:59
   24-Jan-2022 17:44:59
   24-Jan-2022 17:44:59
   24-Jan-2022 17:44:59
   24-Jan-2022 17:45:00
   24-Jan-2022 17:45:00
   24-Jan-2022 17:45:00
   24-Jan-2022 17:45:00
   24-Jan-2022 17:45:01

Also, I would suggest using HH instead of hh to not confuse am with pm (5pm = 17hr).

T.Format = 'HH:mm:ss.SSSSSSSSS'

T = 10×1 datetime array
   17:44:59.020799316
   17:44:59.280000000
   17:44:59.539203125
   17:44:59.798395019
   17:44:59.971196533
   17:45:00.230398437
   17:45:00.489600341
   17:45:00.662401611
   17:45:00.921603759
   17:45:01.180795654

Answer 2

[Daniel Melendrez]

Your answers are awesome, guys!

This clarifies a lot.

Let me test your recommended solutions and I will get back asap with the final answer.

Have a great day

[Daniel Melendrez]

To mark this question as answered, the trick was on this line, as recommended by Stephen:

T = datetime(N,'ConvertFrom','datenum','Format','hh:mm:ss.SSSSSSSSS')

and just to clarify: I was aware the the precision limit from DOUBLE type values was limiting the resolution from my values, however, the data was imported using the Import Data tool from ML. I couldn't find any options for saving them using uint_64.

Now I know that DATETIME and DURATION are the way to go.

Thank you once agian!