Error using xlim for datetime values

I need to collect data from a text file, which I am attaching to this question, and the data collected needs to be plotted and the x axis limits need to be changed. The simplified code is as follows.

DayFile = ('SI010218.txt');
DayFileX = fopen(DayFile, 'rt');
header_cell = textscan(DayFileX, '%[^\n]', 1, 'HeaderLines', 12);
header_fields = regexp(header_cell{1}{1}, ';', 'split');
junk = fgetl(DayFileX);   %because of the ^\n the \n is still in the buffer
units_line = fgetl(DayFileX);
units_fields = regexp(units_line, ';', 'split');
numfields = length(header_fields);
fmt_cell = repmat({'%f'}, 1, numfields);
fmt_cell{1} = '%[^;]';  %D format cannot handle embedded blanks
fmt_cell{100} = '%{HH:mm:ss}D';
fmt = [fmt_cell{:}];
data_cell = textscan(DayFileX, fmt, 'delimiter', ';', 'CollectOutput', true);
fclose(DayFileX);
data_cell2D = [num2cell( datetime( data_cell{1}, 'InputFormat', 'dd.MM.uuuu HH:mm:ss') ), ...
                 num2cell( data_cell{2} ), ...  %numeric
                 num2cell( data_cell{3}  - dateshift(data_cell{3}, 'start', 'day') ), ...  %datetime, make it duration
                 num2cell( data_cell{4} ) ];    %numeric
adjusted_headers = matlab.lang.makeUniqueStrings( matlab.lang.makeValidName( header_fields ) );
table1 = cell2table(data_cell2D, 'VariableNames', adjusted_headers);
table1.Properties.VariableDescriptions = header_fields;
table1.Properties.VariableUnits = units_fields;    
plot((table1{:,1}),(table1{:,5}));
startD = '20180201010000';
finishD = '20180201050000';
disp(startD);
disp(finishD);
tstart = datetime(startD,'InputFormat','yyyyMMddhhmmss');
tend = datetime(finishD,'InputFormat','yyyyMMddhhmmss');
disp(tstart);
disp(tend);
xlim([tstart tend]);

This will give me the following error.

>> test
20180201010000
20180201050000
 01-Feb-2018 01:00:00
01-Feb-2018 05:00:00

Error using set
While setting the 'XLim' property of Axes:
Value must be a 1x2 vector of numeric type in which the second element is larger than the first and may be Inf

Error in xlim (line 43)
        set(ax,'xlim',val);

Error in test (line 44)
xlim([tstart tend]);

The first four lines clearly mean that the limits I wish to have on the date/time xaxis are recognized. The 'tend' is also definitely larger that 'tstart'. So the second element is larger than the first. Why am I getting this error?

[Kevin Chng]

tstart = datetime(2018,2,1,1,0,0)
tend = datetime(2018,2,1,5,0,0)
xlim([tstart tend]);

Do you mind to try this see whether got any error or not?

is table1{:,1} a datetime variable?

[jonas]

If you work with time-series, you should utilize the functionality of timetables. I'm guessing that Kevin Chng is correct, and the x-axis is not set to actual datetime format.

[Chamath Vithanawasam]

@Kevin it is in datetime format actually. I attached the .txt file so you can try it out yourself. The data should save itself to a folder called 'table1'. When I tried your approach, it gave me the same error.

>> test

tstart = 

     01-Feb-2018 01:00:00

tend = 

     01-Feb-2018 05:00:00

Error using set
While setting the 'XLim' property of Axes:
Value must be a 1x2 vector of numeric type in which
the second element is larger than the first and may
be Inf

Error in xlim (line 43)
        set(ax,'xlim',val);

Error in test (line 48)
xlim([tstart tend]);

[dpb]

That's a version problem -- I didn't think of that. Early incarnations of the overlaoded plot with the datetime ruler axes accepted it in the argument list but still used datenum to plot with as the original plot and datetick combination.

The implementation was completed to use datetime for it internally by R2017b but I don't know the exact release.

Use

tstart = datenum(2018,2,1,1,0,0)
tend = datenum(2018,2,1,5,0,0)
xlim([tstart tend]);

and it will work.

Just type xlim at the command line and you'll see the datenum values returned instead of datetime values.

Answer 1

[dpb]

And, while above Answer is true, the comment about using readtable is sufficiently worthwhile will add another based on it with the actual file--

opt=detectImportOptions('SI010218.txt');         % get Matlab's best guess of file format
opt=setvartype(opt,'TimeStamp','datetime');      % set the time column to datetime from char
opt=setvaropts(opt,'TimeStamp','InputFormat','dd.MM.yyyy HH:mm:ss');  % and set input format
opt.VariableUnitsLine=opt.VariableNamesLine+1;   % not req'd, but since is in file...
t=readtable('SI010218.txt',opt);                 % and then just read the table
plot(t.TimeStamp,t.TrfTmp_Max_)                  % plot
t1=datetime(2018,2,1,1,1,0);                     % set the desired limits
t2=datetime(2018,2,1,5,0,0);
xlim([t1 t2])

yields

The variable names in the file aren't very conducive to being used programmatically, unfortunately, but if there are some better names for specific columns of most interest (presuming there are some out the total of over 150), those can be set programmatically either in the opt object for import or after loading.

This removes the other machinations entirely...the table is really a quite useful (newish) enhancement, indeed.

[Chamath Vithanawasam]

This option is much simpler and does work. However, only after 2016b if I am not mistaken. My 2015a MATLAB will not read 'detectImportOptions'

[jonas]

@Chamath, I ran your original code with the data you provided and it works perfectly fine, giving me no error and the exact same output as DPB. There is probably an issue with your release failing to correctly identify the correct class?

[dpb]

OK, release issues. R2015a still uses datenum for the internals even though the plot function will accept datetime. Convert your start/end dates to datenum instead of datetime and then xlim error will also go away; that's the reason for the "must be numeric" error, datenum is just a scaled double. I should've thought of this; ran into it previously w/ R2016b but just didn't think about using an earlier release than R2017b that I'm using here, sorry.

[dpb]

With earlier releases, it's more of a pain to convert data because you have to do all the type conversion formatting explicitly, because readtable won't let you read variable units; if you read variable names then it thinks it knows best and the data must start on the next line. In that case, then, everything is imported as cellstr array...

t=readtable('Si010218.txt','headerlines',12,'delimiter',';');
t(1,:)=[];        % remove the units line
t.TimeStamp=datetime(t.TimeStamp,'inputformat','dd.MM.yyyy HH:mm:ss');

By reading the variable names and 12 header lines, everything is brought in as cell string array so have to convert the numeric values.

I didn't know what all the variable types were so just did those that were numeric by

for i=2:size(t,2)
  try
    t.(i)=str2double(t.(i));
  catch
    disp(i)
  end
end

which left a list of 13 columns that are something else or have missing values or somesuch. But one can then do similar as to Steven and convert them as they should be or you can build a cell string array of formats and pass it as the 'Format' named parameter. Since, as you note above, textscan can't parse the blank in the date string, import it as cellstr as is done by default.

The alternative would be to use

t=readtable('Si010218.txt','headerlines',14,'delimiter',';','readvariablenames',0);
t.Var1=datetime(t.Var1,'inputformat','dd.MM.yyyy HH:mm:ss');

>> t(1:4,1:10)
ans = 
            Var1            Var2     Var3    Var4     Var5     Var6     Var7     Var8    Var9    Var10
    ____________________    _____    ____    _____    _____    _____    _____    ____    ____    _____
    01-Feb-2018 00:00:54    30.16    29.4    29.97    39.46    38.13    39.08    26.1    85.4    100  
    01-Feb-2018 00:01:55    30.16    29.4    29.97    39.46    38.13    39.08    26.1    85.4    100  
    01-Feb-2018 00:02:56    30.16    29.4    29.97    39.46    37.94    39.08    26.1    85.4    100  
    01-Feb-2018 00:03:57    30.16    29.4    29.97    39.46    38.13    39.08    26.1    85.4    100  
>>

You can set variable names as desired and make the fixup to a duration pretty easily.

Just sticking with the default names for syntax,

t.Var100=t.Var100-dateshift(t.Var100,'start','day');

Of course the same issue about plot and the mixture of external interface datetime and internal datenum still exists.

Answer 2

[dpb]

OK, I answered in couple of comments, but they're buried so will post (yet another :) ) Answer to the original problem only.

OK, this is a release issue.

R2015a still uses datenum for the internals even though there is an overloaded plot function that accepts datetime object. I'm not sure when TMW got the internals all to be consistent; by R2017b that I have they are, so my previous example works correctly. Of course, it also has detectImportOptions that simplifies a lot of the complexity that your release doesn't yet have, either.

Convert your start/end dates to datenum instead of datetime and then xlim error will also go away; that's the reason for the "must be numeric" error, datenum is just a scaled double. I should've thought of this; ran into it previously w/ R2016b but just didn't think about using an earlier release than R2017b that I'm using here.

tstart = datenum(2018,2,1,1,0,0)
tend = datenum(2018,2,1,5,0,0)
xlim([tstart tend]);

will solve your xlim problem; that datenum is just scaled double is the reason for the error message about "must be numeric".

You can just type

xlim

at command line to return the current limits and you'll see the datenum values,

datestr(xlim)

will display their interpretation as dates.

[jonas]

+1 to the top!

[dpb]

Thanks, should have recognized the problem right off the bat as had run into it previously. Let trying to simplify the import overwhelm and consequently just whiffed on the actual problem initially.

Answer 3

[Steven Lord]

Looking at this section of your code:

    data_cell2D = [num2cell( datetime( data_cell{1}, 'InputFormat', 'dd.MM.uuuu HH:mm:ss') ), ...
                   num2cell( data_cell{2} ), ...  %numeric
                   num2cell( data_cell{3}  - dateshift(data_cell{3}, 'start', 'day') ), ...  %datetime, make it duration
                   num2cell( data_cell{4} ) ];    %numeric
    adjusted_headers = matlab.lang.makeUniqueStrings( matlab.lang.makeValidName( header_fields ) );
    table1 = cell2table(data_cell2D, 'VariableNames', adjusted_headers);

Why are you stuffing your data into a cell array only to immediately convert that cell array into a table? Just make the table directly.

firstDatetime = datetime( data_cell{1}, 'InputFormat', 'dd.MM.uuuu HH:mm:ss');
firstNumeric = data_cell{2};
firstDuration = data_cell{3} - dateshift(data_cell{3}, 'start', 'day');
secondNumeric = data_cell{4};
adjusted_headers = matlab.lang.makeUniqueStrings( matlab.lang.makeValidName( header_fields ) );    
table1 = table(firstDatetime, firstNumeric, firstDuration, secondNumeric, ...
               'VaribleNames', adjusted_headers);

Or rather than using textscan and converting the raw data into a table, read it in directly into a table using readtable. The 'Format' argument can accept the same formats as textscan.

Your duration column would probably need to be computed after the fact, but table indexing (specifically the T.var syntax in the third row of the table on that page) should make that easy.

[dpb]

Besides Steven's correct response to convert directly instead of indirectly, I'd guess that readtable would save even more effort. Unfortunately, you didn't actually get the input file attached so we can't see just how it is constructed.

[Chamath Vithanawasam]

Terribly sorry, I forgot to upload it. Will upload now.

[Chamath Vithanawasam]

Steven's suggestion gave me the following error.

>> test
Error using setVarNames (line 35)
The VariableNames property must contain one name for each
variable in the table.

Error in table (line 305)
            t = setVarNames(t,vnames); % error if invalid,
            duplicate, or empty

Error in test (line 22)
    table1 = table(firstDatetime, firstNumeric, firstDuration,
    secondNumeric, ...

Answer 4

[dpb]

While Steven L has the better overall, I'll point out the specific cause for the error in the existing code; as the other two respondents indicated, your plot axis isn't time because in

...
table1 = cell2table(data_cell2D, 'VariableNames', adjusted_headers);
...
plot((table1{:,1}),(table1{:,5}));

You dereferenced the table data as table1{:,1} with the "curlies" ('}') which returns the underlying data as the native double array instead of as a table; hence plot() x-axis is numeric and hence the error from xlim

plot(table1(:,1),table1(:,5))

would work, but more readable would be

plot(table1.X,table1.Y)

where 'X,Y' are placeholders for the name of the table variables in column 1 and 5, respectively. Either syntax returns a table object and, presuming (altho I didn't read the code in detail) the X value is a datetime, then the plot will be versus a datetime on the x-axis and then xlim would succeed.

[Chamath Vithanawasam]

This seems plausible, but I am getting an error. I already attached the .txt file from which data will be imported. The workspace shows me the data in table1 as shown in the screenshot below.

This is just a small section though.

So in order to try your method I should do something like this?

X = ('TimeStamp');
Y = ('HsTmp_Max_');

plot(table1.X,table1.Y);

Because it gave me the error.

Error using test (line 28)
Unrecognized variable name 'X'.

Apologies for these trivial issues.

[jonas]

You are using the wrong syntax for dynamic field names, try

plot(table1.(X),table1.(Y));

The braces are not stored in X and Y.