Inserting Date Time into code

The following code uses continuous rainfall data and extracts events based on a certain criteria. Pevents.txt displays the extracted rainfall event and also at the end of the code the total number of events is given. These files are used:

20142018_2.txt (input rainfall data only)

Pevents.txt (extracted event)

I would like to add the date and Time to the code. In other words i would like to see from what day and time is the rainfall event being extracted from. Using this file:

20142018_1.txt(date,time and rainfall event)

function extract_events_onlyP(filename,dt)
P=load(filename);
% -----------------------------------------
% Parameters
% -----------------------------------------
P_tr_mm = 0.51;     % rainfall threshold for the extraction of the event (mm)
P_tr_h  = 4;       % maximum length of non-rainfall period (hour)
No_pio  = 1;       % threshold for zero rainfall (mm)
% -----------------------------------------
% Separating the Rainfall Data into Independent Events 

i=1; jj=1; k=1;
Pevents=[];
while i<length(P)-P_tr_h/dt
    if P(i)>0
        i_in=i;
        
        ii=i_in+P_tr_h/dt-1;
        while sum(P(ii-P_tr_h/dt+1:ii))>=No_pio
            ii=ii+1;
            if ii>length(P),break,end
        end
        i_fin=ii-1;
        for iii=i_fin:-1:i_in
            if P(iii)>0
                i_fin=iii;
                break
            end
        end
        i=i_fin;
        
        if sum(P(i_in:i_fin))>P_tr_mm
            Pev=P(i_in:i_fin);
            Pevents=[Pevents;NaN;k;Pev];
            k=k+1;
        end
    end
    i=i+1;

end



disp(['Total number of events: ',num2str(k-1)])
save Pevents.txt Pevents -ascii -double

[Queena Edwards]

data

Answer 1

[Walter Roberson]

Use fileparts(filename) and take the second output. This will strip off any directory and extension. Now take the first 6 characters. datetime() with 'inputformat'.

Your filename gives the impression of having a year and an hour and minute but no day or month, so I do not see at the moment how to get date and time from the name?

[Queena Edwards]

I'm not sure what you mean. I've attached the data to the original code

Answer 2

[Peter Perkins]

You need to read 20141018_1.txt into a timetable so you have the timestamps for each for of rainfall data. Every other line in that file is blank, and it appears to be UTF-16. Is that really what you have?

Also, it's usually the case that you don't want giant loops like that, and that finding regions of interest can be vectorized. With no comments in the code, I am not going to try to guess what defines your events, but with a clear explanation and an example, people might be able to simplify your code. Maybe not, but your code is pretty opaque, so hard to tell.

[Jeremy Hughes]

Specifically this appears to be UTF-16LE, but UTF-16 is getting treated as UTF-16BE if you leave off the LE.

[Peter Perkins]

Thanks, Jeremy.

>> fname = '20142018_1.txt';
>> opts = detectImportOptions(fname,"Encoding","UTF-16LE");
>> opts.VariableNames = ["TimeStamp" "P"];
>> opts.VariableTypes = ["datetime" "double"];
>> opts = setvaropts(opts,"TimeStamp","DatetimeFormat","dd/MM/yyyy HH:ss");
>> getvaropts(opts,"TimeStamp");
>> tt = readtimetable(fname,opts)
tt =
  477686×1 timetable
       TimeStamp        P
    ________________    _
    01/01/2014 00:00    0
    01/01/2014 00:05    0
[snip]
    31/12/2018 23:50    0
    31/12/2018 23:55    0    

Now do whatever you were doing on P, except do it on tt.P, and tt.TimeStamp(i) (or something like that) gets you the timestamps you need.

[Queena Edwards]

Warning: The encoding 'UTF-16LE' is not supported.
See the documentation for FOPEN. 
> In matlab.io.internal.shared.EncodingInput.set.Encoding (line 19)
  In matlab.io.internal.functions/ExecutableFunction/validateArguments (line 82)
  In matlab.io.internal.functions/ExecutableFunction/validate (line 103)
  In matlab.io.internal.functions/DetectImportOptions/validate (line 37)
  In matlab.io.internal.functions/ExecutableFunction/validateAndExecute (line 97)
  In detectImportOptions (line 137)
  In Change (line 5) 
Warning: The encoding 'UTF-16LE' is not supported.
See the documentation for FOPEN. 
> In matlab.io.internal.shared.EncodingInput.set.Encoding (line 19)
  In matlab.io.internal.mixin/HasPropertiesAsNVPairs/parseInputs (line 34)
  In matlab.io.text.DelimitedTextImportOptions (line 66)
  In delimitedTextImportOptions (line 42)
  In matlab.io.internal.functions/DetectImportOptionsText/getOptsFromDetection (line 138)
  In matlab.io.internal.functions/DetectImportOptionsText/getTextOpts (line 71)
  In matlab.io.internal.functions/DetectImportOptionsText/execute (line 31)
  In matlab.io.internal.functions/DetectImportOptions/execute (line 49)
  In matlab.io.internal.functions/ExecutableFunction/validateAndExecute (line 98)
  In detectImportOptions (line 137)
  In Change (line 5) 

I'm getting this

[Walter Roberson]

That is a warning. In practice for reading it does not affect your execution in any release new enough to support the Encoding option.

[Queena Edwards]

I took out the "Encoding" and "UTF-16LE". It Runs but I'm getting Nat and Nan

    NaT          NaN
    NaT          NaN
    NaT          NaN
    NaT          NaN
    NaT          NaN
    NaT          NaN
    NaT          NaN
    NaT          NaN

[Walter Roberson]

You have a small number of possibilities at this point:

• give up
• be satisfied with the NaT and NaN
• be satisfied with the warning
• turn that particular warning off to hide the problem
• upgrade matlab versions, as the warning goes away around r2019b (which would probably automatically detect the utf16 as well)
• use fopen and fread to read the file as uint8, and pass it through native2unicode to get MATLAB char vector, that you then pass as the first parameter to textscan, with a carefully constructed format as the second parameter in order to parse the data.
• fread, native2unicode and then use text processing methods such as regexp() to extract the data data yourself instead of using textscan.