Advice on how to speed up code?

Good Afternoon,

I am a novice with coding and am looking for some advice on how to speed up this code I've written if anyone has a bit of time/knowledge/advice to give? I would be so grateful. It's taking approximately 55 hours to run which is ridiculous. What I have to start with is a spreadsheet with temperature data, stored in four coloumns - station, date, time, temperature. There are 63 different stations, 189 different days, and 96 time points for each day (every fifteen minutes). What I want to end up with is a matrix of 63 rows, with 18144 columns for each row - each day and time point in chronological order. The data in the spread sheet is not uniform though - each station is not operational for the entire 189 days, and there are random skips in time sometimes. I want my final matrix to have spaces where there is no data as I need to calculate different averages and periods and need to assign the data to the correct time point. The code I have attached is a series of loops - which I know is bad - but I can't think of how to avoid it. I hope this makes sense - I am trying to select for each station, day and time the corresponding value and input this into the correct space in my matrix 'Temp2003Space'.

count = 0;
for station1 = 1:D
 for day1 = 1:E
     for Time2 = 1:Time
      count = count + 1
      F = double(strcmpi(TempExample_raw(2:end,1),UniqueWeatherStations(station1)));
      F(F == 0) = nan;
      G = double(DateNumbers4(2:end,1)== UniqueDateNumbers(day1)); 
      G(G == 0) = nan;
      Time4 = double(strcmp(TempExample_raw(2:end,3),TP2(Time2)));     
      Time4(Time4 == 0) = nan;
      r= find(F==1 & G==1 & Time4==1);
          if r > 0 
          TEMPERATURE = cell2mat(TempExample_raw(r+1,4));
          H = double(DateKey2(1,:)== UniqueDateNumbers(day1));
          H(H == 0) = nan;
          I = double(strcmp(TPRep3(1,:),TP2(Time2)));
          I(I == 0) = nan;
          r2 = find (H==1 & I==1);
          Temp2003Space(station1, r2) = TEMPERATURE;
          end
    end
 end
end

[Adam]

doc profile

is always the best starting point to identify which parts of your code are taking the most time, then you can focus on ways to speed up just these bits.

[Stephen23]

"The code I have attached is a series of loops - which I know is bad "

Why are loops bad? Where did you read that?

To find out where your code is slowest run the profiler: what lines are taking the most time to run?

I notice that Temp2003Space is not preallocated at all: this is going to make your code very inefficient. You could easily calculate the total number of possible sample times, and so allocate the whole matrix before the loop.

You should definitely read this:

https://www.mathworks.com/help/matlab/matlab_prog/techniques-for-improving-performance.html

[Chameleon17]

Oh it is preallocated - I just took off the bit of the code where I load everything as I have tested that and it runs fine and very quickly- it's this section that takes ages. I will look at the profiler, and that link, thank you! I thought loops were slow - and that it's better to just do an operation to the entire matrix as a whole rather than bit by bit. I don't have an exact reference for loops being bad - it's just something I think I thought people said you should avoid.

Answer 1

[Steven Lord]

If you can order the data so the times represent the rows and the stations the column, you could import your data as a timetable and use the retime and fillmissing functions to perform calculations on various time periods.