Inefficient tall timetable lookup script, how to optimize?

Hi all,

I have two tables defined, one tall timetable tableA (1Mx18) and one regular table tableB (46x5).

• tableA contains sensor observations logged with a timestamp, sensorID and its measurements (16 features).

• tableB contains a log where each sensor is located in time.

Goal is to horconcat the location and sublocation from tableB to tableA within the period of the specific sensorID (tableB.startDate, tableB.endDate). In the mentioned example here above, the location of observations 1 and 2 from tableA must be 'N105'. To do so, I've written a very inefficient loop which checks for all rows in A if the timestamp is within the timeperiod in tableB and if the sensorID matches:

tableA = gather(talltimetableA);  
for kk = 1:size(tableA,1)
      for jj = 1:size(tableB,1)
          if tableA.sensorID(kk) == tableB.sensorID(jj) && isbetween(tableA.TimeStamp(kk),tableB.StartDate(jj),tableB.EndDate(jj)) == 1
              tableA.Location(kk) = tableB.Location(jj);
              tableA.SubLocation(kk) = tableB.SubLocation(jj);
          end
      end
  end

I need a 30-minute resolution at minimum for the observations in tableA. I'm pretty sure this script can optimized such that the script does not need 19 hours to be evaluated.

Thanks in advance,

Bas

Answer 1

[Star Strider]

I cannot follow what you want to do well enough to provide any specific code. However, the join (link) function or one of its friends (linked to in and at the end of that page) are probably able to do what you want.

[BasO]

Thank you for your reply, however join needs unique keys to join tableB to tableA. I've updated the question to provide more clarity.