Find a portion of a signal using dtw ?

I have the two attached signals and I want to know if "ref" is included somehow (time warping could happen) inside "det" signal.

Figure 1 show ref and det signals:

As you can see ref "can fit" inside det (ref is similar to det(1:100) aprox ) so I try to adjust them normalizing signals and using dtw:

refn=ref/max(ref);
detn=det/max(det);
[~, ix ,iy]= dtw(detn,refn);
figure;plot(refn(iy));hold on;plot(detn(ix));
max(normxcorr2(refn(iy),detn(ix)))

Obtaining:

with a normalized correlation of 0.86

I would love to be able to dtw adjust ref signal over det with the nedd of both signal be the same size so the adjust could be better. For example in this case I could have the signals between x=57:204 and get a better correlation (0.99) of both signals:

x=detn(ix);
y=refn(iy);
max(normxcorr2(x(57:204),y(57:204)))

But I'm missing the first part of ref signal (which could fit quita well) and also some samples at the end because dtw always thinks that both signals have to be the same size but I'm looking also for the possibility of being just a fragment of the other.

Any clue on how to get this adjustment of signals when one can be a portion of the other?

All the best

[Umar]

Hi David,

It sounds like you are seeking guidance on adjusting signals using Dynamic Time Warping (DTW) to improve correlation between signals, especially when one signal is a portion of the other. If this is the case then you need to modify the DTW algorithm to allow for partial matching. One approach is to use a modified DTW variant that considers partial matches by adjusting the constraints on the warping path by implementing a custom DTW algorithm that allows for partial matching by adjusting the warping path constraints.Then, define a cost function that penalizes mismatches between the signals' lengths and apply the modified DTW algorithm to find the optimal alignment between the signals, considering partial matches.

[David Santos]

It sounds good but I'm not sure how to modify dtw algorithm to allow for partial matching, do you mean to use the native dtw function "maxamp"?

[Umar]

Hi David,

Please see my response to your posted comments.

how to modify dtw algorithm to allow for partial matching

You need to adjust the constraints on the warping path by modifying the step patterns in the DTW algorithm to allow for partial matches.

do you mean to use the native dtw function "maxamp"?

Well, native dtw function in Matlab can be used for Dynamic Time Warping (DTW) calculations. However, for the specific requirement of partial matching with adjusted constraints, a custom DTW algorithm needs to be implemented. This custom algorithm will enable the adjustment of warping path constraints to allow for partial matches between signals, providing a more tailored solution to the problem at hand.

Please let me know if you have any further questions.

[David Santos]

any guidance or clues on how to implement that custom dtw? the internal matlab algorithm is hide on a mex function that I can't review

[Umar]

Hi David,

Here's a simplified example to get you started. This custom algorithm should allow for the adjustment of warping path constraints to facilitate partial matches between signals, offering a more personalized solution to the specific problem being addressed.

function [distance, warpPath] = customDTW(signal1, signal2, constraint)
% Initialize variables
n = length(signal1);
m = length(signal2);
warpPath = zeros(n+m, 2);

% Fill the warpPath matrix with constraints
for i = 1:n+m
    warpPath(i, 1) = max(1, i-constraint);
    warpPath(i, 2) = min(n, i+constraint);
end

% Calculate the DTW distance
D = inf(n, m);
D(1, 1) = abs(signal1(1) - signal2(1));

for i = 1:n
    for j = max(1, i-constraint):min(m, i+constraint)
        cost = abs(signal1(i) - signal2(j));
        D(i, j) = cost + min([D(i-1, j), D(i, j-1), D(i-1, j-1)]);
    end
end

distance = D(n, m);
end

So, I define a function customDTW that takes two input signals signal1 and signal2, along with a constraint value. The function calculates the DTW distance between the two signals while considering the constraints for the warping path, it also initializes variables and creates a matrix warpPath to store the warping path constraints based on the input constraint value.

It then computes the DTW distance by iteratively calculating the cost of aligning each point in signal1 with each point in signal2 within the specified constraints.The final DTW distance is stored in the variable distance, which is returned along with the warpPath matrix. Please let me know if you have any further questions.

[David Santos]

Thanks a ot for your answer Umar!

So if I want that signal1 stay unmodified (no warping at all) should I put that constraint to 0?

% Fill the warpPath matrix with constraints
for i = 1:n+m
    warpPath(i, 1) = max(1, i-0);
    warpPath(i, 2) = min(n, i+constraint);
end

And for the indexes (ix, iy in the matlab dtw function) i Guess It must be obtained from the D matrix but do you know how?

All the best

[Umar]

Hi David,

To keep signal1 unmodified (no warping), you should set the constraint to 0 in the warpPath matrix. By setting warpPath(i, 1) = max(1, i-0) and warpPath(i, 2) = min(n, i+constraint), it will make sure no warping for signal1.

Now, to obtain indexes (ix, iy) from the D matrix in DTW function, you can find the optimal warping path by backtracking through the accumulated cost matrix D. The indexes can be extracted by following the path with the lowest accumulated cost. It sounds like you are making progress.