Match background onto signal

Hey there,

I am trying to match a background from a measurement onto a signal. Both signals are just a 1x20002 array. Sometimes the noise is so high that you can't see a peak in the signal at all but if you subtract the background at the correct spot you have a very nice peak. The problem is: due to jitter in my experimental setup background and signal do not always match very well in phase.

My solution so far works but is awfully slow: I take the signal (sig), create a three times longer signal (sig_long) with the original signal in the middle in case you wanna shift it by it's whole length. Then I do a for loop for as many steps I want where I subtract the background from the middle shifted from -shift to +shift (as iterations of the for loop). In each iteration I calculate the area of the middle of the long signal with trapz and store it in a variable. Then, whenever the area is smaller than the area from the former iteration (which means that the background fits better in this spot) I store the shift-index as new best match index. The problem is: all the trapz-ing takes forever. My signals and backgrounds are 20002 datapoints long so if I shift from for example -5000 to +5000 points MATLAB has to trapz the signal 10000 times! That's a lot! So.. it works.. but it's ugly!

Does anyone have any suggestion on how to improve this? Maybe with FFT? But those signals are definately non sinusodial..

Cheers ~Patrick

Answer 1

[Image Analyst]

Sounds like you're just describing an autocorrelation, like you can do with xcorr(). But I don't know why you're doing it. What do you expect to get from that? That's not going to denoise the signal. Do you just want to denoise the signal? There are methods for denoising. You can use median filter, convolution, Savitzky-Golay filter, LOWESS, LOESS, RLOESS, etc. It depends on the nature of the signal and the nature of the noise. Can you give us a plot of what the pure, perfect signal alone should look like, what a perfect noise-free background looks like, what pure noise alone would look like, and what the noise+signal looks like? Also, provide some data and code to read it in - that would make it easy for us to help you.

[Image Analyst]

What if you did a normalized cross correlation (demo attached) to find out where they overlap best? Then shift to that point and then subtract?

Answer 2

[Patrick]

Hey,

I've attached a background and a signal file. If you subtract both you know what it should look like (maybe with smooth of about 150). Btw: you have to take the second column of both files. The first is just the x-axis. I just read it in with "csvread(signal.txt,5);". The problem is that I collected those traces kina manually and waited till the signal and the background overlapped on the scope perfectly but since I wanna automate everything matlab would have to do the shift for me. The reason it shifts is that there is jitter in the whole setup so if you collect a background with the detector covered and a signal with the detector open those two don't nessecarily overlap very well. But as you can see: the "signal" is so full of noise that you almost can't see nothing but if you subtract the background it looks ok.

I hope this clarifys my problem.

Thank you for your help.

Patrick

Answer 3

[Patrick]

Thanks guys! Xcorr did the trick! I didn't really know about this (mathematical) function in general but I did some research now and it's just what I wanted but unbelievably much faster! ;)

[Image Analyst]

Patrick, be careful with cross correlation. OK, hands up those of you who think the cross correlation is highest when the signals overlap the "best". OK, I see most of the hands up . Don't be like most people who think that the cross correlation signal is highest when the two signal overlap best. This is not necessarily the case and it's easy to think of situations where it's not the case. It only tells you where the sum of the products of the two arrays is highest, which may not be where you would logically think the arrays overlap best. A few spurious noise elements could throw the best shift way off. That's why I recommended normalized cross correlation instead of correlation.