How to filter noise from a time series without losing important informations?

22 ビュー (過去 30 日間)
Hello everyone,
as described in the topic I have a time series like this:
Zoomed in you can see there is a lot of noise.
I need to filter the noise in the green rectangle without losing informations like the spike in the green circle.
As can be seen in the frequency domain there is a lot going on.
I have tried smoothing and filtering endlessly with Data Analyser Toobox and the Signal Processing Toolbox. I have tried the Filter Designer just like the filter funktion with various settings like Savitzky-Golay Filter or Exponential Moving Average Filter or Bandpass. But everytime I get rid of the noise I also lose to much important informations.
Can anybody tell if and how I can achive this?
I have attached the data.mat with the time series in case there is someone who likes to try.
Best regards
Fabian
  2 件のコメント
Sharmin Kibria
Sharmin Kibria 2021 年 6 月 25 日
編集済み: Sharmin Kibria 2021 年 6 月 25 日
Did you try denoising it using the Signal analyzer app? I tried to denoise the signal with symlet 4 wavelet (default setting) and got the attached denoised version. I was able to preserve 98% of the original signal energy in the denoising process. Do you think it is good enough?
Thanks
Sharmin
Fabian Lürßen
Fabian Lürßen 2021 年 6 月 28 日
Yes, indeed I tried. But a rest of the nois remains and although 98% of the original signal energy is preserved the form of the curves is changed.

サインインしてコメントする。

採用された回答

Mathieu NOE
Mathieu NOE 2021 年 6 月 28 日
hello
this is my suggestion and the result : as you can see , the large amplitude spikes are not changed by the filtering (perfect overlay with raw data)
clc
clearvars
load('data.mat');
x = data(:,1);
y = data(:,2);
t = data(:,3);
dt = mean(diff(t));
Fs = 1/dt;
ind = find(t>3.9 & t < 4);
t = t(ind);
x = x(ind);
y = y(ind);
samples = length(y);
figure(1)
plot(t,x);
%% "smart" data smoothing
N = 9;
xs = medfilt1(x, N,'truncate');
xs = medfilt1(xs, N,'truncate');
% replace noisy data by smoothed (only low amplitude signal is affected)
ind = find(abs(xs)<1);
xx = x;
xx(ind) = xs(ind);
figure(1)
% plot(t,x,'b',t,xs,'r',t,xx,'g');legend('Raw','Smoothed');
plot(t,x,'b',t,xx,'r');legend('Raw','"smart" Smoothed');
title(['Data samples at Fs = ' num2str(round(Fs)) ' Hz / Smoothed with medfilt1' ]);
grid on
  2 件のコメント
Mathieu NOE
Mathieu NOE 2021 年 6 月 28 日
hello
even simpler code , only one stage of filtering , and noisy low amplitude signal simply replaced by zeros
clc
clearvars
load('data.mat');
x = data(:,1);
y = data(:,2);
t = data(:,3);
dt = mean(diff(t));
Fs = 1/dt;
ind = find(t>3.9 & t < 4);
t = t(ind);
x = x(ind);
y = y(ind);
samples = length(y);
figure(1)
plot(t,x);
%% "smart" data smoothing
N = 9;
xs = medfilt1(x, N,'truncate');
% replace noisy data by smoothed (only low amplitude signal is affected)
ind = find(abs(xs)>3);
xx = zeros(size(x));
xx(ind) = x(ind);
figure(1)
% plot(t,x,'b',t,xs,'r',t,xx,'g');legend('Raw','Smoothed');
plot(t,x,'b',t,xx,'r');legend('Raw','"smart" Smoothed');
title(['Data samples at Fs = ' num2str(round(Fs)) ' Hz / Smoothed with medfilt1' ]);
grid on
Mathieu NOE
Mathieu NOE 2021 年 7 月 8 日
hello again
does my suggestion answer your request ?
all the best

サインインしてコメントする。

その他の回答 (0 件)

カテゴリ

Help Center および File ExchangeSmoothing and Denoising についてさらに検索

製品


リリース

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by