polyfit with time vector.

Hi, Im encountered some problems while detrending some data. I have data for example:

15:02:13 0.24
15:02:14 0.25
15:02:16 0.27

and I am trying to find the linear fit to this data to remove the drift.

But when using polyfit and then polyval I get NaN values in polyval and Warning: Polynomial is badly conditioned. Add points with distinct X values, reduce the degree of the polynomial, or try centering and scaling as described in HELP POLYFIT.

My code:

%[x y] = ginput(5) % get points with a click 5 points 

dxp =  [ 7.3633
    7.3633
    7.3633
    7.3633
    7.3633]*1.0e+05;

dyp =[0.1380
    0.1380
    0.1344
    0.1344
    0.1292];

degree = 1; % to be adjusted
[pol, S, mu] = polyfit(dxp, dyp, degree);
xp = polyval(pol, t1, [], mu);
dI1 = I1 - xp; % detrended phase

The time vector is the the form above with 10^5 something. I think this is causing the problem. The question is how I can find the slope of the curve to remove the drift .. and plot the detrended curve? I don't know how to solve this. Any suggestions?

Answer 1

[John D'Errico]

You think you have 5 points.

In reality, you have ONE point, with 5 replicates.

dxp =  [ 7.3633
  7.3633
  7.3633
  7.3633
  7.3633]*1.0e+05;

ONE point. One single value for x.

How many points determine a line? TWO points, at least. With only one point, no line is possible. EVER. PERIOD.

Yes, I know, but datenum gave you those numbers!!!!

datenum('15:02:13 0.2')
ans =
   7.3633e+05
datenum('15:02:14 0.25')
ans =
   7.3633e+05

Did it give you those numbers? REALLY?

format long g
datenum('15:02:13 0.2')
ans =
          736330.626539352
datenum('15:02:14 0.25')
ans =
          736330.626550926

They ARE different. You just chose to supply the wrong numbers when you used polyfit.

You needed to save the EXACT results of datenum in a vector.

By the way, if you tried to use a higher order polyfit than 1 (linear), polyfit will start to fail here again. But that is a completely different story.

Garbage in, garbage out.

[thestargazer]

But this time vector is at least 10 min long, Then why I'm I getting the same values?

Im using

[x y] = ginput(3)

to get the points.

[thestargazer]

I did a solution around it. I picked certain times in my figure and I used datenum('15:50:00','HH:MM:SS') as x-values to the values I got with input at those times. It doesn't complain. I do not know why I couldn't take the 7.3633e+05 values directly from the plot, but maybe it doesn't give the last digits to separate the values. Actually, yes I think that was the problem. I should have used different format. Ok my bad. Problem solved!

[John D'Errico]

Again, you need to use a scheme that gives you sufficient precision to distinguish the numbers as different.

[Steven Lord]

I would avoid using datenum in this case.

start = datetime('now');
d = randi([1 180], 1, 10);
sampledata = start + cumsum([0, seconds(d)]);
timeBetweenPoints = minutes(sampledata-start)

You wouldn't know what d is in your problem; what you would know is start (the earliest time in your time vector; use min to compute it if your time vector is not sorted) and sampledata (the whole time vector.) Compute timeBetweenPoints (number of minutes since the start -- if your data has a different granularity you may want to use seconds, hours, days, etc.)

Alternately, you could subtract off the mean or median time instead of the min and convert that result to your desired units to use in polyfit.