Help with optimizing the curve fitting function

I need to to fit a set of data to a function of the form y=a*x^b. I need to do this on the order of 50,000 times. the fit() function takes way too long. I have been trying to find alternatives, I found articles related to fitoptions, fittype, nlinfit, etc. However I wasn't able to understand how to work with them in the way that I need. if anyone can help me out with this I would Greatly appreciate it.

Thank You

[Birdman]

Have you tried Curve Fitting Toolbox? Because I recently tried out to fit two arrays size of 50000 and it quickly finds the coefficients a and b.

Answer 1

[John D'Errico]

The simple answer is to log your model. Then a call to polyfit will suffice, and polyfit is fast.

log(y) = log(a) + b*log(x)

So a first order model for polyfit. (Think about it.)

P1 = polyfit(log(x),log(y),1);
b = P1(1);
a = exp(P1(2));

Note that this can sometimes play hell with the error structure, but it may actually be a good thing, if y varies by an order of magnitude or more. Then some points in the fit will get far too much weight applied to them. So the log transformation turns it into a proportional noise problem.

There are other schemes one could use, such as the use of custom code that would employ a partitioned least squares solver, and a sparse jacobian matrix. But that will take a lot of time on your part to learn all you need.

Or, you could learn to use the parallel processing toolbox. Again, a lot of effort when simple use of polyfit might be entirely sufficient on the logged model.

[Betzalel Fialkoff]

Thank you, this is a brilliant and elegant solution, it reduced my run time to ~25seconds from ~30minutes. Lifesave!

[John D'Errico]

Yeah, but now you need to look as if you are doing something useful in those 29.5 minutes you just saved. ;-) BACK TO WORK FOR YOU!