Smoothing Numerical Differentiation Result

I want to get the derivative of this S-shaped curve this way (x*(dy/dx)) which is expected to be like the normal distribution bell-shaped curve, I used x(2:end).*diff(y)./diff(x) , gradient function and central difference method. but the result was very noisy since it is a numerical differentiation. My question, is there a way to smooth the result to get a better derivative curve?

Answer 1

[Jim Riggs]

The attached file contains some higher-order methods for computing numerical derivatives. You can start with this. For very well behaved data, further smoothing might be achieved by curve fitting a function to the data and using the function derivative. If a more general method is desired, there are a number of ways to filter noisy data (for example, Matlab function "filter").

[Ahmed Zankoor]

It helped a little, I think the problem is my data. Thank you so much.

[Jim Riggs]

If you need to smooth noisy data, you should explore the "filter" function.

Here is another tip from my experience in data analysis: Many digital filters inserts a time lag into the filtered data. The more "filtering" that you apply, the greater the time lag. If data timing is important, and you are post-processing time series data (i.e. not running in real-time) you can filter the data in the forward time direction, then filter it again in reverse time order. The reverse filtering will remove the delay from the forward filter. Just keep in mind that you are also doubling the amount of filtering that you apply, so configure your filter accordingly. There may be some filter options in Matlab that do this automatically for you.

[Ahmed Zankoor]

The problem that I can not understand is that the data I want to find the derivative for is not that noisy yet I get a bad derivative, you can see the attached figures. So I do not think it needs filtering.

[Ahmed Zankoor]

I found the problem, the x variable is generated using normrnd (random variables following normal distribution) and the differences between the values vary greatly. for example dx=[.2 .01 ...] that is why when we compute the derivative its values show heavy noise.