Neural Network (NARX) performance interpretation - how small is small?

3 ビュー (過去 30 日間)
Charles
Charles 2017 年 8 月 18 日
回答済み: Greg Heath 2017 年 8 月 19 日
I have built a neural network to forecast currency pair prices (trend) a few steps ahead. How do I interpret the performance plot results? For instance, the error histogram, the NMSE (what is considered small for currency pairs? How do I assess if these metrics are optimal?

回答 (1 件)

Greg Heath
Greg Heath 2017 年 8 月 19 日
If you naively model a function with a straight line, the MSE is just the variance of the function and varies with the scale of the function.
In previous MATLAB posts I have used the notations
MSE00 = vart1 = mean(var(target'),1)
and use it as a standard for measuring MSE.
Using this NAIVE model as a comparative standard, the normalized MSE is defined by
NMSE = MSE/mean(var(target',1))
which is related to the statistical RSQUARE via
RSQUARE = 1 - NMSE
For reasonable models both NMSE and RSQUARE lie within the closed interval [ 0,1].
For most regression and classification models, I use a goal
NMSEgoal = 0.01 % Regression & Classification
However, in order to obtain that goal for CLOSED LOOP time series, I use the OPEN LOOP goals
NMSEgoal = 0.005 or 0.001
I don't know of anyone who complains when Rsquare >= 0.99 or equivalently, NMSE <= 0.01.
Hope this helps.
Thank you for formally accepting my answer
Greg

カテゴリ

Find more on Deep Learning with Time Series and Sequence Data in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by