Double Precision is rounding at the 5th digit?
3 ビュー (過去 30 日間)
古いコメントを表示
(Newbie question)
I am trying to do some basic math and I am running into what appears to be a rounding or precision problem. MATLAB uses doubles, which should be good out to 15 digits. It appears that MATLAB is rounding at less than 4. What am I missing? Note: I do not have access to the Symbolic toolkit, and frankly can't understand why any additional code would be needed for this basic equaiton.
value1 = double(0.999969482421875)
eps(value1)
value2 = double(32767) ./ double(32768)
Output is
value1 =
1.0000
ans =
1.1102e-16
value2 =
1.0000
0 件のコメント
採用された回答
その他の回答 (0 件)
参考
製品
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!