フィルターのクリア

Double Precision is rounding at the 5th digit?

4 ビュー (過去 30 日間)
Brian Rose
Brian Rose 2019 年 4 月 24 日
編集済み: Stephen23 2019 年 4 月 24 日
(Newbie question)
I am trying to do some basic math and I am running into what appears to be a rounding or precision problem. MATLAB uses doubles, which should be good out to 15 digits. It appears that MATLAB is rounding at less than 4. What am I missing? Note: I do not have access to the Symbolic toolkit, and frankly can't understand why any additional code would be needed for this basic equaiton.
value1 = double(0.999969482421875)
eps(value1)
value2 = double(32767) ./ double(32768)
Output is
value1 =
1.0000
ans =
1.1102e-16
value2 =
1.0000

採用された回答

Stephen23
Stephen23 2019 年 4 月 24 日
編集済み: Stephen23 2019 年 4 月 24 日
Change the display format., e.g.
format long g
Although beginners often confuse the two, how data is stored in memory and how data is displayed are really two totally different things.

その他の回答 (0 件)

カテゴリ

Help Center および File ExchangeNumeric Types についてさらに検索

製品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by