precision problem ? why ans for 10 ^ 5 * 0.0633 is 6.3299e+03

2 ビュー (過去 30 日間)
andy
andy 2021 年 2 月 26 日
編集済み: James Tursa 2021 年 2 月 26 日
anyway to fix this ?
>> 10 ^ 5 * 0.0633
ans =
6.329999999999999e+03
>> 10 ^ 6 * 0.0633
ans =
6.329999999999999e+04

回答 (2 件)

KALYAN ACHARJYA
KALYAN ACHARJYA 2021 年 2 月 26 日
編集済み: KALYAN ACHARJYA 2021 年 2 月 26 日
>> 10 ^ 5 * 0.0633
ans =
6.3300e+03
>> format shortG
>> 10 ^ 5 * 0.0633
ans =
6330
See the other Format options, you can set the different format the output display
  1 件のコメント
Steven Lord
Steven Lord 2021 年 2 月 26 日
Using a different display format masks the issue.
andy, can you post for me the exact decimal value of one divided by three? The only characters you're allowed to use to write that answer are the digits 0 through 9 and a single . for a decimal point. You're not allowed to say that the answer is anything repeating; you must write all the decimal places of your answer explicitly.
Now multiply the answer you gave me by three. The result will not be equal to one. If you had been able to write an infinite number of decimal places it would have been, but you didn't. You rounded off one divided by three and that roundoff error affected the answer.

サインインしてコメントする。


James Tursa
James Tursa 2021 年 2 月 26 日
編集済み: James Tursa 2021 年 2 月 26 日
Normal floating point arithmetic effects. See this link:
The root issue is that 0.0633 cannot be represented exactly in IEEE double floating point format, and downstream calculations using this will be affected as a result. Calculations that mathematically should result in a known value often do not. You need to write floating point code that is tolerant of these effects. E.g., this is the closest number to 0.0633 that IEEE double precision can represent exactly
>> num2strexact(0.0633,'fixed')
ans =
'0.0632999999999999951594276126343174837529659271240234375'

カテゴリ

Help Center および File ExchangeLogical についてさらに検索

タグ

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by