フィルターのクリア

calculation error or double precision limit?

1 回表示 (過去 30 日間)
Andreas
Andreas 2015 年 9 月 17 日
編集済み: Stephen23 2015 年 9 月 17 日
Hello,
I was wondering about the result when calculating following simple formula:
K>> 0.067 * 1e9
ans =
6.7000e+07
Following gives the correct result (integer):
K>> 0.067 * 1e8 * 1e1
ans =
67000000
Where is the difference in the calcualation?
Is it an error or a double precision limit?

回答 (1 件)

Stephen23
Stephen23 2015 年 9 月 17 日
編集済み: Stephen23 2015 年 9 月 17 日
You should download this and give it a try with the same examples as above:
Also have a look at the intermediate values, such as 0.067 * 1e8. This shows that there are a few digits that creep in after the end of the double precision limit (fifteen digits).

カテゴリ

Help Center および File ExchangeLogical についてさらに検索

製品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by