Rounding using 'fix'... possibly a bug?
3 ビュー (過去 30 日間)
古いコメントを表示
Could someone tell me if this is a bug, or is there some logical reason 'fix' would round:
.3/.1 = 2
whereas
.4/.1 = 4
K>> fix(.3/.1)
ans =
2
K>> fix(.4/.1)
ans =
4
K>> fix(.7/.1)
ans =
6
K>> fix(.5/.1)
ans =
5
1 件のコメント
Jan
2013 年 10 月 14 日
It is such surprising that the well known floating point effects appear for beginners usually for the value 0.3 . Usually it is:
find((0:0.1:1) == 0.3) % replies [] !!!
採用された回答
James Tursa
2013 年 10 月 14 日
To see what is going on you can examine the exact floating point values. Note that none of your original numbers except 0.5 can be represented in IEEE double format exactly:
>> num2strexact(.7)
ans =
0.6999999999999999555910790149937383830547332763671875
>> num2strexact(.5)
ans =
0.5
>> num2strexact(.4)
ans =
0.40000000000000002220446049250313080847263336181640625
>> num2strexact(.3)
ans =
0.299999999999999988897769753748434595763683319091796875
>> num2strexact(.1)
ans =
0.1000000000000000055511151231257827021181583404541015625
>> num2strexact(.7/.1)
ans =
6.99999999999999911182158029987476766109466552734375
>> num2strexact(.5/.1)
ans =
5
>> num2strexact(.4/.1)
ans =
4
>> num2strexact(.3/.1)
ans =
2.999999999999999555910790149937383830547332763671875
You can find num2strexact here:
その他の回答 (1 件)
参考
カテゴリ
Help Center および File Exchange で Coordinate Transformations and Trajectories についてさらに検索
製品
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!