rounding down using floor

so im using:
floor(10^decimal*(total/AVERAGE_COUNT))/10^decimal;
where:
decimal = 4, AVERAGE_COUNT = 9, total = 8.9289
this entry gives me 0.9920, however the answer should be 0.9921
furthermore, when I just do
total/AVERAGE_COUNT;
it gives me 0.9921 which is correct but i need to use the floor function for the rest of my data.
how do i get around this

3 件のコメント

Walter Roberson
Walter Roberson 2020 年 9 月 1 日
Which MATLAB release are you using, and which operating system?
I get 0.9921 on R2020a on Mac
marcus peixoto
marcus peixoto 2020 年 9 月 1 日
I'm on R2020a on Windows
per isakson
per isakson 2020 年 9 月 1 日
On my R2018b, Win10 this script
%%
format long
decimal = 4; AVERAGE_COUNT = 9; total = 8.9289;
x = floor(10^decimal*(total/AVERAGE_COUNT))/10^decimal
outputs
x =
0.992100000000000
>>

サインインしてコメントする。

回答 (1 件)

Dana
Dana 2020 年 9 月 1 日

0 投票

I'm on R2020a on Windows 10 and I also get 0.9921.
Is it possible that total does not equal exactly 8.9289? For example, if total actually equals 8.92889 (which would usually display as 8.9289 in the command window), then you'll end up with 0.9920.

カテゴリ

質問済み:

2020 年 9 月 1 日

回答済み:

2020 年 9 月 1 日

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by