How to reduce precision
2 ビュー (過去 30 日間)
古いコメントを表示
Hello, I am using cross products to find an angle between two vectors (not using dot product because I am also using the cross product resultant vector) and in the limits, I get numbers such as e^-15 which I do not need. I need at most 4 digits after the point. Is there a code or simple way other than dividing with 0.001, rounding and mltiplying again with 0.001. Any help would be appreciated. F?rat
0 件のコメント
回答 (1 件)
Walter Roberson
2011 年 12 月 28 日
Multiplying by 10000, rounding, and dividing by 10000 is about the best possible in MATLAB.
Warning: Except in the cases that are integer multiples of exact powers of 2, such as 0.75 (3 * 2^(-2)), it is not possible in MATLAB to truncate floating point numbers to have exactly a certain number of decimal places. Floating point numbers are represented in binary, and none of 0.1 and 0.01 and 0.001 and 0.0001 have exact binary representations. You will end up with low-order bits set so as to come as close as possible to a multiple of 0.0001 in binary representation, but you will never get 0.0001 exactly.
Try
fprint('%.99g\n', 0.0001)
and you will see that the value is not exactly 0.0001
1 件のコメント
Walter Roberson
2011 年 12 月 28 日
Note: the Symbolic Toolbox can do decimal arithmetic, as can the Fixed Point Toolbox.
参考
カテゴリ
Help Center および File Exchange で Logical についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!