Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
712 views
in Technique[技术] by (71.8m points)

floating point - Java double precision sum trouble

I would like to know why I get this error. (this is Display log of Eclipse debug)

var
     (double) 2.8
tot.getIva()
     (java.lang.Double) 0.17
var+tot.get()
     (double) 2.9699999999999998

I can not understand why I did not get simply 2.97!

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

If you wanted 2.97, you should have used BigDecimal.

doubles are stored as fractions in binary, not decimal. So 3.75, for example, is just stored as 2^1 + 2^0 + 2^(-1) + 2^(-2).

2.8 and 0.17 cannot be represented exactly as binary fractions, so there's going to be some rounding error.

You may also find this article helpful.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...