Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.1k views
in Technique[技术] by (71.8m points)

java - float number is not the expected number after subtraction

I have the following statement:

float diff = tempVal - m_constraint.getMinVal();

tempVal is declared as a float and the getMinVal() returns a float value.

I have the following print out:

diff=0.099999905, tempVal=5.1, m_constraint.getMinVal()=5.0

I expect the diff is 0.1 but not the above number. how to do that?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Floats use the IEEE754 to represent numbers, and that system has some rounding errors.

Floating point guide

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Wikipedia on IEE754

Bottom-line if you are doing arithmetic and it needs to be exact don't use float or double but us BigDecimal


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...