Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
478 views
in Technique[技术] by (71.8m points)

floating point - Python Decimal Module - Undesired Float-Like Output?

This I imagine is extremely simple - but why in the following are the two values for y not == 0? I thought the whole point of the decimal module was to get rid of the float dust ...The following is an extremely simplified version of a mathematical routine that passes numbers around as variables.

from decimal import * 
getcontext().prec = 2

q = Decimal(0.01)

x = Decimal(0.10) * Decimal(0.10)

y = Decimal(x) - Decimal(q)

print(x,y, Decimal(y))

'''
x== 0.010 
y== -2.1E-19 
Decimal(y) == -2.1E-19
'''
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Try specifying the numbers as strings

>>> Decimal('0.10') * Decimal('0.10') - Decimal('0.0100')
>>> Decimal('0.000')

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...