Wednesday, May 13, 2009

The Point at Which JRuby Loses Precision on Float

Was needing to check to see whether a number in JRuby was a round number or not, and planned to use this:
myfloat == myfloat.floor 
This appears to work:
irb(main):037:0> 2.000000000000001.to_f == 2.000000000000001.to_f.floor  
=> false
However, any higher level of precision and it breaks down without any warning:
irb(main):038:0> 2.0000000000000001.to_f == 2.0000000000000001.to_f.floor
=> true
Some more info is here including a proposed solution down in the comments:
Julian Burgess said, June 12, 2008 at 7:31 am Unless I’m dumb, isn’t string the answer? x = “1.00000000000000000000000000000000001″.to_d => # puts x.to_s(”F”) “1.00000000000000000000000000000000001″
(However, to_d isn't available in JRuby 1.2.0 it seems.)

No comments: