Re: floor() and idiv

>>>>> "Andrew" == Andrew Welch <andrew.j.welch@xxxxxxxxx> writes:

    >> A bug in the spec or in Saxon ?
    >> 
    Xmlizer> floor(1000000000000000000000000000000001 div
    Xmlizer> 100000000000000000000000000000001),
    Xmlizer> 1000000000000000000000000000000001 idiv
    Xmlizer> 100000000000000000000000000000001,
    Xmlizer> (1000000000000000000000000000000001 div
    Xmlizer> 100000000000000000000000000000001) cast as xs:integer
    >> 
    Xmlizer> gives
    >> 
    Xmlizer> 10, 9, 10
    >> 
    >> Saxon, I think.
    >> 
    >> Gestalt gives 10, 10, 10.

    Andrew> So in Gestalt how do you determine when to use the
    Andrew> equivalent of BigDecimal in Java, or do you not need to
    Andrew> work about that sort of thing?

    Andrew> (In Java the result using standard division is
    Andrew> 9.999999999999998 which I guess just gets truncated to 9,
    Andrew> but using BigDecimal.divide() you get the correct 10)

I always use the equivalent of BigDecimal if I cannot guarantee to
avoid errors with INTEGER_64. I'm not sure how you can get a rounding
error with integer or decimal arithmetic - those constants are integer constants,
not doubles, in XPath 2.0. I believe that double arithmetic is
non-compliant. Anyway, I am far from au fait with Java myself, so i am
prepared to be surprised.
-- 
Colin Adams
Preston Lancashire