[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: floor() and idiv
>>>>> "Andrew" == Andrew Welch <andrew.j.welch@xxxxxxxxx> writes: >> A bug in the spec or in Saxon ? >> Xmlizer> floor(1000000000000000000000000000000001 div Xmlizer> 100000000000000000000000000000001), Xmlizer> 1000000000000000000000000000000001 idiv Xmlizer> 100000000000000000000000000000001, Xmlizer> (1000000000000000000000000000000001 div Xmlizer> 100000000000000000000000000000001) cast as xs:integer >> Xmlizer> gives >> Xmlizer> 10, 9, 10 >> >> Saxon, I think. >> >> Gestalt gives 10, 10, 10. Andrew> So in Gestalt how do you determine when to use the Andrew> equivalent of BigDecimal in Java, or do you not need to Andrew> work about that sort of thing? Andrew> (In Java the result using standard division is Andrew> 9.999999999999998 which I guess just gets truncated to 9, Andrew> but using BigDecimal.divide() you get the correct 10) I always use the equivalent of BigDecimal if I cannot guarantee to avoid errors with INTEGER_64. I'm not sure how you can get a rounding error with integer or decimal arithmetic - those constants are integer constants, not doubles, in XPath 2.0. I believe that double arithmetic is non-compliant. Anyway, I am far from au fait with Java myself, so i am prepared to be surprised. -- Colin Adams Preston Lancashire
|
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|