[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: floor() and idiv

Subject: Re: floor() and idiv
From: Colin Paul Adams <colin@xxxxxxxxxxxxxxxxxx>
Date: Thu, 06 Nov 2008 19:50:02 +0000
Re:  floor() and idiv
>>>>> "Andrew" == Andrew Welch <andrew.j.welch@xxxxxxxxx> writes:

    >> A bug in the spec or in Saxon ?
    >> 
    Xmlizer> floor(1000000000000000000000000000000001 div
    Xmlizer> 100000000000000000000000000000001),
    Xmlizer> 1000000000000000000000000000000001 idiv
    Xmlizer> 100000000000000000000000000000001,
    Xmlizer> (1000000000000000000000000000000001 div
    Xmlizer> 100000000000000000000000000000001) cast as xs:integer
    >> 
    Xmlizer> gives
    >> 
    Xmlizer> 10, 9, 10
    >> 
    >> Saxon, I think.
    >> 
    >> Gestalt gives 10, 10, 10.

    Andrew> So in Gestalt how do you determine when to use the
    Andrew> equivalent of BigDecimal in Java, or do you not need to
    Andrew> work about that sort of thing?

    Andrew> (In Java the result using standard division is
    Andrew> 9.999999999999998 which I guess just gets truncated to 9,
    Andrew> but using BigDecimal.divide() you get the correct 10)

I always use the equivalent of BigDecimal if I cannot guarantee to
avoid errors with INTEGER_64. I'm not sure how you can get a rounding
error with integer or decimal arithmetic - those constants are integer constants,
not doubles, in XPath 2.0. I believe that double arithmetic is
non-compliant. Anyway, I am far from au fait with Java myself, so i am
prepared to be surprised.
-- 
Colin Adams
Preston Lancashire

Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.