[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Floating point error

Subject: Re: Floating point error
From: David Carlisle <davidc@xxxxxxxxx>
Date: Wed, 25 Jan 2006 17:42:34 GMT
floating point error
  <product>
    <quantity>3000</quantity>
    <price>6.4</price>
  </product>
 (the actual quantity and price may vary, but they're always
 valid numbers).

6.4 can't be stored exactly as a binary floating point number (for the
same reason that you can't store 1/3 as a finite-precision decimal) so
in principle there is always the potential for rounding error, but given
that you are just apparently looking for 4 or 5 significant figures of
accuracy I'd have expected you to get the exact answer here. (I get
19200 with saxon) I assume that you are using xalan and that xalan just
hands this over to the underlying JVM to do double arithmetic.

xslt2 (eg saxon8) offers decimal arithmetic which can then store 6.4
exactly.

not sure what to recommend as a workaround, you could try multiplying
all your prices by 100 (and dividing by 100 at the end) so that it all
looks like integer arithmetic which then in principle could be exact
(although XSLT1 always uses floating point arithmetic, even for
integers)

David



________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________

Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.