[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: msxml sum() and precision problem

Subject: Re: msxml sum() and precision problem
From: David Carlisle <davidc@xxxxxxxxx>
Date: Thu, 19 Sep 2002 13:35:06 +0100
msxml float precision
> and the stunning browser-result:

I haven't checked exactly what IEEE arithmetic should produce in that
case, but why do you find it surprising? 1/100 can not be exactly
expressed in binary floating point (just as 1/3 can not be expressed in
finite length base 10 decimals). Since none of the three numbers you are
adding up may be expressed exactly, and neither can the sum, it isn't
surprising that any binary system used to encode the addition is out in
the least significant position.

David

_____________________________________________________________________
This message has been checked for all known viruses by Star Internet
delivered through the MessageLabs Virus Scanning Service. For further
information visit http://www.star.net.uk/stats.asp or alternatively call
Star Internet for details on the Virus Scanning Service.

 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.