|
[XSL-LIST Mailing List Archive Home]
[By Thread]
[By Date]
[Recent Entries]
[Reply To This Message]
Re: XSL SUM BUG?
Subject: Re: XSL SUM BUG?
From: "J.Pietschmann" <j3322ptm@xxxxxxxx>
Date: Thu, 19 Feb 2004 21:19:46 +0100
|
Andrew wrote:
I've following xml:
<aa>
<plan SUMM="700000.1"/>
</aa>
execute sum(//plan/@SUMM) and you'll get 700000.09999...- what the ...?
but if you change 7 on 1 or 2 all works as expected- 1 preserved.
tested on .NET/xmlspy.
Computers usually don't represent floating point numbers the same
way humas do. Internally, a binary representation is used, which
can't excatly represent fractions which aren't a power of 1/2.
Therefore if you convert 0.1 into the internal representation and
back to decimal, you may experience roundoff errors.
The common workaround is to use the format() function to tell
the processor to round to a specific precision.
J.Pietschmann
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
|
PURCHASE STYLUS STUDIO ONLINE TODAY!
Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!
Download The World's Best XML IDE!
Accelerate XML development with our award-winning XML IDE - Download a free trial today!
Subscribe in XML format
| RSS 2.0 |
|
| Atom 0.3 |
|
|
Cart
