[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

RE: Unexpected result in sum

Subject: RE: Unexpected result in sum
From: "Michael Kay" <mike@xxxxxxxxxxxx>
Date: Sat, 10 Nov 2007 00:01:11 -0000
RE:  Unexpected result in sum
> Why does this not output 0.00?  When formatted, I end up with -$0.00.

The sum is not zero because of the rounding errors that occur with floating
point arithmetic. When values such as -1.11 are converted to binary floating
point, a small rounding error occurs. You can avoid this (if it's
significant) by using decimal arithmetic rather than floating point.
Alternatively, you can round off to two decimal places after doing the
summation.

There's a legitimate question about whether format-number() when applied to
a very small negative number should output 0.00 or -0.00. I assume that's
how you are doing the formatting. Saxon is doing what the spec says it
should do, but it seems less than ideal.

This is nothing to do with negative zero, by the way. The total is not
negative zero, but a small non-zero negative number, and the rules for
format-number say that a minus sign is used when the supplied value is
negative, even if the value after rounding is zero.

Michael Kay
http://www.saxonica.com/

> 
> I've read what I can find on negative zero, but can't see how 
> this node set and operation could produce it.
> 
> Using Saxon 8.9 processor (internal to Oxygen.)
> 
> Input:
> <?xml version="1.0" encoding="UTF-8"?>
> <a>
>     <b>1103.86</b>
>     <b>1829.30</b>
>     <b>-853.77</b>
>     <b>-243.17</b>
>     <b>-1.11</b>
>     <b>-1296.66</b>
>     <b>588.52</b>
>     <b>849.25</b>
>     <b>-1976.22</b>
> </a>
> 
> Output:
> <?xml version="1.0" encoding="UTF-8"?>
> <a>
>    <sum-b>-2.2737367544323206E-13</sum-b>
>    <sum-nodes>-2.2737367544323206E-13</sum-nodes>
> </a>
> 
> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
> version="2.0">
>     <xsl:output method="xml" indent="yes"/>
> 
>     <xsl:template match="/a">
>         <a>
>             <sum-b>
>                 <xsl:value-of select="sum(b)"/>
>             </sum-b>
> 
>             <xsl:variable name="nodes" select="b"/>
> 
>             <sum-nodes>
>                 <xsl:value-of select="sum($nodes)"/>
>             </sum-nodes>
>         </a>
>     </xsl:template>
> </xsl:stylesheet>
> 
> Thanks!
> Angela Williams
> Office: 512.344.1547 ~ Fax: 512.397.6656
> Angela.Williams@xxxxxxxxxxxxxxxxxx    

Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.