[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: XSLT 2.0 Decimal number silliness
Jeff Kenton <jkenton@xxxxxxxxxxxxx> writes: > I've been looking at XSLT 2.0 decimal numbers (again), and I still > think the WC needs to do better. > [...] > <xsl:value-of select="2.0 div 3.0" />~ > <xsl:value-of select="0.444 div 0.666" />~ > [...] > Both lines of output should be equal. Why? The majority of implementations will use a binary format such as IEC 60559 to hold floating point numbers. 2.0 and 3.0 can be converted exactly but the representations of 0.444 and 0.666 will be inexact (0.44400000000000001 and 0.66600000000000004 in my test). You might get a better match with 0.5 div 0.75. If you want fixed point number semantics try moving the decimal points to the right before dividing. Floating point arithmetic is inexact by definition. Your results _are_ the same to within the precision specified by the language and/or implementation. -- Pete Forman -./\.- Disclaimer: This post is originated WesternGeco -./\.- by myself and does not represent pete.forman@xxxxxxxxxxxxxxx -./\.- opinion of Schlumberger, Baker http://petef.port5.com -./\.- Hughes or their divisions.
|
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|