[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: basic math functions: problem with substraction re

Subject: Re: basic math functions: problem with substraction result (newbie question)
From: David Carlisle <davidc@xxxxxxxxx>
Date: Wed, 13 Aug 2003 10:54:45 +0100
xsl math function

> I would need the result of $var1 - $var2 to be = 2.2, as expected

you should _never_ expect exact results when doing numerical
calculations on a computer. A binary machine can not store 
either of your input numbers, 2.4 and 0.2 nor your requested result of
2.2 exactly, it can only store an approximation. (For the same reason
that a base 10 system as we usually use can not store one third exactly
You can only have 0.3333333333... to any finite precision that you want
but you can not store a finite base 10 representation of a third.

Similarly in base two arithmetic you can't represent one fifth exactly.

If you want to show the number rounded to fewer decimal places, then use
the format-number() function.

David

________________________________________________________________________
This e-mail has been scanned for all viruses by Star Internet. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________

 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.