[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Avoiding roundoff errors in calculations?

• To: "Roger L. Costello" <costello@m...>
• Subject: Re: Avoiding roundoff errors in calculations?
• From: "Chiusano Joseph" <chiusano_joseph@b...>
• Date: Tue, 02 Sep 2003 08:54:59 -0400
• Cc: xml-dev@l...
• Organization: Booz Allen Hamilton
• References: <3F4FA206.5E2367A2@m...>

<Quote>
I am considering returning the result in numerator/denominator form, and
leaving it up to the recipient to do the division (and decide what
precision is suitable), i.e.,

<Length>
    <value>
        <numerator>33.02</numerator>
        <denominator>600</denominator>
    </value>
    <unit rdf:resource="#Meter"/>
</Length>
</Quote>

Leaving the precision to the recipient may not be desirable,
particularly if the calculation is an "aggregate calculation" that is
carried out by multiple processes each on a different processor (or
different programming language), which may lead a less-than-accurate
precision in the final result.

So an alternative approach is to also specify the required precision for
the calculation result in the XML request:

<Length>
    <value precision="15">
        <numerator>33.02</numerator>
        <denominator>600</denominator>
    </value>
    <unit rdf:resource="#Meter"/>
</Length>

where "precision" is the number of decimal places for the result. If a
process cannot honor this precision, it could throw an exception.

Kind Regards,
Joe Chiusano
Booz | Allen | Hamilton

"Roger L. Costello" wrote:
> 
> Hi Folks,
> 
> Suppose that I would like to convert this Length to meters (there are 6
> picas per inch):
> 
> <Length>
>     <value>13</value>
>     <unit rdf:resource="#Pica"/>
> </Length>
> 
> Doing the arithmetic:
> 
> 13 picas * in/6 picas * 2.54 cm/in * m/100 cm
> 
>     = 33.02/600 m
> 
>     = 0.055033333333333333333333333333 m (the "3" repeats infinitely)
> 
> The problem is that everyone has different precision requirements.  Some
> people would be happy with this as the result:
> 
> <Length>
>     <value>0.055</value>
>     <unit rdf:resource="#Meter"/>
> </Length>
> 
> For other people such a roundoff could be catastrophic.
> 
> Even if I were to return a result that is as precise as my machine can
> produce,
> 
> <Length>
>     <value>0.055033333333333333333333333333</value>
>     <unit rdf:resource="#Meter"/>
> </Length>
> 
> there is still a roundoff that may be unacceptable to some.
> 
> I am considering returning the result in numerator/denominator form, and
> leaving it up to the recipient to do the division (and decide what
> precision is suitable), i.e.,
> 
> <Length>
>     <value>
>         <numerator>33.02</numerator>
>         <denominator>600</denominator>
>     </value>
>     <unit rdf:resource="#Meter"/>
> </Length>
> 
> What are your thoughts on this?  What techniques do people use to avoid
> introducing roundoff errors?  /Roger
> 
> -----------------------------------------------------------------
> The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
> initiative of OASIS <http://www.oasis-open.org>
> 
> The list archives are at http://lists.xml.org/archives/xml-dev/
> 
> To subscribe or unsubscribe from this list use the subscription
> manager: <http://lists.xml.org/ob/adm.pl>

begin:vcard 
n:Chiusano;Joseph
tel;work:(703) 902-6923
x-mozilla-html:FALSE
url:www.bah.com
org:Booz | Allen | Hamilton;IT Digital Strategies Team
adr:;;8283 Greensboro Drive;McLean;VA;22012;
version:2.1
email;internet:chiusano_joseph@b...
title:Senior Consultant
fn:Joseph M. Chiusano
end:vcard

• Follow-Ups:
  • Re: Avoiding roundoff errors in calculations?
    • From: Rick Marshall <rjm@z...>
  • Re: Avoiding roundoff errors in calculations?
    • From: "Chiusano Joseph" <chiusano_joseph@b...>

• Prev by Date: RE: file extensions best practice
• Next by Date: Re: file extensions best practice
• Previous by thread: file extensions best practice
• Next by thread: Re: Avoiding roundoff errors in calculations?
• Index(es):
  • Date
  • Thread