[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Free Tool for Efficient XML Data Compression

• From: Eric Bohlman <ebohlman@n...>
• To: Elliotte Rusty Harold <elharo@m...>
• Date: Sat, 18 Dec 1999 19:01:41 -0800 (PST)

On Sat, 18 Dec 1999, Elliotte Rusty Harold wrote:

> While XMill sounds interesting, I really have to take issue with this 
> statement.  I've seen no evidence that XML based, uncompressed file 
> formats are larger than the corresponding binary file formats. This 
> is a common fear about XML but I have not seen it borne out in my 
> tests. For instance, my 700K, very verbose baseball statistics 
> example is more than two megabytes in both FileMaker 3 and Microsoft 
> Excel.

Slightly playing devil's advocate here, I rather strongly suspect that
what you're seeing is the size advantage of a variable-length-field data
format over a fixed-length-field data format; your database files probably
have 1.3+ megabytes of blank padding.  What does the comparison look like
if you export the files to CSV?

While I do believe there's a lot of FUD running around regarding the size
of XML files, and that shouldn't be the primary selling point of something
like XMill, it does appear that the XMill developers are on to something:
XML allows specifying the structure of data, and XMill appears to be able
to use that structural information to get better compression than would be
available by treating the data as an unstructured blob of text.  It's very
often the case that the average "entropy" of the contents of each "field"
of a set of "records" is much lower than the "entropy" of the set as a
whole, and thus the sum of the gains from compressing each "column"
individually can be greater than the gain from compressing the entire set
as if it were homogeneous.



xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev@i...
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1
To unsubscribe, mailto:majordomo@i... the following message;
unsubscribe xml-dev
To subscribe to the digests, mailto:majordomo@i... the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa@i...)

• References:
  • Re: Free Tool for Efficient XML Data Compression
    • From: Elliotte Rusty Harold <elharo@m...>

• Prev by Date: Re: SAX2: Namespace proposal
• Next by Date: Re: SAX2: Namespace proposal
• Previous by thread: Re: Free Tool for Efficient XML Data Compression
• Next by thread: RE: Free Tool for Efficient XML Data Compression
• Index(es):
  • Date
  • Thread