[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Big XML file

• To: Elliotte Rusty Harold <elharo@metalab.unc.edu>
• Subject: Re: [xml-dev] Big XML file
• From: Aleksander Slominski <aslom@cs.indiana.edu>
• Date: Thu, 10 Jul 2003 23:31:46 -0400
• Cc: "Miguel A. Robles" <marobles.sag@mail.sgae.es>, xml-dev@lists.xml.org
• In-reply-to: <p04330101bb331cc212f7@[192.168.254.4]>
• References: <019701c33ca7$03299e50$b4fb850a@adagio.sgae.es> <p04330101bb331cc212f7@[192.168.254.4]>
• User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2) Gecko/20030624

Elliotte Rusty Harold wrote:

> At 2:24 PM +0200 6/27/03, Miguel A. Robles wrote:
>
> Dear colleagues,
>
> I usually work with XML for sending on information between
> different servers or applications. For example, currently I'm
> working with web services and everithing works fine. The
> problem appears now, because I have to parse a document
> containing a lot of information. DOM is not enough
> to accomplish the object because the document
> is extremly large, and I don't know how SAX deals with this
> kind of files.
>
> I know that XML is not intended for containing so much
> information, but I have to think about a possible solution.
>
> What do you think I could act to fix the problem?
>
>
> You could use XOM: http://www.cafeconleche.org/XOM/
>
> The latest version provides a streaming, tree-based approach that 
> allows you to work with pieces of the tree in sequence and then 
> discard them. 

hi,

i believe that this approach was first attempted in PullDOM in Python 
and in Java it was my XPP2 where i made it completely transparent if 
tree is built dynamically and still gives additional control to the user 
if needed:
http://www.extreme.indiana.edu/xgws/xsoap/xpp/download/PullParser2/doc/api/org/gjt/xpp/XmlPullNode.html

> For many record-like documents this is much more convenient than SAX 
> while still using only slightly more memory than the underlying SAX 
> parser. Indeed, it can process arbitrarily large documents with 
> effectively constant memory usage. And unlike SAX the document is 
> fully read-write. Unlike some similar approaches this doesn't require 
> you to learn XPath to preidentify the nodes of interest. 

i believe that XPath access is sometimes much more convenient and there 
is nothing that prevents to have both. interesting way to proceed would 
be to implement subset of XPath that can execute directly over event 
stream or over such incremental tree t minimize memory usage.

> There are limits. This won't help if you really do need to have the 
> entire document in memory at once, or if you need to move about 
> randomly in the tree. However, if you can work with a peephole view of 
> the document stream, then XOM gives you a larger peephole than SAX. It 
> can show you whole elements of your choice rather than individual 
> tokens. I've found this approach to be shockingly useful for many 
> applications. 

i agree but sometimes you may want to bypass tree creation and access 
underlying XML event stream directly and that is also possible in XPP2.

> In many cases elements are the right granularity for processing a 
> document. DOM's document level view is too large. SAX's token level 
> view is too small. XOM's element level view is just right. :-)

XPP3 has refined XML tree model that is now called XB1 (Xml pull Builder 
version 1) and has much better ease of use than XPP2 (that was my first 
attempt ...) as it is modeled directly after XML Infoset.

So you could use too XPP2 or XPP3 
http://www.extreme.indiana.edu/xgws/xsoap/xpp/

thanks,

alek

-- 
If everything seems under control, you're just not going fast enough. —Mario Andretti

• References:
  • Re: [xml-dev] Big XML file
    • From: Elliotte Rusty Harold <elharo@metalab.unc.edu>

• Prev by Date: recently received books
• Next by Date: XSLT way
• Previous by thread: Re: [xml-dev] Big XML file
• Next by thread: Re: [xml-dev] Big XML file
• Index(es):
  • Date
  • Thread