[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Processing huge XML files

• To: Thomas Lee <ytlee@c...>
• Subject: Re: Processing huge XML files
• From: Barrie Slaymaker <barries@s...>
• Date: Fri, 11 Oct 2002 14:27:14 -0400
• Cc: xml-dev <xml-dev@l...>
• In-reply-to: <3DA62CB4.3000408@c...>; from ytlee@c... on Fri, Oct 11, 2002 at 09:43:16AM +0800
• References: <3DA62CB4.3000408@c...>
• User-agent: Mutt/1.2i

On Fri, Oct 11, 2002 at 09:43:16AM +0800, Thomas Lee wrote:
> Hi all,
> 
> We need to process very large XML files (up to a few ten MB). So we 
> can't process the whole file with DOM in memory. We are prepared to use 
> SAX to parse a large XML file and store into our data structure in the 
> disk (propietary data structure or RDBMS). We'll also provide a set of 
> APIs for access the contents in the XML file. Now the XML content change 
> is not required.
> 
> However, instead of developing a customized solution, I'd like to know 
> if there is any existing tools and library handling large XML files in disk.

XML::Twig (in Perl)

    http://search.cpan.org/author/MIROD/XML-Twig-3.08/Twig.pm

is an ideal way of handling large files containing XML data structures
(as opposed to document oriented XML): you configure it to parse an
particular element (or set of elements) in to a data structure one at a
time, then handle those data structures.

- Barrie

• References:
  • Processing huge XML files
    • From: Thomas Lee <ytlee@c...>

• Prev by Date: Re: RE: evolvable formats
• Next by Date: RE: What is Tag Soup?
• Previous by thread: Re: Processing huge XML files
• Next by thread: RE: RE: evolvable formats
• Index(es):
  • Date
  • Thread