[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

RE: Processing huge XML files

• To: "'Thomas Lee'" <ytlee@c...>,"'xml-dev'" <xml-dev@l...>
• Subject: RE: Processing huge XML files
• From: "Michael Kay" <michael.h.kay@n...>
• Date: Fri, 11 Oct 2002 11:03:00 +0100
• Importance: Normal
• In-reply-to: <3DA62CB4.3000408@c...>
• Reply-to: <michael.h.kay@n...>

> 
> We need to process very large XML files (up to a few ten MB). So we 
> can't process the whole file with DOM in memory. We are 
> prepared to use 
> SAX to parse a large XML file and store into our data 
> structure in the 
> disk (propietary data structure or RDBMS). We'll also provide 
> a set of 
> APIs for access the contents in the XML file. Now the XML 
> content change 
> is not required.
> 
> However, instead of developing a customized solution, I'd 
> like to know 
> if there is any existing tools and library handling large XML 
> files in disk.
> 

Actually, a few tens of megabytes is not that huge: I've heard of people
using Saxon to transform 100Mb files successfully, though it's certainly
not easy.

But really, when you get above 50Mb or so, you need to start looking at
XML databases. And I'm not saying that only because my company sells one
[1].

Michael Kay
Software AG
home: Michael.H.Kay@n...
work: Michael.Kay@s... 

[1] http://www.softwareag.com/tamino/

• Follow-Ups:
  • Re: Processing huge XML files
    • From: "Rick Jelliffe" <ricko@a...>

• References:
  • Processing huge XML files
    • From: Thomas Lee <ytlee@c...>

• Prev by Date: Re: JITTs and DOM
• Next by Date: Re: JITTs and DOM
• Previous by thread: Re: Processing huge XML files
• Next by thread: Re: Processing huge XML files
• Index(es):
  • Date
  • Thread