RE: Large transforms (was Re: GByte Transforms)

> There are people who have 100 megabyte product catalogs in 
> XML files.  They 
> obviously don't serve the whole thing to a web page, but they 
> do use XSLT to 
> process those catalogs, and produce small web pages or XML 
> output for reports.
> 
> So, there are several questions you can ask here.  First, 
> what does it take 
> for an XSLT processor to handle XML too big to live in memory 
> all at once? [ 
> That was Kevin Jones's question. ]  Second, what other 
> techniques will help 
> beat the problem? XML databases? What else?

I certainly feel that if you have this much data, then in most cases it is
worth investing a little effort into storing it in a way that makes
retrieval easier and faster. Which is to say, putting it in a database.

There might be some exceptions, such as streams of data coming down from a
satellite; and I suppose you might want to transform the data on its way
into the database. But I haven't come across such applications myself.

Michael Kay