[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: Performance tips to speed up multiple transforms
On Fri, 2010-11-05 at 18:12 +0000, Neil Owens wrote: > Evening all > > I'm transforming around 500 individual XML log files - ~250MB worth - > into 4 o[...] I guess it's slow 'cos I'm shelling out to a hidden > command box every time I transform a log file(open a shell, start > java, open the Saxon .jar file, run the transform, append the 4 > results to the 4 'master' output files, close everything, repeat ). > > What would the smart person be looking at to speed up the process? > Write the whole script in Java? .NET? C++... Or just make a > Transform 'do' the whole thing? I'd probably consider a single transform. You might run into memory problems -- as I recall, there's a way to tell Saxon you've finished with a particular input document, which may help. (Longer term, the new XSLT 3 streaming support will help even more, I expect, but the design isn't final yet). An alternative for you might be four separate passes, one for each output file. I admit sometimes I use a perl or sed script (on Linux) to combine lots of small XML files, then check the result (e.g. against a DTD or schema, or just for well-formedness) and then use XSLT to split it up in some different way. Liam -- Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ Pictures from old books: http://fromoldbooks.org/ Ankh: irc.sorcery.net irc.gnome.org www.advogato.org
|
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|