[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: java heap management for transform

Subject: Re: java heap management for transform
From: Michael Kay <mike@xxxxxxxxxxxx>
Date: Fri, 26 Nov 2010 16:59:54 +0000
Re:  java heap management for transform
If you are processing the source documents one at a time, try saxon:discard-document()

http://saxonica.com/documentation/extensions/functions/discarddocument.xml

Michael Kay
Saxonica

On 26/11/2010 16:37, ac wrote:
Hi,

I have an xslt2 stylesheet that reads a little over 700 xml files and combines them, after some processing, into a single xml file. The source files average a little under 100KB each, with the transformation, with an expected output a little over 70MB. The source files are ordered alphabetically and somewhere between H and I, I get a java heap out-of-memory exception, with Saxon 9.1, however I change the java memory settings (initially I had 128 and 1024). The system has 4GB RAM and The task Manager reports that around 1.8GB is used during the transform (whatever the xms and xmx settings). The command line script is listed below. Does any one have any suggestion. Any help would be appreciated.

java -Dxms2048m -Dxmx2048m -Dfile.encoding=utf8 -cp .;\dev\jboss-3.2.5\server\default\deploy\xml.war\WEB-INF\lib\xercesImpl.jar;\dev\jboss-3.2.5\server\default\deploy\xml.war\WEB-INF\lib\saxon.jar;\dev\jboss-3.2.5\server\default\deploy\xml.war\WEB-INF\lib\dom4j.jar;\dev\jboss-3.2.5\server\default\deploy\xml.war\WEB-INF\lib\dnaos.jar;\dev\jboss-3.2.5\server\default\deploy\xml.war\WEB-INF\lib\sxutil.jar -Djavax.xml.parsers.DocumentBuilderFactory=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl -Djavax.xml.parsers.SAXParserFactory=org.apache.xerces.jaxp.SAXParserFactoryImpl net.sf.saxon.Transform -warnings:recover -o:cosmos.html -s:build\%1.xml -xsl:dnaos.xslt %2 %3 %4 %5 %6 %7 %8 %9

The reported exception follows:

Regards
ac


Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at net.sf.saxon.om.FastStringBuffer.ensureCapacity(FastStringBuffer.java:413)
at net.sf.saxon.om.FastStringBuffer.append(FastStringBuffer.java:104)
at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValue(TinyParentNodeImpl.java:87)
at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValueCS(TinyParentNodeImpl.java:39)
at net.sf.saxon.tinytree.TinyNodeImpl.getTypedValue(TinyNodeImpl.java:89)
at net.sf.saxon.expr.Atomizer.evaluateItem(Atomizer.java:178)
at net.sf.saxon.expr.UntypedAtomicConverter.evaluateItem(UntypedAtomicConverter.java:173)
at net.sf.saxon.functions.NormalizeSpace.effectiveBooleanValue(NormalizeSpace.java:110)
at net.sf.saxon.expr.FilterIterator$NonNumeric.matches(FilterIterator.java:182)
at net.sf.saxon.expr.FilterIterator.getNextMatchingItem(FilterIterator.java:65)
at net.sf.saxon.expr.FilterIterator.next(FilterIterator.java:44)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:321)
at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:210)
at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:174)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Instruction.process(Instruction.java:93)
at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:296)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Instruction.process(Instruction.java:93)
at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:296)
at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:557)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345)
at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:210)
at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:174)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Instruction.process(Instruction.java:93)
at net.sf.saxon.instruct.ForEach.processLeavingTail(ForEach.java:321)
at net.sf.saxon.instruct.Choose.processLeavingTail(Choose.java:686)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203)

Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.