XML Editor
Sign up for a WebBoard account Sign Up Keyword Search Search More Options... Options
Chat Rooms Chat Help Help News News Log in to WebBoard Log in Not Logged in
Conferences Close Tree View
+ Stylus Studio Feature Requests (1192)
- Stylus Studio Technical Forum (14621)
-> + Conerting Excel to XML (5)
-> + count the number of 1 appearin... (10)
-> + Import validation errors (2)
-> + Custom Validation Saxon (4)
-> + How Can i Check if else condit... (2)
-> + Stylus Studio 2009 Ent still b... (2)
-> + cannot download SS2009 (2)
-> + Use of import-schema wants sch... (4)
-> + Replacement Install Media/Bits (3)
-> + XML to Flat File (5)
-> + Select one of multiple attribu... (13)
-> + Web Service - SOAP Error (3)
-> + I find a BUG in stylus studio ... (8)
-> + Text to XML conversion issue (2)
-> + Fixed width flat files (3)
-> + Stylus Studio Crashes when val... (3)
-> + XSL Changes doesn't reflect in... (2)
-> + Custom Conversion Definition T... (6)
-> + Support for UK EDI Tradacoms ... (2)
-> + How to find and remove all ins... (2)
-> + XML to cobol Sequential file (2)
-> + Pipe delimited flat file conve... (2)
-> + Cannot Function Block > DataDi... (3)
-> + License Authentication failed (2)
-> + Flat File Conversion (3)
-> + Struzzo.exe sing 50% of CPU (2)
-> + Saxon9 Transformation error ha... (2)
-> + com.ddtek.xquery3.XQQueryExcep... (10)
-> + XML attribute name (2)
-> + JAVA and XML (2)
-> + DB Connection Error (3)
-> + Dicstionaty Librarys for other... (2)
-> + Passing parameters (3)
-> + Xmlconverter does not work (2)
-> + non-printing characters in Fil... (3)
-> + Including row nos while conve... (5)
-> + Getting Evaluation Copy except... (2)
-> + SS 2008 R2 Bug? (4)
-> + Java Heap Space (2)
-> + generate report from .xsl file... (3)
-> + Can't encode 0x4 in text (3)
-> + Recognize Japanese Characters (2)
-> + converting .xsl to xml using c... (2)
-> - Uninstall doesn't clean up reg... (1)
-> + CDISC (5)
-> - creating database from XML sch... (1)
-> + error: side-by-side configurat... (3)
-> - How to convert pdf into rdf fo... (1)
-> + Apache FOP (5)
-> + XPath Query Editor 'Buggy' (3)
-> - Feature request (1)
-> + XML Convertors (2)
-> + XMLConverters version 3.2.0.0 ... (2)
-> + JVM/stylus studio abort on sav... (3)
-> + using oasis catalog in pipelin... (6)
-> + Another whitespace issue with ... (2)
-> + saxon 9.1.0.2 (2)
-> + xslt change namespace (4)
-> - xslt change namespace (1)
-> + HL7 Conversion (3)
-> + mixed content mapping (2)
-> + HL7 to XML to HL7 (5)
-> + newest version of Saxon SA? (3)
-> + ORM / ORU Seqments (7)
-> + XSLT is causing Java.lang OutO... (3)
-> + Create POJOs from EDI (4)
-> - Embedded message (1)
-> + How to create Documentation fo... (2)
-> + xslt transformation (2)
-> - remove BOM (1)
-> + Changing data as it goes into... (5)
-> + How to (graphically) map one X... (2)
-> + Stylus Studio 2008 Release 2 -... (2)
-> + keyboard shortcuts to switch b... (2)
-> + Help with getting Output XML f... (8)
-> + Document created from schema (2)
-> + XSD Validation error (2)
-> + Set the value of an attribute ... (2)
-> + Saving as XML, Source Data Typ... (2)
-> + Own templates in file/new (4)
-> + Does Stylus Studio support the... (2)
-> - How to transform XML file to T... (1)
-> + Extract XML file from a HTML s... (2)
-> + Crashing, high cpu usage, larg... (6)
-> + Creating an xml document based... (2)
-> + Problem with restriction (2)
-> + EDI Invalid date time error (2)
-> + How to Compose a Web Service C... (5)
-> + Java Complie errors with XML C... (2)
-> + Creating xml file from CSV uss... (4)
-> + XML Converters (6)
-> + schema validation (15)
-> - 9/30 Customer Showcase Webinar... (1)
-> + ETL (2)
-> + xsd:sequence question (3)
-> + contains an invalid path (5)
-> - Custom converter to map flat f... (1)
-> + validating a 4.01 transitional... (2)
-> + Cannot Use Stylus Studio On Mu... (5)
-> + Missing DB to XML option (2)
-- Previous [901-920] [921-940] [941-960] Next
+ Website Feedback (249)
+ XSLT Help and Discussion (7625)
+ XQuery Help and Discussion (2016)
+ Stylus Studio FAQs (159)
+ Stylus Studio Code Samples & Utilities (364)
+ Stylus Studio Announcements (113)
Topic  
Postnext
Lee HumphriesSubject: Memory hogging
Author: Lee Humphries
Date: 24 Jul 2003 10:21 PM
Back to one of my old complaints (nag, nag, nag ...)

I've got a very simple XSLT that stuffs two XML files into a container XML.

I just tried running it using the Stylus internal XSLT processor (5.0 build 127b) and it consumed over a Gig of RAM (mostly virtual - real slow - thrash that disk) trying to join together a 27Meg file with a 4.5Meg file.

Now I know the DOM can consume one order of magnitude more RAM than the size of the file. But both of these files had simple structures (although neither had a schema defined), so the DOM shouldn't be that hungry.

So how does 27 + 4.5 = 1024+ ?

Regards, Lee

Postnext
Minollo I.Subject: Re: Memory hogging
Author: Minollo I.
Date: 24 Jul 2003 10:39 PM
It's surely not the DOM which is causing the memory consumption you are
seeing. More likely it's a matter of output handler and/or of large XPath
expressions/XSLT variables.

Would it be possible for you to send us a testcase, maybe working on a
smaller version of the XML documents?

Minollo

Postnext
Lee HumphriesSubject: Re: Memory hogging
Author: Lee Humphries
Date: 25 Jul 2003 12:25 AM
Here's the XSLT in question - as you can it's pretty tiny. The file that it includes and throws into a variable is the 4.5Meg file.

To test it just take any two XML files, change the file name in the document function to match one of the files and supply the other as the scenario test case.

Regards, Lee


ApplicationExportReprocess1.xsl
Tiny Excess El Tea

Postnext
Minollo I.Subject: Re: Memory hogging
Author: Minollo I.
Date: 25 Jul 2003 09:16 AM
Thanks Lee.
We'll look into this.

Minollo

Posttop
Lee HumphriesSubject: Re: Memory hogging
Author: Lee Humphries
Date: 27 Jul 2003 07:46 PM
Hi Minollo,

One reason why I assumed your implementation of the DOM may be involved is that we've experienced apparently similar issues with MSXML 4.0

We had been initially using the MSXML DOM when importing and processing some larger XML files (from 800kb to 60Mb) however we found it unbelievably slow, and it got exponentially slower (roughly) based on the number of elements in the XML file.

So we switched to using SAX.

Later though, I wanted to eliminate some of the handcoded validation we had done for our SAX implementation in favour of using an XSD, as the SAX based validation also seemed to be taking too long.

So I set up a DOM instance with the XSD applied against it first, then imported the XML. Now it runs a hell of a lot faster (about 3 orders of magnitude faster) and load time appears to be linear with respect to the number of elements, memory consumption is also drastically reduced.

That's why I was wondering whether your DOM implementation may be suffering with these large files not having any schemas defined against them.

Regards, Lee

   
Download A Free Trial of Stylus Studio 6 XML Professional Edition Today! Powered by Stylus Studio, the world's leading XML IDE for XML, XSLT, XQuery, XML Schema, DTD, XPath, WSDL, XHTML, SQL/XML, and XML Mapping!  
go

Log In Options

Site Map | Privacy Policy | Terms of Use | Trademarks
Stylus Scoop XML Newsletter:
W3C Member
Stylus Studio® and DataDirect XQuery ™are from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2016 All Rights Reserved.