SSDN - Java memory error

XML Editor

Sign Up

Search

Options

Chat

Help

News

Log in

Not Logged in

Topic

Topic Page 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

tom stoop

Subject: Java memory error
Author: tom stoop
Date: 27 Jan 2005 04:31 AM

When I open a large .txt file using the "convert to XML using adapter" option I get the message
"File System adapter. Encountered an I/O error, java.lang.OutOfMemoryError: Java heap space"
This occours only with large textfiles. Smaller ones using the same .conv adapter works OK.
I assume I have to change some settings but I don't no where.
Can anybody help me ?

Thanks in advance.

Tony Lavinio

Subject: Java memory error
Author: Tony Lavinio
Date: 27 Jan 2005 12:21 PM

Can you tell us which version of Stylus studio you are using?

Also, on that dialog, there is a button that says "Show Details".
Would you please grab a snapshot and attach it showing those details?

A similar problem was found in the past, but fixed in recent versions,
so we'd like to know if we have a regression, or you've found something
new, or if it can be solved just by updating.

tom stoop

Subject: Java memory error
Author: tom stoop
Date: 28 Jan 2005 04:16 AM

I am using version 6 release 1 (for details see the attachment).
After your reply on my message I also did the "check for latest version" and noticed I was not using the latest version, so I downloaded that and did the update.
The result was that I got no java error anymore, but now Stylus just hangs, when converting a file using adapter. Even with small files.

I'm using Stylus just a few days now with an evaluation key to see if this is the right products for our needs.
Important for us is that we can convert large (up to 1 Gb) flat files to XML.

I hope you can help soving this problem.

Tom stoop

tom stoop

Subject: Java memory error
Author: tom stoop
Date: 28 Jan 2005 04:16 AM

Sorry, forgot the attachment

Tom

Stylus_version.doc

Tony Lavinio

Subject: Java memory error
Author: Tony Lavinio
Date: 28 Jan 2005 08:43 PM

We discovered a problem using the JDK 1.5, and it is fixed in
our next patch which is supposed to be coming out the first week
in February - it will be 6r2 build 287f.

The JDK 1.4 works fine.

The user interface for Convert-to-XML will only load up to the
first megabyte, since it's used for defining the map only.

However, the streaming engine that is called via the Java API should
handle documents of any size, since it only buffers one line at a time.

See http://www.stylusstudio.com/docs/v62/d_flatfileconversion11.html
for information on using any of the adapters outside of the Stylus
Studio environment.

jason smith

Subject: Java memory error
Author: jason smith
Date: 04 Apr 2005 08:57 PM
Originally Posted: 04 Apr 2005 08:34 PM

Tony is the java error due to not having set a big enough Xms for the desired task?<br> The reason I ask is after I develop a simple XML to Flat file it works fine for small files, but gives me problems with large ones. When I process it with Saxon independently I get the same get outofmemory problems using Saxon. I have tried to to increase the Xms amount, but it doesn't seem to help. In this case would the stack be causing the problem? The files file I am working with are 400megs in size. Jason

Tony Lavinio

Subject: Java memory error
Author: Tony Lavinio
Date: 05 Apr 2005 09:38 AM

If you get the same error from Saxon, then the problem is with
the transform itself using too much memory. The author of Saxon
maintains a very active mailing list; perhaps asking him might
give some clues on how to reduce memory consumption.

We've done quite a bit of work with the Convert-to-XML and large
datasets, and it shouldn't be the culprit, since it never holds more
than one row in memory at a time. However, XSLT engines need to
hold the source and result documents simultaneously, as well as any
intermediate results, so that's probably the source of the troubles.

Is there any possibility of breaking down the incoming files? Or
rewriting the XSLT transforms? Usually messing with the -X settings
is a course of last resort, and often only provides temporary benefits.

Tony Lavinio

Subject: Java memory error
Author: Tony Lavinio
Date: 05 Apr 2005 09:42 AM

I just reread your message; somehow I missed it. 400 MEGS? Okay,
that's probably the problem. XSLT doesn't generally do too well
with files that large - no matter which engine you select.

I'd still try posting on the Saxon mailing list and asking there.
See http://sourceforge.net/mail/?group_id=29872 and choose the
saxon-help list (the announce list there isn't used anymore).

jason smith

Subject: Java memory error
Author: jason smith
Date: 05 Apr 2005 04:37 PM

Thanks Tony,

Yeah I decided to write a script that divides the XML into manageable chunks of
around 100megs. This seems to work ok.

Jason Smith
benhadad_smith@hotmail.com

P.S. Is that an old photo or do you never age?

Topic Page 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Powered by Stylus Studio, the world's leading XML IDE for XML, XSLT, XQuery, XML Schema, DTD, XPath, WSDL, XHTML, SQL/XML, and XML Mapping!

Go to Conference:

Log In Options Username: Password:

Site Map | Privacy Policy | Terms of Use | Trademarks

Stylus Studio® and DataDirect XQuery ™are from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2016 All Rights Reserved.