Subject:Java memory error Author:tom stoop Date:27 Jan 2005 04:31 AM
When I open a large .txt file using the "convert to XML using adapter" option I get the message
"File System adapter. Encountered an I/O error, java.lang.OutOfMemoryError: Java heap space"
This occours only with large textfiles. Smaller ones using the same .conv adapter works OK.
I assume I have to change some settings but I don't no where.
Can anybody help me ?
Subject:Java memory error Author:Tony Lavinio Date:27 Jan 2005 12:21 PM
Can you tell us which version of Stylus studio you are using?
Also, on that dialog, there is a button that says "Show Details".
Would you please grab a snapshot and attach it showing those details?
A similar problem was found in the past, but fixed in recent versions,
so we'd like to know if we have a regression, or you've found something
new, or if it can be solved just by updating.
Subject:Java memory error Author:tom stoop Date:28 Jan 2005 04:16 AM
I am using version 6 release 1 (for details see the attachment).
After your reply on my message I also did the "check for latest version" and noticed I was not using the latest version, so I downloaded that and did the update.
The result was that I got no java error anymore, but now Stylus just hangs, when converting a file using adapter. Even with small files.
I'm using Stylus just a few days now with an evaluation key to see if this is the right products for our needs.
Important for us is that we can convert large (up to 1 Gb) flat files to XML.
Subject:Java memory error Author:Tony Lavinio Date:28 Jan 2005 08:43 PM
We discovered a problem using the JDK 1.5, and it is fixed in
our next patch which is supposed to be coming out the first week
in February - it will be 6r2 build 287f.
The JDK 1.4 works fine.
The user interface for Convert-to-XML will only load up to the
first megabyte, since it's used for defining the map only.
However, the streaming engine that is called via the Java API should
handle documents of any size, since it only buffers one line at a time.
Subject:Java memory error Author:jason smith Date:04 Apr 2005 08:57 PM Originally Posted: 04 Apr 2005 08:34 PM
Tony is the java error due to not having set a big enough Xms for the desired task?<br>
The reason I ask is after I develop a simple XML to Flat file it works
fine for small files, but gives me problems with large ones. When I process
it with Saxon independently I get the same get outofmemory problems using Saxon.
I have tried to to increase the Xms amount, but it doesn't seem to help.
In this case would the stack be causing the problem?
The files file I am working with are 400megs in size.
Jason
Subject:Java memory error Author:Tony Lavinio Date:05 Apr 2005 09:38 AM
If you get the same error from Saxon, then the problem is with
the transform itself using too much memory. The author of Saxon
maintains a very active mailing list; perhaps asking him might
give some clues on how to reduce memory consumption.
We've done quite a bit of work with the Convert-to-XML and large
datasets, and it shouldn't be the culprit, since it never holds more
than one row in memory at a time. However, XSLT engines need to
hold the source and result documents simultaneously, as well as any
intermediate results, so that's probably the source of the troubles.
Is there any possibility of breaking down the incoming files? Or
rewriting the XSLT transforms? Usually messing with the -X settings
is a course of last resort, and often only provides temporary benefits.
Subject:Java memory error Author:Tony Lavinio Date:05 Apr 2005 09:42 AM
I just reread your message; somehow I missed it. 400 MEGS? Okay,
that's probably the problem. XSLT doesn't generally do too well
with files that large - no matter which engine you select.
I'd still try posting on the Saxon mailing list and asking there.
See http://sourceforge.net/mail/?group_id=29872 and choose the
saxon-help list (the announce list there isn't used anymore).