[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Out of memory error with XML and Java ResultSet


resultset memory
Hi Jason,

Do you know the typical size of the XML text generated from each row? 
Also, do you need to have the full document buffered in memory or do you 
really want it as a stream that's being sent somewhere else?

In general, you're going to want to avoid any approach (such as DOM) 
that builds a representation of the XML structure in memory. Even just 
buffering the output text in memory is probably going to cause problems 
if this gets longer than a few MB. Your best bet is probably just to 
stream it out, either directly to the destination or to a file on the 
local system.

  - Dennis

Brown, Jason B. wrote:

>Greetings.
>
>I am new to this mailing list.  I was wondering if someone could help me
>with a small problem I am having trying to create an XML buffer using
>data from a Java ResultSet.  The ResultSet object contains approximately
>300,000 rows, which I want to place in a XML buffer.  I managed to
>develop an XML implementation that uses JAXP 1.1.  The logic for the
>processing is this: I execute the SQL query, obtain the ResultSet, loop
>through the result set and as each row in the result set is encountered,
>I use getLong methods, etc to get the data and write it to a
>StreamResult object.
>
>Here is a code snippet:
>			rsMetaData = rs.getMetaData();
>
>			// Provide the System.out object as the output
>stream for now
>			xmlWrtr = new CisXmlWriterSAX(new
>StreamResult(xmlData), 
>							((String)
>CUtil.getEnv("AdhocDataDtd")));
>
>			numOfColumns = rsMetaData.getColumnCount();
>
>			//
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>			// Start XML document
>			xmlWrtr.startXMLDocument();
>
>			// Set root element
>			xmlWrtr.addXMLElement("AdhocClaims");
>
>			System.out.println("<<>> Building XML file...");
>
>			while (rs.next()) {
>				// Calculate number of rows in result
>set
>				numOfRows++;
>				
>				CDebug.print("" + numOfRows);
>
>				// Create the Row parent element (child
>element to AdhocClaims element)
>				xmlWrtr.addXMLElement("Row");
>
>				for (int colIndex = 1; colIndex <=
>numOfColumns; colIndex++) {
>
>				.....
>					tempStr =
>crossRef.getReadableName(rsMetaData.getColumnName(colIndex));
>
>					// Create the attributes for the
>element
>					xmlWrtr.addElementAttr("name",
>"CDATA", tempStr);
>
>					// Add Column element
>					xmlWrtr.addXMLElement("Column");
>
>					// Add the Value element and a
>value for the Value element
>					xmlWrtr.addElementValue(null,
>"");
>
>					// End Column element
>					xmlWrtr.endXMLElement("Column");
>				}
>				.....
>			}
>
>			xmlWrtr.endXMLElement("AdhocClaims");
>			
>			// This is done when finished with adding all
>elements within root element
>			xmlWrtr.endXMLDocument();
>			......
>
>xmlWrtr is a class that contains the following objects:
>	private SAXTransformerFactory transFact = null;
>
>	/**
>	* Contains where the resulting XML will be placed/contained
>	*/
>	private StreamResult resultStream = null;
>	
>	private TransformerHandler handler = null;
>
>	private Transformer serializer = null;
>
>	private AttributesImpl attr = null;
>
>My main goal with this posting is to obtain a memory-friendly solution
>for taking information from a Java ResultSet and placing it in a XML
>buffer.
>
>Thanks.
>
>Jason B. Brown
>CIS IS Claims
>215-241-4609
>x2-4609 (internal), 11th Floor
>mailto:jason.b.brown@i...
>
>
>
>CONFIDENTIALITY NOTICE: This E-Mail is intended only 
>for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you have received this communication in error, please do not distribute and delete the original message.  Please notify the sender by E-Mail at the address shown. Thank you for your compliance..
>
>
>-----------------------------------------------------------------
>The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
>initiative of OASIS <http://www.oasis-open.org>
>
>The list archives are at http://lists.xml.org/archives/xml-dev/
>
>To subscribe or unsubscribe from this list use the subscription
>manager: <http://lists.xml.org/ob/adm.pl>
>
>  
>


PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
 

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.


Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.