[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: To improve XML file creation and parsing performance


xml file creation
Does the software inserting records into the new database perform batch 
inserts? Or does it insert one record at a time? This can greatly affect 
performance due to the number of INSERT statements being performed.

Note that writing software to perform batch inserts is difficult if that 
software is designed to handle deeply nested XML [1] as opposed to 
tabular XML [2]. The reason is that in deeply nested XML, the software 
must cache records of a similar type until it has enough records to 
perform a batch insert. If the XML is tabular, the software can simply 
insert records as it processes the file.

Therefore, if you're writing your own software, you might want to use 
tabular XML and perform batch inserts. If you're using somebody else's 
software and it handles deeply nested XML, chances are that it doesn't 
do batch inserts.

Another solution here is to use XSLT or custom software (including a 
custom, schema-specific parser) to convert the data to the bulk load 
syntax used by the database and use the bulk loading facilities of the 
database. I've heard of at least one company that experimented with this 
and got better performance.

-- Ron

[1] Deeply nested documents impose a hierarchy over the graph formed by 
the relational schema. For example, line item elements are nested inside 
their corresponding sales order elements.

[2] Tabular documents have the hierarchical form database / tables / 
table / row / column, using the obvious mapping from database to XML. 
For example, line item elements and sales order elements are stored in 
parallel at the same level of nesting.

Chiusano Joseph wrote:
> How large is your XML file? Also, the issue may not be with the XML file
> itself (or at least not all of it) - it may have to do with the target
> database (i.e. is it properly tuned, are the proper indexes present and
> active, etc.)
> 
> Kind Regards,
> Joseph Chiusano
> Booz Allen Hamilton
> O: 703-902-6923
> C: 202-251-0731
> Visit us online@ http://www.boozallen.com
>  
> 
> 
>>-----Original Message-----
>>From: Sube Singh [mailto:sube.verma@g...] 
>>Sent: Tuesday, August 16, 2005 2:55 AM
>>To: xml-dev@l...; alan-xml-dev@e...
>>Subject:  To improve XML file creation and parsing 
>>performance
>>
>>Hi,
>>
>>We would like to create a XML file from a database. After 
>>creation of XML file we will write it on client socket. After 
>>getting file at client end we will parse the XML file and 
>>insert datas in client database.There are 10 Lakhs records in 
>>database. It do this work XML take update 3 hours.
>>
>>Can any one suggest us how to improve the XML file parsing 
>>and creation performance.
>>
>>We hope that will get the solution of problem with help the 
>>help of your valueable suggestion.
>>
>>Regards,
>>
>>-Sube


PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
 

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.


Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.