[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Got a huge XML document?

  • From: Hermann Stamm-Wilbrandt <STAMMW@de.ibm.com>
  • To: "Costello, Roger L." <costello@mitre.org>
  • Date: Fri, 13 Sep 2013 16:01:06 +0200

Re:  Got a huge XML document?
Hello,

DataPower appliances provide an XSLT 1.0+EXSLT processor.
With new v6.0 firmware XQuery 1.0 and JSONiq processors are available.

The 9004 appliance models provide 4GB of main memory, the 9005 models 96GB.

DataPower provides XSLT streaming processing since more than 8 years.
Of course there are restrictions on streaming processing, see [1] and
slides 7-10 of [2], but no limits on what can be processed.

From posting [3], even 25GB XML files can be processed non-streaming:
...
If service is not streaming, the 500MB on 9004 devices should be good,
sometimes 1GB is possible, on 9005 XI52 I have processed a XML message of
size 25GB with a non-streaming service successfully.
...

[1]
http://pic.dhe.ibm.com/infocenter/wsdatap/v5r0m0/index.jsp?topic=%2Fcom.ibm.dp.xm.doc%2Foptimizingthroughstreaming.htm
[2] http://www-01.ibm.com/support/docview.wss?uid=swg27019118
[3]
https://www.ibm.com/developerworks/community/forums/html/topic?id=50c29113-9de3-4de1-b447-5caf4a390000#fbb7c129-5aeb-43a2-b139-3810adcee7ca


Mit besten Gruessen / Best wishes,

Hermann Stamm-Wilbrandt
Level 3 support for XML Compiler team and Fixpack team lead
WebSphere DataPower SOA Appliances
https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/
https://twitter.com/HermannSW/     http://www.stamm-wilbrandt.de/ce/
----------------------------------------------------------------------
IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschaeftsfuehrung: Dirk Wittkopp
Sitz der Gesellschaft: Boeblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294


|------------>
| From:      |
|------------>
  >--------------------------------------------------------------------------------------------------------------------------------------------------|
  |"Costello, Roger L." <costello@mitre.org>                                                                                                         |
  >--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| To:        |
|------------>
  >--------------------------------------------------------------------------------------------------------------------------------------------------|
  |"xml-dev@lists.xml.org" <xml-dev@lists.xml.org>,                                                                                                  |
  >--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Date:      |
|------------>
  >--------------------------------------------------------------------------------------------------------------------------------------------------|
  |09/12/2013 06:15 PM                                                                                                                               |
  >--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Subject:   |
|------------>
  >--------------------------------------------------------------------------------------------------------------------------------------------------|
  | Got a huge XML document?                                                                                                                |
  >--------------------------------------------------------------------------------------------------------------------------------------------------|





Hi Folks,

In the past it was difficult to process huge XML documents using XSLT
because the entire XML document had to fit in memory.

Now, with XSLT 3.0 it is possible to process XML documents of unlimited
size, using a technique called streaming.

I am putting together a tutorial on streaming and would like to show an
example of stream processing a huge XML document.

Got a huge XML document that you can share? The bigger, the better. A
gigabyte sized XML document would be awesome.

/Roger

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php






[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]


PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
 

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.


Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.