SSDN - XPath Query Editor slow performance... newbie ignorance resolved!

XML Editor

Sign Up

Search

Options

Chat

Help

News

Log in

Not Logged in

Topic

Topic Page 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Jim Salmons

Subject: XPath Query Editor slow performance... newbie ignorance resolved!
Author: Jim Salmons
Date: 06 Sep 2007 10:38 AM
Originally Posted: 05 Sep 2007 01:32 PM

NUTSHELL PROBLEM: Incredibly slow performance of the XPath Query Editor

Document source is product data about tools, attachements, and accessories for a vendor's product line. A Stylus Studio-generated document schema is associated with the file prior to doing the XPath query.

Sample file: 1.4 MB
File lines: 29,837
Document Root: ITEMS
Document being a collection of: ITEM elements
Number of ITEM elements in sample file: 29
Number of ITEM/IMAGES/IMAGE/URL elements in sample file: 4,409
Target XPath Query: /ITEMS/ITEM/IMAGES/URL
Time to execute query: 6:18 min:sec
Time to open result in new XML document: 6:12 min:sec

Actual target document: 44.5 MB
File lines: 854,037
Document Root: ITEMS
Document being a collection of: ITEM elements
Number of ITEM elements in actual file: 3,151
Number of ITEM/IMAGES/IMAGE/URL elements in sample file: 81,573
Target XPath Query: /ITEMS/ITEM/IMAGES/URL
Time to execute query: UNK
Time to open result in new XML document: UNK

Background info on my hardware: XP Pro x32, 3.0+ GB RAM, 1.86 Ghz dual core Intel CPU

QUESTION #1: True/False - Opening an XPath Query Editor query result in a new XML document involves re-executing the query. If so, you might as well skip the Execute Query task and go right to opening the result in a new document to save half the time required to do your work.

If the answer to questions #1 is true, then the UI is unfortunately deceptive. The Execute green arrow and arrow-into-XML_document icon should be grouped in a drop-down icon list so the user can understand that these two tasks are comparable in execution 'load' and only differ in format and location in which the query result is displayed.

QUESTION #2: Will Stylus Studio _ever_ produce a result to my XPath query on the actual 44.5 MB source file (as described above)?

So far, I haven't had the patience/confidence to wait for a result. After well over 1.5 hours of monitoring the animation of the Processing dialog, I pressed Stop to convince myself that the process was not hung. (The CPU load throughout this ordeal was around 50-53% with the Struzzo.exe process using about 256 MB RAM.) The process wasn't hung, but the aborted query result was only a small portion of the final result expected. This makes we fear that, even if Stylus Studio will eventually produce a result, it will be _many_ hours before it completes. How useful is that?

QUESTION #3: I am a novice user of Stylus Studio. What, if anything, am I obviously missing that will optimize my basic use of this powerful tool?

Unless I can get useful levels of performance, I might as well go back to using REBOL to write and run scripts that generate comparable results in a fraction of the time it will take to do the same things in Stylus Studio. I am tending (hoping) that my performance problems are due to my lack of experience with this tool and XML-associated technologies.

Thanks to any and all who have the time and experience to help me get more out of my Stylus Studio use.

--Jim Salmons--
Iowa Tool Inc.

Minollo I.

Subject: XPath Query Editor slow performance
Author: Minollo I.
Date: 05 Sep 2007 01:41 PM

Just to be sure we are talking about the same thing, you are referring the the ability to run XPath expressions in the context of the XML editor, right?

That interface is not meant to be particularly scalable; it's mostly meant to run relatively simple tests. If you are running queries against large XML documents that can return large result sets, I would suggest doing that through the XQuery editor, which will leverage performance and scalability of the embedded DataDirect XQuery editor.

Jim Salmons

Subject: XPath Query Editor slow performance
Author: Jim Salmons
Date: 05 Sep 2007 02:02 PM

>Just to be sure we are talking
>about the same thing, you are
>referring the the ability to
>run XPath expressions in the
>context of the XML editor,
>right?
>

Yes, as a newbie, I am on a self-directed random walk that is somewhat directed by the pop-up prompts to watch streaming video tutorials and help resources.

>That interface is not meant to
>be particularly scalable; it's
>mostly meant to run relatively
>simple tests. If you are
>running queries against large
>XML documents that can return
>large result sets, I would
>suggest doing that through the
>XQuery editor, which will
>leverage performance and
>scalability of the embedded
>DataDirect XQuery editor.

Thanks for the tip. At this point, I have only a rudimentary sense of what to do where. I'll take your advice and report my learning/progress accordingly.

--Jim--

Jim Salmons

Subject: XPath Query Editor slow performance
Author: Jim Salmons
Date: 06 Sep 2007 10:53 AM

Thanks to Minollo's quick and helpful reply, I now know not to use the XPath Query Editor for any heavy-lifting!

Once I moved to a full XQuery document/editor and wrote a comparable but even more useful XQuery, I can get my sample document results in about 812 ms!!! :-) Instant stress relief. I do, however, need to tweak my Java heap space in order to be able to produce and report a result for it.

While my ignorance has been overcome in general, I still think that newbie's experience could be enhanced if it was more clear that the XPath Query Editor is a lightweight, supplemental tool. As we newbie's tend to be 'random walk experimenters', who knows how many potential sales could be lost to our avoidable mistakes such as mine.

Thanks again, Minollo!

--Jim--

Minollo I.

Subject: XPath Query Editor slow performance
Author: Minollo I.
Date: 06 Sep 2007 11:36 AM

Of course you are right; we will review the current design of that component as soon as we have a chance.

Jim Salmons

Subject: XPath Query Editor slow performance
Author: Jim Salmons
Date: 06 Sep 2007 12:38 PM

My final report on using an XQuery document/editor to do my task rather than the XPath Query Editor...

The full 44.5 MB source file was processed in 133.2 seconds, producing an output file of 8.5 MB of 91,029 lines with all the expected URL results.

>Of course you are right; we
>will review the current design
>of that component as soon as
>we have a chance.

Yes, the most helpful thing might be a dialog, triggered by a long process time that says, "Please be aware that the XPath Query Editor is a lightweight tool for quick explorations. Your query would be much better handled by using an XQuery document in the XQuery toolset. Click 'here' to go there now..." and SS opens a new XQuery document with the query inserted from the XPath Query Editor that caused the long execution.

Thanks again, Minollo.

Minollo I.

Subject: XPath Query Editor slow performance
Author: Minollo I.
Date: 06 Sep 2007 12:51 PM

Most likely the actual processing time is much less than that; when running XQuery in Stylus Studio, the time to get the result includes the compilation of the XQuery and the rendering of the result in the preview window, that are external to the actual processing time.

BTW, any chance you can post the XQuery you are running, so that we can see if there is any optimization that could significantly affect processing time? A sample of the input data would help too. Feel free to email stylus-field-report@progress.com if you prefer keeping that private.

Thanks.

Topic Page 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Powered by Stylus Studio, the world's leading XML IDE for XML, XSLT, XQuery, XML Schema, DTD, XPath, WSDL, XHTML, SQL/XML, and XML Mapping!

Go to Conference:

Log In Options Username: Password:

Site Map | Privacy Policy | Terms of Use | Trademarks

Stylus Studio® and DataDirect XQuery ™are from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2016 All Rights Reserved.