XML Editor
Sign up for a WebBoard account Sign Up Keyword Search Search More Options... Options
search
Chat Rooms Chat Help Help News News Log in to WebBoard Log in Not Logged in
Home » Boards » Stylus Studio Developer Network » XSLT Help and Discussion » big input file for XSLT
Conferences Close Tree View
+ Stylus Studio Feature Requests (1192)
+ Stylus Studio Technical Forum (14621)
+ Website Feedback (249)
- XSLT Help and Discussion (7625)
-> + How to add page number in to v... (2)
-> + XSL Grouping (7)
-> + Repeat images using FOR EACH l... (7)
-> + How to get File name (2)
-> + image and text on the same lin... (2)
-> + Create XSLT stylesheets for us... (2)
-> + Merge Multiple Files While Che... (2)
-> + xslt 3 level or multi level gr... (6)
-> + How to create xsl which will c... (2)
-> + xsl:variable and Result Tree F... (7)
-> - XSL-FO fo:table continued from... (1)
-> + XSLT Java issue (2)
-> + problem with hyperlink (4)
-> + substring-after function probl... (4)
-> + Xalan Processing (3)
-> + Memory utilization high with m... (2)
-> + check all parent node groups s... (2)
-> + CSV DOC to XML CONVERSION (11)
-> + XSLT: Ordered List (based on r... (4)
-> + XSLT Mapping - Issue (4)
-> + XSL following-sibling, how to ... (5)
-> + need xslt elements help (2)
-> + Dynamic generation of URL in X... (2)
-> + List Created from Section Navi... (2)
-> + xslt elements help (2)
-> + Map field if source is not nul... (3)
-> + XSLT group and sort whith gene... (4)
-> + Nested XML value (2)
-> + Specifying condition with coun... (4)
-> + xsl:function using found in er... (4)
-> + How to remove an unwanted spac... (2)
-> + Question about variable substi... (2)
-> + XSL for Comparing nodes and di... (4)
-> + External lookup list (8)
-> + XSL HTML output now needs to b... (2)
-> + Adding a new sub element to th... (4)
-> + How do I get rid of 'Tip' node... (9)
-> + Trying to dynamically create n... (2)
-> + Display XML link, using XSLT a... (9)
-> + calling templates based on ord... (2)
-> + How to add internal links usin... (4)
-> + How to call user defined java ... (3)
-> + XSLT and WebApps past and futu... (3)
-> + passing condition through java... (3)
-> + Apllying a template from withi... (4)
-> + Getting xsl element in javascr... (3)
-> + The document() function inside... (3)
-> + Footnote text move to para (2)
-> + XSL: multiple element come to ... (3)
-> + How to use following-sibling &... (2)
-> + How to use FOP 0.95 with Stylu... (4)
-> + Error during creating PDF from... (2)
-> + Help creating a comma delimite... (4)
-> + Selective Add using XSLT recur... (3)
-> + Help: convert without parent e... (2)
-> - Correct my substring function ... (1)
-> + Old xsl into reports (4)
-> + How to save xslt converted fil... (3)
-> + How to find non tagging text? (5)
-> + Help: Hex entity to Character... (6)
-> + EDI QUOTES MESSAGE NEED TO REP... (2)
-> + How to convert in multi level ... (4)
-> + Need some help with a styleshe... (3)
-> - Conditional Formattinf Formula... (1)
-> + Creating HTML Unordered LIsts.... (2)
-> + xslt params uknown number of i... (2)
-> + How to use fonts other than ba... (2)
-> - Converting existing HTML into ... (1)
-> + Help to tune up this XSLT (2)
-> + XSLT - is only converting the ... (2)
-> + XSLT Editor tab (4)
-> + Convert multiple files from XM... (2)
-> + how to use in select expressi... (2)
-> + Help needed in XSL transformat... (2)
-> + How to add a string with math ... (5)
-> + How to add a string with math ... (6)
-> - br element (1)
-> + Stylesheet directed terminatio... (2)
-> + Who can help me whit sum funct... (2)
-> + XSLT Works in Stylus Studio, n... (3)
-> - Filter by Attribute (1)
-> + Display x number of items whic... (6)
-> + MultiColumn Reporting (3)
-> + cant get namespace declaration... (3)
-> + Performance xsl:for each and n... (3)
-> + How to assign value to a XSLT ... (3)
-> + Select element value based on ... (2)
-> + Version 10 of Saxon? (8)
-> + How to get the value of a node... (5)
-> + Display Xpath using (4)
-> + XSLT to report (2)
-> + xslt help for a noobie (2)
-> + XSL-FO XML to PDF with XSLT (3)
-> + Using XSLT to transform XML to... (4)
-> + XSLT to sort a list alphabetic... (4)
-> + How to Get Colum Values based ... (2)
-> + non persisting predicate filte... (5)
-> + HTML to XSLT (2)
-> + extra carriage returns in "csv... (2)
-> + extra carriage returns in "csv... (3)
-- Previous [301-320] [321-340] [341-360] Next
+ XQuery Help and Discussion (2017)
+ Stylus Studio FAQs (159)
+ Stylus Studio Code Samples & Utilities (364)
+ Stylus Studio Announcements (113)
Topic  
Postnext
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 06 Feb 2009 03:39 PM
I need to run XSLT with a big input file about 60M. It looks Stylus Studio runs forever. I think it should be the memory problem. However, after I add -Xmx512M into Parameters of Java VM from Options menu, it reports error, like "can't find out INI something" and then Stylus Studio becomes crazy so that I can't open it properly. I also try to run XSLT from command line, but no lucky. Any advices? Really appreciate.
Postnext
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 06 Feb 2009 03:47 PM
Additional information:
My input file size is 69M.
The maximum size of Allocated stack size of XSLT I can set in Stylus Studio is 64M.
Is it the problem?
How to enlarge the maximum setting for Allocated Stack Size?
Postnext
Minollo I.Subject: big input file for XSLT
Author: Minollo I.
Date: 06 Feb 2009 05:34 PM
You are likely just running into memory troubles; I assume you are using the (default) Saxon XSLT processor, right? (XSLT > Scenario settings).

You can increase your heap, but you may want to be careful because if you set it to a value that cannot be achieved on your machine, the JVM will fail to initialize. The stack size is not a problem if you are running Saxon (which is what we would suggest doing anyway).

You may want to consider using XQuery; the embedded DataDirect XQuery is a highly scalable engine.
Postnext
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 08 Feb 2009 10:59 PM
Originally Posted: 08 Feb 2009 10:43 PM
Thank you for your relpy.

Yes, you are correct. I am running XSLT by Saxon XSLT processor. I add -Xmx256m to parameters of JVM, but it still doesn't work. I couldn't add more value, like -Xmx512m, because just like what you said, the JVM failed to initialize.

Question 1: if I use command line to run XSLT, like StylusXslt, how can I set the heap value for it?

Question 2: are there any other way to improve the memory of XSLT processor?

Question 3: I am not well with xQuery sibling. Could you please give me some suggestions about my sample if I use xQuery to replace XLST code?
My input XML structure likes this:
<ROOT>
<STATMENT>
.......
</STATEMENT>
<ACCOUNT>
...
</ACCOUNT>
<TRANSACTION>
....
</TRANSACTION>
<TRANSACTION>
....
</TRANSACTION>
<ACCOUNT>
....
</ACCOUNT>
<TRANSACTION>
....
</TRANSACTION>
</ROOT>

I want to get the output XML like this:
<ROOT>
<STATMENT>
.......
<ACCOUNT>
...
<TRANSACTION>
....
</TRANSACTION>
<TRANSACTION>
....
</TRANSACTION>
</ACCOUNT>
<ACCOUNT>
...
<TRANSACTION>
....
</TRANSACTION>
</ACCOUNT>
</STATEMENT>
</ROOT>

How to use xQuery child syntax to structure this XML?

Thanks in advance!
Postnext
Minollo I.Subject: big input file for XSLT
Author: Minollo I.
Date: 08 Feb 2009 11:06 PM
In the XSLT Scenario properties you can choose which processor to use to "process" your stylesheet; Saxon is the default processor used by Stylus Studio; "built-in" is a simpler XSLT processor built-in Stylus Studio and good mostly for debugging (step-by-step execution). You are apparently using Saxon, so, no need to worry there.

Your specific problem is a typical positional grouping issue; some additional details here: http://www.xml-connection.com/2008/02/convertig-proprietary-file-formats-to.html

This XQuery may be a start:
declare function local:getRelatedAccounts($item) {
let $nextItem :=$item/following-sibling::*[local-name()!="ACCOUNT" and local-name()!="TRANSACTION"][1]
for $related in $item/following-sibling::*[local-name()="ACCOUNT"]
where if ($nextItem) then $related << $nextItem else true()
return $related
};
declare function local:getRelatedTransactions($item) {
let $nextItem := $item/following-sibling::*[local-name()!="TRANSACTION"][1]
for $related in $item/following-sibling::*[local-name()="TRANSACTION"]
where if($nextItem) then $related << $nextItem else true()
return $related
};
<ROOT> {
for $statement in /ROOT/STATEMENT
return
<STATEMENT> {
for $account in local:getRelatedAccounts($statement)
return
<ACCOUNT name="{$account/string()}"> {
for $transaction in local:getRelatedTransactions($account)
return
<TRANSACTION> {
$transaction/string()
} </TRANSACTION>
} </ACCOUNT>
} </STATEMENT>
} </ROOT>

Postnext
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 09 Feb 2009 12:27 PM
Originally Posted: 09 Feb 2009 12:14 PM
Thank you for your great help. Two more questions:
Question 1: could you please give more explaination about "where if ($nextItem)then $related<<$nextItem else true()". I am confused about this condition.

Question 2: if there is a statement trailer record that should be put in STATEMENT, but after ACCOUNT. How to code it?
I add the following code, but the result is not correct.

declare function local:getRelatedStatTrailer($item) {
let $nextItem :=$item/following-sibling::*[local-name()!="STATEMENTTRAILER"][1]
for $related in $item/following-sibling::*[local-name()="STATEMENTTRAILER"]
where if ($nextItem) then $related << $nextItem else true()
return $related
};

.....

<STATMENTS>
{
<STATEMENT>
{
<ACCOUNT>
</ACCOUNT>
{
for $stattrailer in local:getRelatedStatTrailer($statement)
return
<TOTAL_RECORDS>{$stattrailer/TOTAL-RECORDS/text()}</TOTAL_RECORDS>

}
}</STATEMENT>
}</STATEMENTS>

If there are two statements, the first statement returns both the first trailer and the second trailer; the second statement is ok.

And also there are more than one element in $stattrailer, but if I don't want to add <Trailer></Trailer>, I only can add one emelement into the code. If I add more, it will report error. For example, I also want to add <TOTAL_ACCOUNT>{$stattrailer/TOTAL-ACCOUNT/text()}</TOTAL_ACCOUNT> under <TOTAL_RECORD> directly, how to do it?

Thanks again!
Postnext
Minollo I.Subject: big input file for XSLT
Author: Minollo I.
Date: 09 Feb 2009 12:43 PM
>Question 1: could you please give more explaination about "where if
>($nextItem)then $related<<$nextItem else true()". I am confused about
>this condition.

It's filtering the "related" items to fetch only the ones that come *before* the next container element, or all of them if there are no more container elements.


>Question 2: if there is a statement trailer record that should be put >in STATEMENT, but after ACCOUNT. How to code it?

I think you want to change your function into:
declare function local:getRelatedStatTrailer($item) {
let $nextItem :=$item/following-sibling::*[local-name()="STATEMENT"][1]
for $related in $item/following-sibling::*[local-name()="STATEMENTTRAILER"]
where if ($nextItem) then $related << $nextItem else true()
return $related
};

$nextItem needs to somehow identify the next group; as you don't have sub-groups, in your case it's probably easier to identify the next group just finding the next "STATEMENT".

The code should look something similar to this:
</ROOT>
for $statement in $input/STATEMENT
return
<STATEMENT> {
for $account in local:getRelatedAccounts($statement)
return
<ACCOUNT name="{$account/string()}"> {
for $transaction in local:getRelatedTransactions($account)
return
<TRANSACTION> {
$transaction/string()
} </TRANSACTION>
} </ACCOUNT>,
for $stattrailer in local:getRelatedStatTrailer($statement)
return
<TOTAL_RECORDS>{$stattrailer/TOTAL-RECORDS/text()}</TOTAL_RECORDS>
} </STATEMENT>
} </ROOT>
Postnext
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 09 Feb 2009 01:45 PM
I use your old example to do it, but it doesn't work. I attached the xml input file and xquery file. Could you please take time to look at them?
Also, I want to add
<total_30>{$record11/total_30/text()}</total_30>
under <total_20></total_20>.
I use
{
for $record11 in local:getRelated11($record_10)
return
<total_20>{$record11/total_20/text()}</total_20>
<total_30>{$record11/total_30/text()}</total_30>
}
but it reports error.

It seems I have to use:
<trailer>
<total_20>{$record11/total_20/text()}</total_20>
<total_30>{$record11/total_30/text()}</total_30>
</tailer>
but I don't want to <trailer></trailer> in the output file. Any advices?


UnknownproprietaryFile_trailer.xquery


UnknownproprietaryInputFile_trailer.xml
Postnext
Minollo I.Subject: big input file for XSLT
Author: Minollo I.
Date: 09 Feb 2009 02:04 PM
Instead of me trying to guess what you are trying to do, and you trying to map my suggestions on your real usecase, why don't you post a sample/cleaned-up example of your incoming XML and the XML you would like to get as a result? That would speed up things.
Postnext
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 09 Feb 2009 02:31 PM
My case is really like the case in your topic http://www.xml-connection.com/2008/02/convertig-proprietary-file-formats-to.html but the "trailer" part. I updated your sample files and attached them in my previous message. After updaing, it is exact same as my case. Could you please take a look my attached file? This is the easiest way for me because the raw data file is very complex and it is hard for me to create a sample data file for you. Thanks a lot.

By the way, could you please tell me where I can get the syntax for xQuery function, like following-sibling, <<,... and so on.

Postnext
Minollo I.Subject: big input file for XSLT
Author: Minollo I.
Date: 09 Feb 2009 04:15 PM
A very small mistake; change != into = when getting the nextItem:
declare function local:getRelated11($item) {
let $nextItem := $item/following-sibling::*[local-name()="record_10"][1]
for $related in $item/following-sibling::*[local-name()="record_11"]
where if($nextItem) then $related << $nextItem else true()
return $related
};

You'll find all the details you are looking for (and more) in the XQuery (http://www.w3.org/TR/xquery/) and XPath (http://www.w3.org/TR/xpath20) specifications.

Postnext
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 09 Feb 2009 08:18 PM
I really appreciate your great help!
Could you please take a look my another question?

I use
{
for $record11 in local:getRelated11($record_10)
return
<total_20>{$record11/total_20/text()}</total_20>
<total_30>{$record11/total_30/text()}</total_30>
}
but it reports error.

It seems I have to use:
<trailer>
<total_20>{$record11/total_20/text()}</total_20>
<total_30>{$record11/total_30/text()}</total_30>
</tailer>
but I don't want to have <trailer></trailer> in the output file. Any advices?
Postnext
Minollo I.Subject: big input file for XSLT
Author: Minollo I.
Date: 09 Feb 2009 08:43 PM
XQuery syntax:

{
for $record11 in local:getRelated11($record_10)
return (
<total_20>{$record11/total_20/text()}</total_20>,
<total_30>{$record11/total_30/text()}</total_30>
)
}
Posttop
Fairy LeeSubject: big input file for XSLT
Author: Fairy Lee
Date: 10 Feb 2009 10:45 AM
Originally Posted: 10 Feb 2009 10:43 AM
Thank you for your patience and great help. I really appreciate.

Have a good day,
Fairy
   
Download A Free Trial of Stylus Studio 6 XML Professional Edition Today! Powered by Stylus Studio, the world's leading XML IDE for XML, XSLT, XQuery, XML Schema, DTD, XPath, WSDL, XHTML, SQL/XML, and XML Mapping!  
go

Log In Options

Site Map | Privacy Policy | Terms of Use | Trademarks
Stylus Scoop XML Newsletter:
W3C Member
Stylus Studio® and DataDirect XQuery ™are from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2016 All Rights Reserved.