Tool development: by Perl-wrapped XQuery

[David A. Lee]

xmlstar looks very useful !!!

Taking a quick look though it seems it will suffer exactly the same 
performance problems as most other scripting languages because it has no 
way to "glue" the commands together in process.  So you end up having a 
process execution for each command and full text serialization in and out.
( also missing the one key thing this list is about aka "xquery" ).


David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Jakob Fix wrote:
> Talking of tools, what is the experience of people on this list with
> xmlstarlet (http://xmlstar.sourceforge.net/) which seems to me like a
> close contender to xmlsh (but haven't looked closely)?
>
> cheers,
> Jakob.
>
>
>
> On Fri, Sep 4, 2009 at 13:09, David A. Lee<http://x-query.com/mailman/listinfo/talk> wrote:
>   
>>
>>     
>>> Now my questions: - would you recommend alternatives for rapid tool
>>> development? - if taking a similar approach - would you like to recommend
>>> special details, perhaps the script language actually used, or other aspect?
>>>
>>>       
>> In my opinion the problem with doing this in perl is that unless the xquery
>> implementation itself is in perl or atleast runs within the same process you
>> will run into horrible performance problems.   See my (with Norm Walsh's)
>> paper :
>>
>> http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html
>>
>> What we found is that for our test cases there is a 100-200x (yes 10000 % -
>> 20000 % ) performance penalty of using a scripting language to call xml
>> processing programs.   This *can* be optimized but the exact use cases of
>> using a off-the-shelf scripting language to do this kind of thing is
>> typically by the audience of people who do not want to spend the extra
>> effort to optimize it, or who are not experts in the type of software
>> development/languages required to  do it, or both. i.e its exactly why they
>> are using scripting - so they don't have to do all that extra work.
>>
>> This is the primary reason xmlsh was invented instead of re-using an
>> existing scripting language.   I took a "toy" program in a scripting
>> language it worked great.
>> But when I loaded up all the files I needed it to run it died a horrible
>> death.   This is what I call "The Brick Wall" and why scripting XML
>> processes fail so many of us.   The presentation cited above has some good
>> charts and figures as well as the full test case code.
>>
>> This is why I suggest either (both)
>>
>> * Use a scripting language that already is 'in process' with all the XML
>> core languages you want to use (xquery, xslt etc)
>> -> examples XProc, xmlsh
>>
>> * Encourage scripting languages developers to embed these XML languages
>> directly into the scripting languages (say perl).
>> -> This is hard work and may in fact involve re-implementing many of the
>> core tools from scratch.
>> -> Some of the work is done but is incomplete ... I've seen references to
>> XSLT implementions native in perl where the author quoted  something like
>>  "This isnt a complete implementation of XSLT 1.0 but it works pretty good
>> for me".
>>
>>
>> --
>> David A. Lee
>> http://x-query.com/mailman/listinfo/talk  http://www.calldei.com
>> http://www.xmlsh.org
>> 812-482-5224
>>
>>
>>
>> _______________________________________________
>> http://x-query.com/mailman/listinfo/talk
>> http://x-query.com/mailman/listinfo/talk
>>
>>     
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090904/c0df45fc/attachment.htm