XQuery Talk Mailing List

[Announce] Poll for hands-on XML training offerings in Europe Q1'2010 (XQuery Talk)

2009-11-19T10:05:37Z

Please ignore the following if you have no interest in XML-related training.

In Q1'2010 I will be in Europe a couple of times for some standards 
meetings (OASIS and ISO) and conferences ( http://www.XMLPrague.cz 
and PEPPOL) and I like to take advantage of my presence there by 
offering Crane's hands-on training classes in the week before or 
after each of the events, open to public subscription.

I'll be in Copenhagen in February and in Prague and Stockholm in March.

But it is a gamble to decide on which training class to offer and 
find no-one is interested in that particular topic, because of hotel 
and event obligations that need to be paid in advance or reserved 
with the risk of forfeit.  We realize not many people in these 
economic times have money for training, so we are now trying to 
target our offerings specifically where we learn there is demand and 
availability of students.

Last year's http://www.XMLPrague.cz conference was great and we did 
our training adjacent to the conference and our students were 
conference attendees.  The venue we have chosen again in Prague next 
year has room for only 6 students (a large 14th century hotel room), 
so if XML training at this time of year and in conjunction with the 
conference is of interest to you, it would be important to sign up 
your interest soon.

To address this urgency to start making plans, I am polling the 
various XML lists for any interest in training in these cities early 
next year.  If you are interested in hands-on training on any of 
XSLT, XQuery, XSL-FO, the Universal Business Language or code lists 
in XML, and you would be interested in attending, please let us know 
and we will figure out what we can do.  To see the proposed dates, 
please visit:

   http://www.CraneSoftwrights.com/forms/interest.php
   (the page has links to syllabi)

On the form, please indicate all of the offered dates that you would 
be interested in, and we'll make a decision based on the response 
level we get for each class and date.

Thanks!  Please direct any questions to me off-list.

. . . . . . . . . Ken

p.s. it hasn't been decided but I might be in Perth Australia in 
January or February, so you won't see it on the form, but if you are 
in that area, please let me know in a mail message if you are 
interested in training on any of our subjects and we'll see about 
setting up a publicly-subscribed class.

cc: XML-Dev, XSL List, XQuery Talk, XML-Doc, UBL Dev, CLR Dev, 
Antenna House list, RenderX list, MarkLogic list

--
Vote for your XML training:   http://www.CraneSoftwrights.com/q/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/q/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:http://x-query.com/mailman/listinfo/talk
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/q/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

XQuery in Web 2.0 Expo in NY

2009-11-10T17:04:13Z

Dear all,

there is a BOF session organized in the Web 2.0 Expo in New York
next week about the role of XQuery in the architecture for
new Web .20 applications.

If you are interested, please join.

Information bellow:


  Nov 16-19  at the Javits center for the Web 2.0 expo and conference :
             There is an XML BOF (Birds of a Feather) at the Westin  
Times Square (270 West 43rd Street) on Tuesday evening at 8pm in room  
Nederlander - see
                   http://www.web2expo.com/webexny2009/public/schedule/stype/BoF
                  http://www.web2expo.com/webexny2009/public/schedule/detail/11318


There is a free registration for this Web 2.0 event - see:
                   http://www.web2expo.com/webexny2009/public/content/whats-free 
    - EXPOPASS



Best regards
Dana

XQuery count()

2009-11-09T20:09:39Z

Hi,

> I need to count the total number of distinct collections in my db (eXist XML
> Database).
> How can i do this in a compact way using XQuery?

This won't be possible without using extension functions. Different
systems will have different ideas of what constitutes a "collection"
(which can be queried with fn:collection). In eXist, you could use a
recursive approach:

import module namespace xdb="http://exist-db.org/xquery/xmldb";

declare function local:scan-collection($collection) {
  $collection,
  for $child in xdb:get-child-collections($collection) return
local:scan-collection(concat($collection, "/", $child))
};

local:scan-collection("/db")

The following solution is much shorter, but it might be slower:

distinct-values(
  for $doc in collection("/db")
  return util:collection-name($doc)
)

Wolfgang

XQuery count()

2009-11-09T19:24:51Z

Thanks to all.
Both Wolfgang approaches work well.

Regards!

2009/11/9 G. Ken Holman 

> At 2009-11-09 18:55 +0000, Ricardo Queirós wrote:
>
>> I need to count the total number of distinct collections in my db (eXist
>> XML Database).
>>
>
> How are you defining "distinct" in your situation?  There are many possible
> ways:  string value, tree structure, tree structure of string values, etc.
>
>
>  How can i do this in a compact way using XQuery?
>>
>
> Consider using deep-equal() ... if you have two root nodes, say in $a and
> $b, you can get true/false with deep-equal( $a, $b ) and it will check the
> tree structure and the node values in the tree structure and things like
> attributes will be checked regardless of their order (because attributes in
> XML documents are unordered).
>
> But be aware this is an "expensive" function, and could take up a lot of
> processing resources to accomplish the comparison.
>
> I hope this helps.
>
> . . . . . . . . . . . Ken
>
>
> --
> Upcoming:  hands-on XSLT, XQuery and XSL-FO Washington DC Nov 2009
> Interested in other classes?  http://www.CraneSoftwrights.com/q/i/
> Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/q/
> Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
> Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
> Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
> G. Ken Holman                 mailto:http://x-query.com/mailman/listinfo/talk
> Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/q/bc
> Legal business disclaimers:  http://www.CraneSoftwrights.com/legal
>
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>



-- 
Ricardo Queirós
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091109/48f7cd3a/attachment.htm

XQuery count()

2009-11-09T18:55:18Z

Hi,

I need to count the total number of distinct collections in my db (eXist XML
Database).
How can i do this in a compact way using XQuery?

Thanks!


-- 
Ricardo Queirós
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091109/43319729/attachment.htm

XQuery count()

2009-11-09T14:07:10Z

At 2009-11-09 18:55 +0000, Ricardo Queirós wrote:
>I need to count the total number of distinct 
>collections in my db (eXist XML Database).

How are you defining "distinct" in your 
situation?  There are many possible ways:  string 
value, tree structure, tree structure of string values, etc.

>How can i do this in a compact way using XQuery?

Consider using deep-equal() ... if you have two 
root nodes, say in $a and $b, you can get 
true/false with deep-equal( $a, $b ) and it will 
check the tree structure and the node values in 
the tree structure and things like attributes 
will be checked regardless of their order 
(because attributes in XML documents are unordered).

But be aware this is an "expensive" function, and 
could take up a lot of processing resources to accomplish the comparison.

I hope this helps.

. . . . . . . . . . . Ken

--
Upcoming:  hands-on XSLT, XQuery and XSL-FO Washington DC Nov 2009
Interested in other classes?  http://www.CraneSoftwrights.com/q/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/q/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:http://x-query.com/mailman/listinfo/talk
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/q/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

XQuery Implementation for PHP

2009-11-04T10:03:02Z

look like "soup from stone" (or nails or axe - depend from country)  -
php should leave our lives at all

Stephen Leacock  - "I detest life-insurance agents: they always argue
that I shall some day die, which is not so." -
http://www.brainyquote.com/quotes/authors/s/stephen_leacock.html


2009/11/4 William Candillon :
> Hello,
>
> The PHP bindings are featured in an IBM developerWorks article:
> http://www.ibm.com/developerworks/xml/library/x-zorba/index.html.
>
> Best regards,
>
> William
>
> On Fri, Oct 16, 2009 at 7:17 AM, Matthias Brantner
>  wrote:
>> Hello,
>> the Zorba XQuery processor (zorba-xquery.org) provides language bindings for
>> PHP.
>> Matthias
>> On 16.10.2009, at 05:00, Wely Live wrote:
>>
>> Hello,
>>
>> I am new in XQuery.
>> Could somebody suggest is there any free implementation of XQuery for PHP?
>> Thanks.
>>
>> Regards,
>> Wely
>> _______________________________________________
>> http://x-query.com/mailman/listinfo/talk
>> http://x-query.com/mailman/listinfo/talk
>>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

XQuery Implementation for PHP

2009-11-03T22:34:54Z

Hello,

The PHP bindings are featured in an IBM developerWorks article:
http://www.ibm.com/developerworks/xml/library/x-zorba/index.html.

Best regards,

William

On Fri, Oct 16, 2009 at 7:17 AM, Matthias Brantner
 wrote:
> Hello,
> the Zorba XQuery processor (zorba-xquery.org) provides language bindings for
> PHP.
> Matthias
> On 16.10.2009, at 05:00, Wely Live wrote:
>
> Hello,
>
> I am new in XQuery.
> Could somebody suggest is there any free implementation of XQuery for PHP?
> Thanks.
>
> Regards,
> Wely
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Implementations of XQuery Update Facility requested

2009-11-02T18:14:07Z

Back in August, the XML Query Working Group announced the availability of 
version 1.0.0 of the XQuery Update Facility Test Suite [1]. This test 
suite reflects the XQuery Update Facility 1.0 Candidate Recommendation [2] 
that was published on June 9. 

We are pleased to have received results from Saxonica. We'd like to 
encourage other implementators to submit their results to us, so that we 
can advance XQuery Update Facility to W3C Recommendation.

                                                -- Andrew

[1] XQuery Update Facility Test Suite
http://dev.w3.org/2007/xquery-update-10-test-suite/

[2] XQuery Update Facility 1.0 
http://www.w3.org/TR/2009/CR-xquery-update-10-20090609/ 

--------------------
Andrew Eisenberg
IBM
4 Technology Park Drive
Westford, MA  01886

http://x-query.com/mailman/listinfo/talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091102/6d65e159/attachment.htm

data camp and apps competition in SF

2009-11-01T13:16:01Z

Hello XQuery fans,

I am just curious, are there any XQuery fans participating in this  
competition,
and using it as an app development language ?

http://datacamp.eventbrite.com/

If yes, it would be really interesting to know how XQuery performed
given the task.

Let us know !

Dana


Other related links:

http://datasf.org/

http://datasf.org/showcase/

http://www.sfexaminer.com/opinion/blogs/under-the-dome/Get-ready-for-an-apps-battle-63159797.html

First XQuery Meetup at Paris, La Cantine, December 3rd 2009

2009-11-01T09:39:32Z

Hello XQuery lovers!

A couple of weeks ago we started a wiki at http://xquery.pbworks.com
in order to help XQuery enthusiasts from the same area to gather
around a snack and a beer. The aim is to promote knowledge sharing and
best practice amongst XQuery developers and business professionals
around the world by organizing local meetups.

Anyone can participate and/or organize a meetup, the wiki contains
some guidelines, please feel free to contact me for more info.

The first Paris XQuery Meetup will be held at la Cantine, December 3rd
2009. More details at http://xquery.pbworks.com/Paris-Meetup.
This event is restricted to 20 people maximum, registration is done on the wiki.
Please tell your friends about it!

Best regards,

William

[ANN] Altova announces v2010 of the MissionKit tool suite

2009-10-29T17:11:40Z

Altova is pleased to announce general availability version 2010 of its
MissionKit XML, database, and UML tools. v2010 is our MOST WANTED
release and includes over 70 new customer-requested features. Just a few
of these include: 

 

* Support for WSDL 2.0, JSON, and SysML technologies

* Database schema comparison

* New stylesheet design paradigm with advanced support for electronic
form design

* Enhanced XBRL functionality

* And much more 

 

More info and screenshots are available at
 

 

Download a fully functional free trial at
 

Best regards,

Liz

 

Liz Andrews

Technical Marketing Manager
Altova, Inc.

www.altova.com

 

________________________________

This e-mail and any attachments are intended only for the person/entity
to which they are addressed and may contain confidential and/or
privileged material. If you received this in error, please notify the
sender and delete the message.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091029/1898d618/attachment.htm

Andy K Chang is on Business Trip with Limited Availability

2009-10-29T14:02:44Z

I will be out of the office starting 10/25/2009 and will not return until
10/30/2009.

I am in Las Vegas for 2009 IBM IOD Conference. My availability is limited but
am available to any important or emergent matters. I will check my emails a
few times a day and be available from my cell. If you need help, please
contact Sundi Mahadevan first

American Express made the following annotations on Thu Oct 29 2009 15:02:46
------------------------------------------------------------------------------
"This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you."

American Express a ajouté le commentaire suivant le Thu Oct 29 2009 15:02:46

Ce courrier et toute pièce jointe qu'il contient sont réservés au seul destinataire indiqué et peuvent renfermer des renseignements confidentiels et privilégiés. Si vous n'êtes pas le destinataire prévu, toute divulgation, duplication, utilisation ou distribution du courrier ou de toute pièce jointe est interdite. Si vous avez reçu cette communication par erreur, veuillez nous en aviser par courrier et détruire immédiatement le courrier et les pièces jointes. Merci.

******************************************************************************
-------------------------------------------------------------------------------

Benchmarking on XQuery Implementation in DBMS

2009-10-29T10:56:45Z

Hi,

I am planning to research the comparison and benchmarking XQuery Implementation in DBMS (SQL Server, Oracle, DB2). There are a few question to discuss as following:
1. I wonder if anybody has ever done the similar research?
    I would be appreciate if anybody could provide my any related source or document.
2. To do the benchmarking, we of course need a sample dataset. Anybody has reference where can I find a good dataset sample?

Thanks!

Regards,
Wely
Post Graduate Student
Nanyang Technological University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091029/1e01edd0/attachment.htm

Benchmarking on XQuery Implementation in DBMS

2009-10-29T10:37:16Z

Hi,

There is also the TPoX Benchmark suite that might be worth taking a look at..

http://tpox.sourceforge.net/

Kind regards,
Geert

>


Drs. G.P.H. Josten
Consultant


http://www.daidalos.nl/
Daidalos BV
Source of Innovation
Hoekeindsehof 1-4
2665 JZ Bleiswijk
Tel.: +31 (0) 10 850 1200
Fax: +31 (0) 10 850 1199
http://www.daidalos.nl/
KvK 27164984
De informatie - verzonden in of met dit emailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend.


> From: http://x-query.com/mailman/listinfo/talk
> [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf Of Ronald Bourret
> Sent: donderdag 29 oktober 2009 6:18
> To: Wely Live
> Cc: http://x-query.com/mailman/listinfo/talk
> Subject: Re:  Benchmarking on XQuery
> Implementation in DBMS
>
> Hello,
>
> You can find a partial list of XML query benchmarks at [1].
> You should also search DBLP [2] for XML and benchmark, as
> there may be more recent research in this area. Finally, be
> sure to read your license agreements carefully.
> Unfortunately, many databases do not allow you to publish the
> results of benchmarks.
>
> -- Ron
>
> [1] http://www.rpbourret.com/xml/XMLDBLinks.htm#Benchmarks
> [2] http://www.informatik.uni-trier.de/~ley/db/indices/t-form.html
>
> Wely Live wrote:
> > Hi,
> >
> > I am planning to research the comparison and benchmarking
> XQuery Implementation in DBMS (SQL Server, Oracle, DB2).
> There are a few question to discuss as following:
> > 1. I wonder if anybody has ever done the similar research?
> >     I would be appreciate if anybody could provide my any
> related source or document.
> > 2. To do the benchmarking, we of course need a sample
> dataset. Anybody has reference where can I find a good dataset sample?
> >
> > Thanks!
> >
> > Regards,
> > Wely
> > Post Graduate Student
> > Nanyang Technological University
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Benchmarking on XQuery Implementation in DBMS

2009-10-29T09:09:48Z

A fair bit of work has been done but more is always needed.
 
I think you need to start out with very clear objectives, especially in the
following areas:
 
* are you going to try and measure how well the products handle something
close to a "real-life" workload (including factors such as updating,
database loading, etc), or are you going to measure a small number of
synthetic queries?
 
* are you going to make strenuous efforts to get the best possible
performance out of each product that you test, or are you simply going to
measure what it achieves "out of the box"?
 
* do you just want to produce some numbers, or do you want to use those
numbers to test some theories or hypotheses?
 
Most of the papers I've seen on XSLT, XPath, or XQuery benchmarks over the
years suffer from one or more of the following drawbacks:
 
* Some of the queries they measure are totally unrepresentative of real
life, for example IIRC one benchmark has a query like
/descendant::*/ancestor::*/descendant::*. Who cares whether that performs
well? (But it can be useful, of course, if your objective is to find out how
path expressions are implemented internally).
 
* Simple flaws in the measurement methodology, for example failing to take
account of Java VM startup time, or to distinguish query compilation,
document parsing, and query evaluation costs. Such flaws can make your
numbers meaningless, but they are surprisingly common in published papers.
 
* Running products in a sub-optimal way. This might say something about the
product's usability or documentation, but if you run with the wrong
configuration options, it tells you very little about the product's true
performance capabilities. This is especially true where your knowledge of
one of the products is much better than your knowledge of others: a factor
that taints all comparative benchmarks produced by vendors.
 
* Making incorrect inferences from the measurements (for example, the
authors of one paper drew inferences about the internal workings of Saxon
which were completely and bizarrely wrong, and they didn't check them with
me before publication.)
 
It's worth remembering that in real life, the performance you achieve
depends much more on your skills as a developer and your knowledge and
experience of the technology you are using than on the absolute capabilities
of the chosen product. Performance problems in real life don't occur because
the technology is intrinsically slow, they occur because it is misused. This
places a severe limitation on the usefulness of benchmark numbers.
 
Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 


  _____  

From: http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf
Of Wely Live
Sent: 29 October 2009 02:57
To: http://x-query.com/mailman/listinfo/talk
Subject:  Benchmarking on XQuery Implementation in DBMS


Hi,
 
I am planning to research the comparison and benchmarking XQuery
Implementation in DBMS (SQL Server, Oracle, DB2). There are a few question
to discuss as following:
1. I wonder if anybody has ever done the similar research?
    I would be appreciate if anybody could provide my any related source or
document.
2. To do the benchmarking, we of course need a sample dataset. Anybody has
reference where can I find a good dataset sample?
 
Thanks!
 
Regards,
Wely
Post Graduate Student
Nanyang Technological University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091029/babab73b/attachment.htm

Benchmarking on XQuery Implementation in DBMS

2009-10-28T22:18:02Z

Hello,

You can find a partial list of XML query benchmarks at [1]. You should 
also search DBLP [2] for XML and benchmark, as there may be more recent 
research in this area. Finally, be sure to read your license agreements 
carefully. Unfortunately, many databases do not allow you to publish the 
results of benchmarks.

-- Ron

[1] http://www.rpbourret.com/xml/XMLDBLinks.htm#Benchmarks
[2] http://www.informatik.uni-trier.de/~ley/db/indices/t-form.html

Wely Live wrote:
> Hi,
> 
> I am planning to research the comparison and benchmarking XQuery Implementation in DBMS (SQL Server, Oracle, DB2). There are a few question to discuss as following:
> 1. I wonder if anybody has ever done the similar research?
>     I would be appreciate if anybody could provide my any related source or document.
> 2. To do the benchmarking, we of course need a sample dataset. Anybody has reference where can I find a good dataset sample?
> 
> Thanks!
> 
> Regards,
> Wely
> Post Graduate Student
> Nanyang Technological University

XQuery Implementation for PHP

2009-10-16T12:00:29Z

Hello,

I am new in XQuery.
Could somebody suggest is there any free implementation of XQuery for PHP?
Thanks.

Regards,
Wely
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091016/f9a8ce3f/attachment.htm

XQuery Implementation for PHP

2009-10-16T09:17:20Z

Hello,

the Zorba XQuery processor (zorba-xquery.org) provides language  
bindings for PHP.

Matthias

On 16.10.2009, at 05:00, Wely Live wrote:

> Hello,
>
> I am new in XQuery.
> Could somebody suggest is there any free implementation of XQuery  
> for PHP?
> Thanks.
>
> Regards,
> Wely
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091016/31feb47c/attachment.htm

XQuery Implementation for PHP

2009-10-16T09:02:03Z

with eXist (http://exist-db.org) you don`t need php anymore (and even
nor Apache and nor MySQL)

On Fri, Oct 16, 2009 at 7:00 AM, Wely Live  wrote:
> Hello,
>
> I am new in XQuery.
> Could somebody suggest is there any free implementation of XQuery for PHP?
> Thanks.
>
> Regards,
> Wely
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Fwd: How you can help promote the Developer Gathering (part of TPAC 2009)

2009-10-07T09:46:33Z

Maybe this is of interest to the XQuery developers in the Bay Area.

http://www.w3.org/2009/11/TPAC/DevMeeting.html

Best regards
Dana


>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091007/e529d039/attachment.htm

parsing dates

2009-10-06T21:21:03Z

> 
> I think that's not quite correct. Typeswitch case clauses 
> expect a SequenceType, which includes ItemType. So the tests 
> can be any node type or any schema type (see 
> http://www.w3.org/TR/xquery/#id-typeswitch and 
> http://www.w3.org/TR/xquery/#doc-xquery-SequenceType).
> 
> Here's an example: note that the order of the 'case' clauses 
> is important. Since xs:ID is derived from xs:string, it must 
> be tested before any xs:string test case.
> 
> xquery version "1.0-ml";
> 
> let $i := "test"
> return
>    typeswitch ($i)
>     case xs:integer return "integer"
>     case xs:ID return "id"
>     case xs:string return "string"
>     default return error((), "UNEXPECTED", $i)
> 

But we aren't starting with an xs:date or xs:gYearMonth; we are starting
with a string that is capable of being converted to an xs:date or
xs:gYearMonth. So "castable as" is indeed the right solution.

Except that matches() would be more flexible, because it can also test for
formats that don't happen to correspond to any known schema-defined type.

>Is that because xs:integer is considered an AtomicType?

xs:integer is most definitely an AtomicType, there's no "considered to be"
about it.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay

parsing dates

2009-10-06T20:09:09Z

> I think that's not quite correct. Typeswitch case clauses
> expect a SequenceType, which includes ItemType. So the tests
> can be any node type or any schema type (see
> http://www.w3.org/TR/xquery/#id-typeswitch and
> http://www.w3.org/TR/xquery/#doc-xquery-SequenceType).
>
> Here's an example: note that the order of the 'case' clauses
> is important. Since xs:ID is derived from xs:string, it must
> be tested before any xs:string test case.
>
> xquery version "1.0-ml";
>
> let $i := "test"
> return
>    typeswitch ($i)
>     case xs:integer return "integer"
>     case xs:ID return "id"
>     case xs:string return "string"
>     default return error((), "UNEXPECTED", $i)
>
> -- Mike

Is that because xs:integer is considered an AtomicType?

Kind regards,
Geert





Drs. G.P.H. Josten
Consultant


http://www.daidalos.nl/
Daidalos BV
Source of Innovation
Hoekeindsehof 1-4
2665 JZ Bleiswijk
Tel.: +31 (0) 10 850 1200
Fax: +31 (0) 10 850 1199
http://www.daidalos.nl/
KvK 27164984
De informatie - verzonden in of met dit emailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend.

parsing dates

2009-10-06T16:34:35Z

On Tue, Oct 6, 2009 at 14:44, David Sewell  wrote:
> On Mon, 5 Oct 2009, Joe Wicentowski wrote:
>
>> As far as your specific question, you could run a series of "if ($date
>> castable as ...) then ... else" statements on the following datatypes:
> typeswitch tests for node kinds, not data types, so the "if castable"
> approach is probably the way to go (barring the custom data type
> solution that Michael mentioned).

Thanks Joe, Mike, and David,

I think you are right that the "if castable as" approach will work the
best. Mike's suggestion (while perhaps more reliable) is probably
overkill.  I don't actually care to validate the dates as a date,
since if someone is putting in funny non-dates in this case, then they
probably deserve the error message they'll get.  The files are a
historical diary and I just want to give users (well, and my
processing) the option of looking at it by day, by month, or by year
through a simple change in a hackable human-readable URL. All the real
date/@when in the files are YYYY-MM-DD but want to let them do
/diary/1788-04-07.html (or .xml) and get that one day's entry or
/diary/1788-04.html to get the month or /diary/1788.html to get the
year.  Obviously in 99% of the time the urls will be generated from
some link they are clicking on but it gives me an easy way to move
from day to month to year if desired.

I suppose another way would be not to treat these as dates at all, but
in canonicalizing the data before loading it into eXist, I could give
unique @xml:ids to each day, month and year, that were in the form of
somethingYYYY on the enclosing element for the year's entries,
somethingYYYY-MM for the element enclosing a month, and
somethingYYYY-MM-DD for the element enclosing a day.  That way when
then ask for /diary/YYYY-MM-DD I would just go get the element with
somethingYYYY-MM-DD as an @xml:id.  I guess that the drawbacks of that
are that I can't really do an date-arithmetic, but that said at this
stage I'm not sure I need to. I was just assuming I should be doing it
by date (rather than ID) because the whole work is date oriented.

Thanks for the help, as I'm now working feverishly on this I'm sure
I'll be asking for help again soon! ;-)

-James

parsing dates

2009-10-06T12:59:37Z

Hi James,

Got it - now I understand.

Well, if your dates are normalized to yyyy-mm-dd, I'd suggest keeping
them as is in date/@when.  This way, you can set up a range index on
@when, and you can do date comparisons (<, >, =) based on the
searches.  See eXist's documentation on setting up indexing, but I'd
use something like:



This would enable comparison queries like:

let $date := request:get-parameter("date", ("1789-01-01"))
let $query-start-date :=
  if ($date castable as xs:date) then $date cast as xs:date
  else if ($date castable as xs:gYearMonth) then concat($date, '-01')
cast as xs:date
  else concat($date, '-01-01') cast as xs:date
let $query-end-date :=
  if ($date castable as xs:date) then $date cast as xs:date
  else if ($date castable as xs:gYearMonth) then concat($date,
functx:days-in-month($month)) cast as xs:date
  else concat($date, '-12-31') cast as xs:date
for $hit in $docs[tei:date/@when ge $query-start-date and
tei:date/@when le $query-end-date]
...

(for the functx days-in-month function, see
http://www.xqueryfunctions.com/xq/functx_days-in-month.html)

Joe

parsing dates

2009-10-06T10:44:09Z

On Mon, 5 Oct 2009, Joe Wicentowski wrote:

> As far as your specific question, you could run a series of "if ($date
> castable as ...) then ... else" statements on the following datatypes:

[...]

> As an alternative to conditional expressions, I'm not sure if
> typeswitch would work for this?

typeswitch tests for node kinds, not data types, so the "if castable"
approach is probably the way to go (barring the custom data type
solution that Michael mentioned).

-- 
David Sewell, Editorial and Technical Manager
ROTUNDA, The University of Virginia Press
PO Box 801079, Charlottesville, VA 22904-4318 USA
Courier: 310 Old Ivy Way, Suite 302, Charlottesville VA 22903
Email: http://x-query.com/mailman/listinfo/talk   Tel: +1 434 924 9973
Web: http://rotunda.upress.virginia.edu/

parsing dates

2009-10-06T10:39:32Z

On 2009-10-06 06:44, David Sewell wrote:
> On Mon, 5 Oct 2009, Joe Wicentowski wrote:
>
>> As far as your specific question, you could run a series of "if ($date
>> castable as ...) then ... else" statements on the following datatypes:
>
> [...]
>
>> As an alternative to conditional expressions, I'm not sure if
>> typeswitch would work for this?
>
> typeswitch tests for node kinds, not data types, so the "if castable"
> approach is probably the way to go (barring the custom data type
> solution that Michael mentioned).

I think that's not quite correct. Typeswitch case clauses expect a 
SequenceType, which includes ItemType. So the tests can be any node type 
or any schema type (see http://www.w3.org/TR/xquery/#id-typeswitch and 
http://www.w3.org/TR/xquery/#doc-xquery-SequenceType).

Here's an example: note that the order of the 'case' clauses is 
important. Since xs:ID is derived from xs:string, it must be tested 
before any xs:string test case.

xquery version "1.0-ml";

let $i := "test"
return
   typeswitch ($i)
    case xs:integer return "integer"
    case xs:ID return "id"
    case xs:string return "string"
    default return error((), "UNEXPECTED", $i)

-- Mike

parsing dates

2009-10-06T08:09:56Z

I don't know if eXist has any schema support. If it does, you can define a
type genericDate that is a union of xs:date, xs:gYearMonth, and xs:gYear.
You can't cast to the union type (sadly) but you can construct an element
and validate against a schema that defines the element as a union type, and
then atomize.

OK, that's a bit longwinded. Sorry!

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 
 

> -----Original Message-----
> From: http://x-query.com/mailman/listinfo/talk 
> [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf Of James Cummings
> Sent: 06 October 2009 00:05
> To: http://x-query.com/mailman/listinfo/talk
> Subject:  parsing dates
> 
> Hi there,
> 
> I've a silly question about the best way to parse dates.  I'm 
> passing a parameter (in eXist) that is part of the URL for an 
> xquery and I want to separate it into its individual parts of 
> year month and day. I could test the string and do 
> substring-before and after and such, or create my own 
> function, but it seemed better practice to use the functions 
> like year-from-date and month-from-date.  So I tried:
> 
> let $date := request:get-parameter("date", ("1789-01-01")) 
> cast as xs:date let $year := fn:year-from-date($date) let 
> $month := fn:month-from-date($date) let $day := 
> fn:day-from-date($date)
> 
> which works fine when the date is YYYY-MM-DD.  But what I 
> want to achieve is being able to handle YYYY-MM-DD, YYYY-MM 
> or just YYYY.  The last two give me an error of "xs:date must 
> not have hour, minute or second fields set" (erm... which it 
> doesn't, which is why I'm confused). I'm happy for $month and 
> $day to be empty if they don't exist.
> 
> I'm sure it is something terribly obvious with the datatype.  
> Should I just be breaking it down to its component parts 
> myself?  It just seemed like I was reinventing the wheel.
> 
> Thanks for any suggestions.
> 
> -James
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk

parsing dates

2009-10-06T01:05:27Z

Hi there,

I've a silly question about the best way to parse dates.  I'm passing
a parameter (in eXist) that is part of the URL for an xquery and I
want to separate it into its individual parts of year month and day. I
could test the string and do substring-before and after and such, or
create my own function, but it seemed better practice to use the
functions like year-from-date and month-from-date.  So I tried:

let $date := request:get-parameter("date", ("1789-01-01")) cast as xs:date
let $year := fn:year-from-date($date)
let $month := fn:month-from-date($date)
let $day := fn:day-from-date($date)

which works fine when the date is YYYY-MM-DD.  But what I want to
achieve is being able to handle YYYY-MM-DD, YYYY-MM or just YYYY.  The
last two give me an error of "xs:date must not have hour, minute or
second fields set" (erm... which it doesn't, which is why I'm
confused). I'm happy for $month and $day to be empty if they don't
exist.

I'm sure it is something terribly obvious with the datatype.  Should I
just be breaking it down to its component parts myself?  It just
seemed like I was reinventing the wheel.

Thanks for any suggestions.

-James

parsing dates

2009-10-05T21:16:57Z

James,

I've encountered the same issue with dates in TEI documents.  What
follows are some thoughts/suggestions about using TEI dates in eXist.

I think the issue is The TEI Guidelines allows/encourages one to mix
date datatypes, thereby conflating data with precision in dates
(http://www.tei-c.org/release/doc/tei-p5-doc/en/html/CO.html#CONADA):

------
"Partial dates or times (e.g. 1990, September 1990, twelvish) can be
expressed in the when attribute by simply omitting a part of the value
supplied. Imprecise dates or times (for example early August, some
time after ten and before twelve) may be expressed as date or time
ranges...  The @when attribute always supplies a normalized
representation of the date given as content of the date element. The
format used should be a valid W3C schema datatype. Some typical
examples follow:

* The year 2001
* September 2001
* 11 Sept 01
* 9/11
* September
* Eleventh of the month
* 8:48
* Sept 11th, 12 minutes before 9 am

Note in the last example the use of a normalized representation for
the date string which includes a time: this example could thus equally
well be tagged using the time element.
The following examples demonstrate the use of the date element to mark
a period of time:
* Those five years — 1918 to
1923 — had been, he suspected, somehow very important."
-----

If you mix 2+ date datatypes, you will encounter some difficulties
when doing what you asked about.  Setting range indexes in eXist is
impossible if you mix datatypes, as far as I know.  I think we might
encode TEI texts more effectively to make processing easier by (1)
either using a consistent datatype or (2) replacing the generic @when
with a new way of encoding which implies the datatype.

As far as your specific question, you could run a series of "if ($date
castable as ...) then ... else" statements on the following datatypes:

* date
* gYear
* gMonth
* gDay
* gYearMonth
* gMonthDay
* time
* dateTime

As an alternative to conditional expressions, I'm not sure if
typeswitch would work for this?

- Joe



On Mon, Oct 5, 2009 at 7:05 PM, James Cummings
 wrote:
> Hi there,
>
> I've a silly question about the best way to parse dates.  I'm passing
> a parameter (in eXist) that is part of the URL for an xquery and I
> want to separate it into its individual parts of year month and day. I
> could test the string and do substring-before and after and such, or
> create my own function, but it seemed better practice to use the
> functions like year-from-date and month-from-date.  So I tried:
>
> let $date := request:get-parameter("date", ("1789-01-01")) cast as xs:date
> let $year := fn:year-from-date($date)
> let $month := fn:month-from-date($date)
> let $day := fn:day-from-date($date)
>
> which works fine when the date is YYYY-MM-DD.  But what I want to
> achieve is being able to handle YYYY-MM-DD, YYYY-MM or just YYYY.  The
> last two give me an error of "xs:date must not have hour, minute or
> second fields set" (erm... which it doesn't, which is why I'm
> confused). I'm happy for $month and $day to be empty if they don't
> exist.
>
> I'm sure it is something terribly obvious with the datatype.  Should I
> just be breaking it down to its component parts myself?  It just
> seemed like I was reinventing the wheel.
>
> Thanks for any suggestions.
>
> -James
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

In search of sample data ...

2009-10-03T11:53:14Z

Hi,

How about online articles, news feeds, blogs? Though these are typically relatively short in length..

Kind regards,
Geert

>


Drs. G.P.H. Josten
Consultant


http://www.daidalos.nl/
Daidalos BV
Source of Innovation
Hoekeindsehof 1-4
2665 JZ Bleiswijk
Tel.: +31 (0) 10 850 1200
Fax: +31 (0) 10 850 1199
http://www.daidalos.nl/
KvK 27164984
De informatie - verzonden in of met dit emailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend.


> From: http://x-query.com/mailman/listinfo/talk
> [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf Of daniela florescu
> Sent: zaterdag 3 oktober 2009 0:42
> To: Hans-Juergen Rennau
> Cc: http://x-query.com/mailman/listinfo/talk
> Subject: Re:  In search of sample data ...
>
> Hans Juergen,
>
> I only have three words of advise, given your request:
>
> www.data.gov
> http://www.recovery.gov/
> www.programmableweb.com/
>
> Best regards
> Dana
>
> On Oct 2, 2009, at 3:03 PM, Hans-Juergen Rennau wrote:
>
> > Hello People,
> >
> > preparing an extensive XQuery workshop, I am looking for
> interesting
> > sample data. What I have in mind is data that
> > - are public and available online via URL (either static or
> as REST-
> > ful webservice)
> > - have intermediate complexity (> 50 data items, <= 5 MB)
> > - must be in English (both tags and contents)
> > - may be dynamic, but not too ephemeral
> > - schema data welcome
> >
> > Should anybody have suggestions, I would be grateful!
> >
> > With kind regards,
> > Hans-Juergen
> >
> >
> >
> >
> > _______________________________________________
> > http://x-query.com/mailman/listinfo/talk
> > http://x-query.com/mailman/listinfo/talk
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

In search of sample data ...

2009-10-02T23:03:46Z

Hello People,

preparing an extensive XQuery workshop, I am looking for interesting sample data. What I have in mind is data that
- are public and available online via URL (either static or as REST-ful webservice)
- have intermediate complexity (> 50 data items, <= 5 MB)
- must be in English (both tags and contents)
- may be dynamic, but not too ephemeral
- schema data welcome

Should anybody have suggestions, I would be grateful!

With kind regards,
Hans-Juergen

In search of sample data ...

2009-10-02T16:41:31Z

Hans Juergen,

I only have three words of advise, given your request:

www.data.gov
http://www.recovery.gov/
www.programmableweb.com/

Best regards
Dana

On Oct 2, 2009, at 3:03 PM, Hans-Juergen Rennau wrote:

> Hello People,
>
> preparing an extensive XQuery workshop, I am looking for interesting  
> sample data. What I have in mind is data that
> - are public and available online via URL (either static or as REST- 
> ful webservice)
> - have intermediate complexity (> 50 data items, <= 5 MB)
> - must be in English (both tags and contents)
> - may be dynamic, but not too ephemeral
> - schema data welcome
>
> Should anybody have suggestions, I would be grateful!
>
> With kind regards,
> Hans-Juergen
>
>
>
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk

[ANN] eXcel Function Library

2009-10-02T14:19:00Z

Hi,

the FLWOR Foundation has developed a set of library modules written in XQuery.
These modules currently consist of 217 functions that provide a functionality similar to the
functionality (e.g. statistic or logic functions) provided in today's spreadsheet applications
(e.g. Microsoft Excel).

All the modules are written entirely in XQuery 1.0 (i.e. without external functions) and have
been tested to work with Zorba, Saxon, and eXist.

You can find more information, documentation, and download links at:
http://sourceforge.net/apps/mediawiki/zorba/index.php?title=EXcel_Function_Library

Best,
Sorin Nasoi

---
Sorin Nasoi
Senior Software Engineer
Enea Romania
Trust Center 319C Splaiul Independentei
District 6, Bucharest, Romania, 060044
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091002/98629469/attachment.htm

using a variable within a regexp pattern in XQuery

2009-10-02T02:38:04Z

Hi Michael,

Thanks a lot for your reply. Sorry about not being more explicit or clear.
I'll bear that in mind for my future emails.

What I meant with "works" is that the search expression and the term did
match. When they matched, the output was the term searched for. When they
didn't match, there was no output.

Thanks a lot for your suggestion. I certainly needed to use concat, but the
expression that eventually did the trick was:

matches(., concat('^', $q, '.*$'))

What I'd like is to control whether the search term matched the entire term,
part of it at the beginning, or part of it at the end, part of it in the
middle, etc.

Thanks again, and good night!
Manuel

2009/10/2 Michael Kay 

>  It's very unhelpful to report that something "doesn't work" without
> saying how it failed. And if you expected the expression to do something
> useful, it's useful to say what you expected it to do: you can't expect us
> to guess the intention behind incorrect code.
>
> If $q is a variable containing part of a regular expression, then you can
> include it in a regex using the concat function:
>
> matches(., concat('(', $q, ')*')
>
>
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay
>
>  ------------------------------
> *From:* http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk] *On
> Behalf Of *Manuel Souto Pico
> *Sent:* 01 October 2009 19:53
> *To:* http://x-query.com/mailman/listinfo/talk
> *Subject:*  using a variable within a regexp pattern in
> XQuery
>
> Hi,
>
> I'm a bit stuck trying to run a regexp pattern in a XQuery search file.
>
> (1) where $record/langSet[@xml:lang="en"]/tig/term[matches(., $q)]
> works
> (2) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
> "$q")]            doesn't work
> (3) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
> "^$q.*$")]         doesn't work
> (4) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
> "^bu.+$")]          works!
>
> Any ideas about how I can get to use regular expressions in this query?
>
> (4) retrieves 'bufer', 'bug' and 'bus'. If $q is 'bu', (1) retrieves as
> well 'attri*bu*te', 'contrast *bu*tton', etc.
>
> Thanks a lot!
>
> Cheers, Manuel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091002/d6025020/attachment.htm

RE using a variable within a regexp pattern

2009-10-02T00:39:34Z

Dear David and Hans-Juergen,

Thanks a lot for your replies and, notably, for Jans-Juergen's explanation.
Very useful indeed!

This is a bit more complex than perl's regular expressions but still quite
straightforward at the end of the day. The solution had crossed my mind very
fast, but then I got distracted and I forgot. But indeed this is what I
needed:

        where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
concat('^', $q, '.*$'))]

Thanks a lot! And keep well :))
Chees, Manuel


2009/10/1 Hans-Juergen Rennau 

> Hi Manual,
>
> if I understand you correctly, you wonder how you can compose a match
> pattern from variable and fixed  parts, e.g. framing a variable term by an
> initial ^ and a trailing $, right?
>
> The solution is simple: use an _expression_ whose string value is the
> desired sequence of characters. Often this will be achieved using the concat
> function, whose arguments in turn may be literals, variable references,
> function calls, path expressions.... Examples:
>
> matches(., concat('^', $q, '$'))
> matches(., concat($prefix, $q, $postfix))
> matches(., concat('^', 'ID', position()))
> matches(., concat('^', $n/@criterion, '.*s'))
>
> So the task becomes very simple if you remember that the way an expression
> is resolved to a value is independent on the context where the value is
> used. Therefore, you need not any special knowledge about the matches
> function - all you need is the general understanding how to build an
> expression that resolves to the intended string.
>
> With kind regards,
> Hans-Juergen
>
> ------------------------------
>
> Message: 4
> Date: Thu, 1 Oct 2009 20:53:29 +0200
> From: Manuel Souto Pico 
> Subject:  using a variable within a regexp pattern in
>    XQuery
> To: http://x-query.com/mailman/listinfo/talk
> Message-ID:
>    
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
> I'm a bit stuck trying to run a regexp pattern in a XQuery search file.
>
> (1) where $record/langSet[@xml:lang="en"]/tig/term[matches(., $q)]
> works
> (2) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
> "$q")]            doesn't work
> (3) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
> "^$q.*$")]         doesn't work
> (4) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
> "^bu.+$")]          works!
>
> Any ideas about how I can get to use regular expressions in this query?
>
> (4) retrieves 'bufer', 'bug' and 'bus'. If $q is 'bu', (1) retrieves as
> well
> 'attri*bu*te', 'contrast *bu*tton', etc.
>
> Thanks a lot!
>
> Cheers, Manuel
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091001/728a81e2/attachment.htm

using a variable within a regexp pattern in XQuery

2009-10-02T00:39:10Z

It's very unhelpful to report that something "doesn't work" without saying
how it failed. And if you expected the expression to do something useful,
it's useful to say what you expected it to do: you can't expect us to guess
the intention behind incorrect code.
 
If $q is a variable containing part of a regular expression, then you can
include it in a regex using the concat function:
 
matches(., concat('(', $q, ')*')
 
Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 


  _____  

From: http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf
Of Manuel Souto Pico
Sent: 01 October 2009 19:53
To: http://x-query.com/mailman/listinfo/talk
Subject:  using a variable within a regexp pattern in XQuery


Hi, 

I'm a bit stuck trying to run a regexp pattern in a XQuery search file.

(1) where $record/langSet[@xml:lang="en"]/tig/term[matches(., $q)]
works
(2) where $record/langSet[@xml:lang="en"]/tig/term[matches(., "$q")]
doesn't work
(3) where $record/langSet[@xml:lang="en"]/tig/term[matches(., "^$q.*$")]
doesn't work
(4) where $record/langSet[@xml:lang="en"]/tig/term[matches(., "^bu.+$")]
works!

Any ideas about how I can get to use regular expressions in this query? 

(4) retrieves 'bufer', 'bug' and 'bus'. If $q is 'bu', (1) retrieves as well
'attribute', 'contrast button', etc.

Thanks a lot! 

Cheers, Manuel


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091001/22c0df09/attachment.htm

RE using a variable within a regexp pattern

2009-10-01T22:13:38Z

Hi Manual,

if I understand you correctly, you wonder how you can compose a match pattern from variable and fixed  parts, e.g. framing a variable term by an initial ^ and a trailing $, right?

The solution is simple: use an _expression_ whose string value is the desired sequence of characters. Often this will be achieved using the concat function, whose arguments in turn may be literals, variable references, function calls, path expressions.... Examples: 

matches(., concat('^', $q, '$'))
matches(., concat($prefix, $q, $postfix))
matches(., concat('^', 'ID', position()))
matches(., concat('^', $n/@criterion, '.*s'))

So the task becomes very simple if you remember that the way an expression is resolved to a value is independent on the context where the value is used. Therefore, you need not any special knowledge about the matches function - all you need is the general understanding how to build an expression that resolves to the intended string.

With kind regards,
Hans-Juergen

------------------------------

Message: 4
Date: Thu, 1 Oct 2009 20:53:29 +0200
From: Manuel Souto Pico 
Subject:  using a variable within a regexp pattern in
    XQuery
To: http://x-query.com/mailman/listinfo/talk
Message-ID:
    
Content-Type: text/plain; charset="iso-8859-1"

Hi,

I'm a bit stuck trying to run a regexp pattern in a XQuery search file.

(1) where $record/langSet[@xml:lang="en"]/tig/term[matches(., $q)]
works
(2) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
"$q")]            doesn't work
(3) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
"^$q.*$")]         doesn't work
(4) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
"^bu.+$")]          works!

Any ideas about how I can get to use regular expressions in this query?

(4) retrieves 'bufer', 'bug' and 'bus'. If $q is 'bu', (1) retrieves as well
'attri*bu*te', 'contrast *bu*tton', etc.

Thanks a lot!

Cheers, Manuel

using a variable within a regexp pattern in XQuery

2009-10-01T21:53:29Z

Hi,

I'm a bit stuck trying to run a regexp pattern in a XQuery search file.

(1) where $record/langSet[@xml:lang="en"]/tig/term[matches(., $q)]
works
(2) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
"$q")]            doesn't work
(3) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
"^$q.*$")]         doesn't work
(4) where $record/langSet[@xml:lang="en"]/tig/term[matches(.,
"^bu.+$")]          works!

Any ideas about how I can get to use regular expressions in this query?

(4) retrieves 'bufer', 'bug' and 'bus'. If $q is 'bu', (1) retrieves as well
'attri*bu*te', 'contrast *bu*tton', etc.

Thanks a lot!

Cheers, Manuel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091001/f1af370d/attachment.htm

FW: Unexpected token syntax error, unexpected DeclareFunction_

2009-10-01T19:29:31Z

At 2009-10-01 15:18 -0700, Anil Shekhar wrote:
>Thanks Ken.  It worked.  I did learn something new today.
>
>I am able to successfully read from a database.  However if I add the argum=
>ent to doc() to read files from the file system I do not see anything being=
>  displayed - blank page.
>
>for $d in doc("file:///C:/Users/ashekhar/Desktop/Mark Logic/Shakespeare/*.x=
>ml")
>
>Can you please help me understand?

Perhaps it is because you've used a singleton access function (the 
doc() function returns a single root node for the one document you 
are addressing) with a wildcard "*.xml" URI implying multiple documents.

If you want multiple documents returned with a single call, use the 
collection() function, but ensure first that the syntax you use in 
your URI is supported by the vendor of the XQuery tool.  All URI 
strings for accessing resources are implementation defined and not 
standardized.

I hope this helps.

. . . . . . . . . . . . Ken

--
Upcoming hands-on code list, UBL, XSLT, XQuery and XSL-FO classes.
Interested in other classes?  http://www.CraneSoftwrights.com/q/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/q/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:http://x-query.com/mailman/listinfo/talk
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/q/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

using a variable within a regexp pattern in XQuery

2009-10-01T16:40:28Z

you haven't said what you want to occur.
But I am guessing maybe this is what you are looking for

*    where $record/langSet[@xml:lang="en"]/tig/term[matches(., 
fn:concat("^",$q,".*$"))] *

David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Manuel Souto Pico wrote:
> Hi,
>
> I'm a bit stuck trying to run a regexp pattern in a XQuery search file.
>
> (1) where $record/langSet[@xml:lang="en"]/tig/term[matches(., 
> $q)]          works
> (2) where $record/langSet[@xml:lang="en"]/tig/term[matches(., 
> "$q")]            doesn't work
> (3) where $record/langSet[@xml:lang="en"]/tig/term[matches(., 
> "^$q.*$")]         doesn't work
> (4) where $record/langSet[@xml:lang="en"]/tig/term[matches(., 
> "^bu.+$")]          works!
>
> Any ideas about how I can get to use regular expressions in this query?
>
> (4) retrieves 'bufer', 'bug' and 'bus'. If $q is 'bu', (1) retrieves 
> as well 'attri*bu*te', 'contrast *bu*tton', etc.
>
> Thanks a lot!
>
> Cheers, Manuel
> ------------------------------------------------------------------------
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091001/861229e5/attachment.htm

FW: Unexpected token syntax error, unexpected DeclareFunction_

2009-10-01T16:18:09Z

Thanks Ken.  It worked.  I did learn something new today.

I am able to successfully read from a database.  However if I add the argument to doc() to read files from the file system I do not see anything being displayed - blank page.

for $d in doc("file:///C:/Users/ashekhar/Desktop/Mark Logic/Shakespeare/*.xml")

Can you please help me understand?
 Thanks
Anil






> Date: Wed, 30 Sep 2009 21:53:30 -0400
> To: http://x-query.com/mailman/listinfo/talk
> From: http://x-query.com/mailman/listinfo/talk
> Subject: Re:  Unexpected token syntax error, unexpected	DeclareFunction_
> CC: 
> 
> At 2009-09-30 18:13 -0700, Anil Shekhar wrote:
> >XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, 
> >unexpected DeclareFunction_
> 
> The message says that your declaration of a function is unexpected.
> 
> >in /shakespeare.xqy, on line 8 [1.0-ml]Line 8 is where I have 
> >defined the function.
> 
> Sure enough, that isn't an expected place for a function definition.
> 
> Reviewing:  http://www.w3.org/TR/xquery/#id-function-calls
> 
> >3.1.5 Function Calls
> >[Definition: The built-in functions supported by XQuery are defined 
> >in [XQuery 1.0 and XPath 2.0 Functions and Operators].] Additional 
> >functions may be declared in a Prolog, imported from a library 
> >module, or provided by the external environment as part of the static context.
> 
> And that is pretty explicit that functions can only be declared in the prolog.
> 
> You have:
> 
> >
> >     
> >         Shakespeare ...
> >     
> >     
> >         
> >         {
> >             declare function local:handle-doc(
> >                 $doc      as document-node(),
> >                 $speeches as element(SPEECH)+
> >             )
> 
> .... which doesn't have the declaration in the prolog, but in the 
> middle of the query.
> 
> So I think the error message is quite justified.
> 
> I hope this helps.
> 
> . . . . . . . . . . Ken
> 
> --
> Upcoming hands-on code list, UBL, XSLT, XQuery and XSL-FO classes.
> Interested in other classes?  http://www.CraneSoftwrights.com/q/i/
> Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/q/
> Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
> Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
> Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
> G. Ken Holman                 mailto:http://x-query.com/mailman/listinfo/talk
> Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/q/bc
> Legal business disclaimers:  http://www.CraneSoftwrights.com/legal
> 
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
 		 	   		  
_________________________________________________________________
Insert movie times and more without leaving Hotmail®.
http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20091001/eb054f0b/attachment.htm

Unexpected token syntax error, unexpected DeclareFunction_

2009-10-01T12:50:41Z

Anil,

Since this is a question specific to MarkLogic Server, I recommend 
joining the mailing list at 
http://developer.marklogic.com/discuss/subscribe.xqy?list=general

You can also search the list: for example, try 
http://marklogic.markmail.org/search/?q=XDMP-UNEXPECTED+1.0-ml (the 
second hit is relevant: http://markmail.org/thread/2nuwbppxeemsrpjs).

The problem you're running into is that the shakespeare demo code is 
somewhat old, and was written in a custom dialect that is not quite 
XQuery 1.0. To use this older demo code, set the "default xquery 
version" for your app server (the one configured to listen on port 8010) 
to "0.9-ml".

-- Mike

On 2009-09-30 18:13, Anil Shekhar wrote:
> Dear Experts,
>
> I am new to XQuery and I need your help.  I am getting the following error when accessed ...
>
> http://127.0.0.1:8010/shakespeare.xqy
> 500 Internal Server Error
>
> XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, unexpected DeclareFunction_
>
> in /shakespeare.xqy, on line 8 [1.0-ml]
>
> Line 8 is where I have defined the function.
>
> Thanks
> Anil
>
>
>
> 
>      
>          Shakespeare ...
>      
>      
>          
>          {
>              declare function local:handle-doc(
>                  $doc      as document-node(),
>                  $speeches as element(SPEECH)+
>              )
>              {
>                  
{ document-uri($doc) }
,
>                  for $speech  in $speeches
>                  let $speaker := $speech/SPEAKER
>                  let $lines   := $speech/LINE
>                  return (
>                      { $speaker/text() },
>                      { $lines/text() }
>                  )
>              };
>
>              for $d in doc("file:///C:/Users/ashekhar/Desktop/Mark Logic/Shakespeare/*.xml")
>                  order by $d ascending
>                  return local:handle-doc($d, $d//SPEECH)
>          }
>          
>      
> )
>
>
>
>
>
>
>
> ________________________________
> Insert movie times and more without leaving Hotmail®. See how.
>

Unexpected token syntax error, unexpected DeclareFunction_

2009-09-30T22:53:30Z

At 2009-09-30 18:13 -0700, Anil Shekhar wrote:
>XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, 
>unexpected DeclareFunction_

The message says that your declaration of a function is unexpected.

>in /shakespeare.xqy, on line 8 [1.0-ml]Line 8 is where I have 
>defined the function.

Sure enough, that isn't an expected place for a function definition.

Reviewing:  http://www.w3.org/TR/xquery/#id-function-calls

>3.1.5 Function Calls
>[Definition: The built-in functions supported by XQuery are defined 
>in [XQuery 1.0 and XPath 2.0 Functions and Operators].] Additional 
>functions may be declared in a Prolog, imported from a library 
>module, or provided by the external environment as part of the static context.

And that is pretty explicit that functions can only be declared in the prolog.

You have:

>
>     
>         Shakespeare ...
>     
>     
>         
>         {
>             declare function local:handle-doc(
>                 $doc      as document-node(),
>                 $speeches as element(SPEECH)+
>             )

.... which doesn't have the declaration in the prolog, but in the 
middle of the query.

So I think the error message is quite justified.

I hope this helps.

. . . . . . . . . . Ken

--
Upcoming hands-on code list, UBL, XSLT, XQuery and XSL-FO classes.
Interested in other classes?  http://www.CraneSoftwrights.com/q/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/q/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:http://x-query.com/mailman/listinfo/talk
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/q/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

Unexpected token syntax error, unexpected DeclareFunction_

2009-09-30T19:13:51Z

Dear Experts,

I am new to XQuery and I need your help.  I am getting the following error when accessed ...

http://127.0.0.1:8010/shakespeare.xqy
500 Internal Server Error
      XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, unexpected DeclareFunction_
in /shakespeare.xqy, on line 8 [1.0-ml]
Line 8 is where I have defined the function.

Thanks
Anil




    
        Shakespeare ...
    
    
        
        {
            declare function local:handle-doc(
                $doc      as document-node(),
                $speeches as element(SPEECH)+
            )
            {
                
{ document-uri($doc) },
                for $speech  in $speeches
                let $speaker := $speech/SPEAKER
                let $lines   := $speech/LINE
                return (
                    { $speaker/text() },
                    { $lines/text() }
                )
            };

            for $d in doc("file:///C:/Users/ashekhar/Desktop/Mark Logic/Shakespeare/*.xml")
                order by $d ascending
                return local:handle-doc($d, $d//SPEECH)
        }
        
    
)









 		 	   		  
_________________________________________________________________
Insert movie times and more without leaving Hotmail®.
http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090930/246e6c1b/attachment.htm

UTF-8

2009-09-29T21:23:12Z

"UTF-8" is default value for encoding attribute in xml prolog

On Tue, Sep 29, 2009 at 8:14 PM, Jesus Rodrigues
 wrote:
> Dear,
>
> I needed read a lot of xml and some of this file haven't the attribute
> encoding="UTF-8". And some xml has too a problem with utf-8 (invalid
> character like this GA�?S). I haven't control about the xml I just needed
> read.
>
> Is possible solution this problem using the saxon? For example, setting the
> encoding dynamically, before the build (newDocumentBuilder().build(stream))?
>
> Regards,
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

UTF-8

2009-09-29T18:48:42Z

If the XML document has an encoding declaration () then you must ensure that the file is actually encoded using this encoding.
 
If the XML document has no encoding declaration, then it should typically be encoded in UTF-8 (though there are other ways of indicating the encoding, for example using byte-order-marks, or in an HTTP header). 
 
The most common cause of encoding problems is when you use an encoding such as iso-8859-1 for your document, but have no encoding declaration on the XML document.
 
If you know the encoding of your document, you can always decode it into a character stream yourself, and then pass the character stream to the XML parser (e.g. in Java in the form of a Reader).
 
Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

 


  _____  

From: http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf Of Jesus Rodrigues
Sent: 29 September 2009 17:15
To: http://x-query.com/mailman/listinfo/talk
Subject:  UTF-8



Dear,




I needed read a lot of xml and some of this file haven't the attribute encoding="UTF-8". And some xml has too a problem with utf-8 (invalid character like this GA�?S). I haven't control about the xml I just needed read.




Is possible solution this problem using the saxon? For example, setting the encoding dynamically, before the build (newDocumentBuilder().build(stream))?




Regards,




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090929/160019a2/attachment.htm

UTF-8

2009-09-29T14:35:50Z

Dear Vyacheslav Sedov

However, sometime the file that I'm try to read wasn't a UFT-8.

is it possible set the encoding dynamically? Or it's necessary open the file
and convert to UFT-8?


Regards


2009/9/29 Vyacheslav Sedov 

> "UTF-8" is default value for encoding attribute in xml prolog
>
> On Tue, Sep 29, 2009 at 8:14 PM, Jesus Rodrigues
>  wrote:
> > Dear,
> >
> > I needed read a lot of xml and some of this file haven't the attribute
> > encoding="UTF-8". And some xml has too a problem with utf-8 (invalid
> > character like this GA�?S). I haven't control about the xml I just needed
> > read.
> >
> > Is possible solution this problem using the saxon? For example, setting
> the
> > encoding dynamically, before the build
> (newDocumentBuilder().build(stream))?
> >
> > Regards,
> >
> > _______________________________________________
> > http://x-query.com/mailman/listinfo/talk
> > http://x-query.com/mailman/listinfo/talk
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090929/155eccb0/attachment.htm

UTF-8

2009-09-29T14:14:31Z

Dear,


 I needed read a lot of xml and some of this file haven't the attribute
encoding=*"UTF-8"*. And some xml has too a problem with utf-8 (invalid
character like this GA�?S). I haven't control about the xml I just needed
read.


 Is possible solution this problem using the saxon? For example, setting the
encoding dynamically, before the build (newDocumentBuilder().build(stream))?


 Regards,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090929/5c8b6218/attachment.htm

Re: Combining XSLT 2.0's Grouping with eXist

2009-09-28T12:21:10Z

Bill,

Yes eXist provides mechanisms for transforming directly using XSLT
without the need for Java -
1) XQuery extension function transform:transform(...) which Andrew
already mentioned.
2) Some sort of custom pipeline using our Controller XQuery approach
3) XProc pipeline using our XProcXQ
4) XSL-PI


Probably best if you join the eXist mailing list for such questions.

2009/9/28 Andrew Welch :
> Hi Bill,
>
> The best thing to do is sign up to one of the mailing lists:
>
> "xquery talk" http://x-query.com/mailman/listinfo/talk
>
> "xml dev" http://www.xml.org/xml-dev/
>
> Ask your question there, hopefully get a helpful response, and then
> the email chain gets archived so its useful for anyone encountering a
> similar problem in the future.
>
> The short answer is yes - most xquery engines provide a way of
> transforming a the result of a query through a proprietary extension,
> the one for eXist is documented here:
>
> http://exist.sourceforge.net/xquery.html#N10375
>
> cheers
> andrew
>
> 2009/9/26 Bill Girten :
>> Andrew,
>>
>> Thanks for the piece you did on this area.  I'm looking for a little
>> clarity, however.
>>
>> I'm an IT consultant that's worked with BEA's Aqualogic Data Services
>> XQuery engine in the past.  I've been playing with eXist 1.26 and now
>> the 1.4RC.  Here's my question:
>>
>> Is it worth looking into the Sitemap stuff (missed that revolution
>> years ago) since Wolgang Meir is deprecating a lot of the Cocoon gear?
>>  Although I love Java, I want to get away from XSL transformation via
>> calling on the process via a Java class.  Is there a way to seamlessly
>> transform the eXist results set without the need of running the XML
>> through Java to get XHTML?
>>
>> Sorry if I sound like a newbie.
>>
>> Thanks,
>>
>> Bill Girten
>>
>
>
>
> --
> Andrew Welch
> http://andrewjwelch.com
> Kernow: http://kernowforsaxon.sf.net/
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>



-- 
Adam Retter

skype :adam.retter
http://www.adamretter.org.uk

Re: Combining XSLT 2.0's Grouping with eXist

2009-09-28T10:24:02Z

Hi Bill,

The best thing to do is sign up to one of the mailing lists:

"xquery talk" http://x-query.com/mailman/listinfo/talk

"xml dev" http://www.xml.org/xml-dev/

Ask your question there, hopefully get a helpful response, and then
the email chain gets archived so its useful for anyone encountering a
similar problem in the future.

The short answer is yes - most xquery engines provide a way of
transforming a the result of a query through a proprietary extension,
the one for eXist is documented here:

http://exist.sourceforge.net/xquery.html#N10375

cheers
andrew

2009/9/26 Bill Girten :
> Andrew,
>
> Thanks for the piece you did on this area.  I'm looking for a little
> clarity, however.
>
> I'm an IT consultant that's worked with BEA's Aqualogic Data Services
> XQuery engine in the past.  I've been playing with eXist 1.26 and now
> the 1.4RC.  Here's my question:
>
> Is it worth looking into the Sitemap stuff (missed that revolution
> years ago) since Wolgang Meir is deprecating a lot of the Cocoon gear?
>  Although I love Java, I want to get away from XSL transformation via
> calling on the process via a Java class.  Is there a way to seamlessly
> transform the eXist results set without the need of running the XML
> through Java to get XHTML?
>
> Sorry if I sound like a newbie.
>
> Thanks,
>
> Bill Girten
>



-- 
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

Can FLWOR be FLOWR?

2009-09-24T17:39:11Z

Schwartz, Christine wrote:
> I’m helping teach some basic XQuery and was asked this question: In a
> FLWOR expression, does the where clause have to come before an order by
> clause?
> 
> Seems like it should, but can’t seem to find this written down anywhere.

The order of the clauses is fixed for XQuery 1.0, with "order by" coming
after "where".

In XQuery 1.1 this restriction will be relaxed, allowing clauses in
almost any order (you have to start with "for", "let" etc.) and multiple
"order by" or "where" clauses.

John

-- 
John Snelson, Oracle Corporation            http://snelson.org.uk/john
Berkeley DB XML:            http://oracle.com/database/berkeley-db/xml
XQilla:                                  http://xqilla.sourceforge.net

Can FLWOR be FLOWR?

2009-09-24T17:27:16Z

Ken and John,

Thanks for your help with this. It's interesting to know that XQuery 1.1 will be changing this required order.

Regards,
 
Chris
 
 
> Snelson, John wrote:
> 
> Schwartz, Christine wrote:
> > I’m helping teach some basic XQuery and was asked this question: In a
> > FLWOR expression, does the where clause have to come before an order
> by
> > clause?
> >
> > Seems like it should, but can’t seem to find this written down
> anywhere.
> 
> The order of the clauses is fixed for XQuery 1.0, with "order by"
> coming
> after "where".
> 
> In XQuery 1.1 this restriction will be relaxed, allowing clauses in
> almost any order (you have to start with "for", "let" etc.) and
> multiple
> "order by" or "where" clauses.
> 
> John
>

Can FLWOR be FLOWR?

2009-09-24T12:18:33Z

At 2009-09-24 11:08 -0400, Schwartz, Christine wrote:
>I'm helping teach some basic XQuery and was asked this question: In 
>a FLWOR expression, does the where clause have to come before an 
>order by clause?

Yes, because it qualifies the set that is to be ordered.

>Seems like it should, but can't seem to find this written down anywhere.

Production [33] indicates that the where clause precedes the order by clause:

http://www.w3.org/TR/2007/REC-xquery-20070123/#id-flwor-expressions

I hope this helps.

. . . . . . Ken


--
Upcoming hands-on code list, UBL, XSLT, XQuery and XSL-FO classes.
Interested in other classes?  http://www.CraneSoftwrights.com/q/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/q/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:http://x-query.com/mailman/listinfo/talk
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/q/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

Can FLWOR be FLOWR?

2009-09-24T12:08:55Z

Hi, 

 

I'm helping teach some basic XQuery and was asked this question: In a
FLWOR expression, does the where clause have to come before an order by
clause?

 

Seems like it should, but can't seem to find this written down anywhere.

 

Thanks,

 

Chris

 

Christine Schwartz

Metadata Librarian

Princeton Theological Seminary Libraries

http://x-query.com/mailman/listinfo/talk

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090924/84cc4959/attachment.htm

Re: [xml-dev] Serialization of XDM - Use cases /Proposal

2009-09-19T00:00:53Z

Good start.

The next section before you go into specification should perhaps be
"Requirements", including assumptions and constraints. This might include
statements such as:

* Serialized XDM will retain information about the descendants of nodes in
the sequence being serialized, but it will not retain information about
their ancestors.

* Serialized XDM will not retain information about node identity: that is,
the recipient of the serialized XDM will not be able to determine whether
two serialized elements originated from the same node or merely from two
nodes that were deep-equal to each other.

* The consumer of the serialized XDM is assumed to have access to the same
schema as the producer of the serialized XDM: that is, a QName identifying a
type is assumed to have the same meaning to both the producer and consumer.

* If A and B are two XDM sequences, then xml-canonicalize(xdm-serialize(A)))
is codepoint-equal to xml-canonicalize(xdm-serialize(B))) if and only if
fn:deep-equal(A, B). [Actually, that's probably not a good requirement,
because fn:deep-equal() discards comments and PIs and you probably don't
want to do that].

These are just example statements, you might want to substitute different
ones: they are just intended to illustrate the kind of thing that needs to
be said.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

> -----Original Message-----
> From: http://x-query.com/mailman/listinfo/talk 
> [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf Of David A. Lee
> Sent: 18 September 2009 22:07
> Cc: http://x-query.com/mailman/listinfo/talk; http://x-query.com/mailman/listinfo/talk; XProc Dev
> Subject:  Re: [xml-dev] Serialization of XDM - 
> Use cases /Proposal
> 
> I have created a first pass at documenting the problem of XDM 
> Serialization and created some use cases.
> I would love any feedback or comments.  This is on a new wiki 
> I created for this purpose.  If you would like to comment 
> directly on the wiki please reply to me and I will give you 
> the invite code (due to the sad state of affairs anonymous 
> comments and editing are disabled due to wiki-spam-bots.  
> I've found spam within 5 minutes of opening a public wiki ...  )
> 
> 
> http://xml.calldei.com/XDMSerialize
> 
> I have NOT included a proposal for a format yet, I'd like to 
> discuss the intent and use cases first before putting up a 
> straw-man proposal.
> 
> Thank you for any contribution !
> 
> I've CC'd this to xproc-dev because one of the use cases if 
> for developers and integrators with XML Pipeline processors 
> such as XProc
> 
> 
> David A. Lee
> http://x-query.com/mailman/listinfo/talk  
> http://www.calldei.com
> http://www.xmlsh.org
> 812-482-5224
> 
> 
> 
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk

[xml-dev] RE: Re: [xml-dev] Serialization of XDM - Use cases /Proposal

2009-09-18T21:06:51Z

Thanks Michael.  I think my next step will be to document what 
attributes of XDM should be maintained through serialization and what 
will be dropped.
I agree that Node Identity is not something that can reasonably be 
maintained.
Ancestry is an interesting concept I hadn't considered ...
e.g. if you serialize a node $node ... its not expected that a 
reconstituted XDM  $node could evaluate  $node/..

Schema equivilence is interesting, I was thinking of addressing (or 
punting !) that concept with type information.
Ignoring (or assuming) common xsd:* types ...

If for example a value is of type   mytype:abc  ... I think agree its a 
good presumption that the target environment has access to the same types.
Otherwise we open the box of having to serialize all the type 
information along with the XDM values. Good call.




David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Michael Kay wrote:
> Good start.
>
> The next section before you go into specification should perhaps be
> "Requirements", including assumptions and constraints. This might include
> statements such as:
>
> * Serialized XDM will retain information about the descendants of nodes in
> the sequence being serialized, but it will not retain information about
> their ancestors.
>
> * Serialized XDM will not retain information about node identity: that is,
> the recipient of the serialized XDM will not be able to determine whether
> two serialized elements originated from the same node or merely from two
> nodes that were deep-equal to each other.
>
> * The consumer of the serialized XDM is assumed to have access to the same
> schema as the producer of the serialized XDM: that is, a QName identifying a
> type is assumed to have the same meaning to both the producer and consumer.
>
> * If A and B are two XDM sequences, then xml-canonicalize(xdm-serialize(A)))
> is codepoint-equal to xml-canonicalize(xdm-serialize(B))) if and only if
> fn:deep-equal(A, B). [Actually, that's probably not a good requirement,
> because fn:deep-equal() discards comments and PIs and you probably don't
> want to do that].
>
> These are just example statements, you might want to substitute different
> ones: they are just intended to illustrate the kind of thing that needs to
> be said.
>
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay 
>
>   
>> -----Original Message-----
>> From: http://x-query.com/mailman/listinfo/talk 
>> [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf Of David A. Lee
>> Sent: 18 September 2009 22:07
>> Cc: http://x-query.com/mailman/listinfo/talk; http://x-query.com/mailman/listinfo/talk; XProc Dev
>> Subject:  Re: [xml-dev] Serialization of XDM - 
>> Use cases /Proposal
>>
>> I have created a first pass at documenting the problem of XDM 
>> Serialization and created some use cases.
>> I would love any feedback or comments.  This is on a new wiki 
>> I created for this purpose.  If you would like to comment 
>> directly on the wiki please reply to me and I will give you 
>> the invite code (due to the sad state of affairs anonymous 
>> comments and editing are disabled due to wiki-spam-bots.  
>> I've found spam within 5 minutes of opening a public wiki ...  )
>>
>>
>> http://xml.calldei.com/XDMSerialize
>>
>> I have NOT included a proposal for a format yet, I'd like to 
>> discuss the intent and use cases first before putting up a 
>> straw-man proposal.
>>
>> Thank you for any contribution !
>>
>> I've CC'd this to xproc-dev because one of the use cases if 
>> for developers and integrators with XML Pipeline processors 
>> such as XProc
>>
>>
>> David A. Lee
>> http://x-query.com/mailman/listinfo/talk  
>> http://www.calldei.com
>> http://www.xmlsh.org
>> 812-482-5224
>>
>>
>>
>> _______________________________________________
>> http://x-query.com/mailman/listinfo/talk
>> http://x-query.com/mailman/listinfo/talk
>>     
>
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: http://x-query.com/mailman/listinfo/talk
> subscribe: http://x-query.com/mailman/listinfo/talk
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090918/f2f7a03b/attachment.htm

Re: [xml-dev] Serialization of XDM - Use cases / Proposal

2009-09-18T18:07:21Z

I have created a first pass at documenting the problem of XDM 
Serialization and created some use cases.
I would love any feedback or comments.  This is on a new wiki I created 
for this purpose.  If you would like to comment directly on the wiki 
please reply to me and I will give you the invite code (due to the sad 
state of affairs anonymous comments and editing are disabled due to 
wiki-spam-bots.  I've found spam within 5 minutes of opening a public 
wiki ...  )


http://xml.calldei.com/XDMSerialize

I have NOT included a proposal for a format yet, I'd like to discuss the 
intent and use cases first before putting up a straw-man proposal.

Thank you for any contribution !

I've CC'd this to xproc-dev because one of the use cases if for 
developers and integrators with XML Pipeline processors such as XProc


David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Local namespace

2009-09-16T14:55:47Z

Florent Georges wrote:
> John Snelson wrote:
> 
>> Yes - that's what I originally suggested. However the XQuery
>> working group felt it was important to be able change a function
>> to be exported / not-exported without changing every place that
>> function is called.
> 
>   Thanks, John and Martin, for those helpful precisions.  BTW, speaking of XQuery 1.1, is there any estimation about when a first draft would be available?

It's available now, although that draft pretty much only has grouping
and windowing in it.

http://www.w3.org/TR/xquery-11/

The WG is actively working to get a more recent draft out, which should
contain a lot of the latest features we've added.

John

-- 
John Snelson, Oracle Corporation            http://snelson.org.uk/john
Berkeley DB XML:            http://oracle.com/database/berkeley-db/xml
XQilla:                                  http://xqilla.sourceforge.net

Local namespace

2009-09-16T12:54:47Z

John Snelson wrote:

> Yes - that's what I originally suggested. However the XQuery
> working group felt it was important to be able change a function
> to be exported / not-exported without changing every place that
> function is called.

  Thanks, John and Martin, for those helpful precisions.  BTW, speaking of XQuery 1.1, is there any estimation about when a first draft would be available?

  Regards,

-- 
Florent Georges
http://www.fgeorges.org/

Fwd: xmlns ignore

2009-09-15T15:35:25Z

You clearly didn't understand David's answer.
 
You can no more "ignore" a namespace declaration in your source document
than you can ignore the first character of each element name. The namespace
is an intrinsic part of the names of your elements, and when selecting
elements by name, you need to be aware of it.
 
If you want to use unqualified names in your XQuery expression to refer to
names in a namespace, you can do this by means of a default namespace
declaration in the XQuery prolog: 
 
declare default element namespace " 
http://www.portalfiscal.inf.br/nfe"
 
However, you need to be aware that this affects the interpretation of all
unprefixed names in your query, not only those in a path expression.
 
If you don't want to change the source of the query, you can achieve the
same effect using the Saxon API:
 
comp.declareNamespace("", " 
http://www.portalfiscal.inf.br/nfe");
 
Your example query will then work as written. Note, however, that this is a
very different thing from "ignoring" the namespace.
 
Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 


  _____  

From: http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk] On Behalf
Of Sérgio Mariano Dias
Sent: 15 September 2009 14:07
To: http://x-query.com/mailman/listinfo/talk
Subject:  Fwd: xmlns ignore


 

Hi,  

This is a complete code test:

         
//--------------------------------------------------------------------------
------------------------------------------------------- 

          Processor proc = new Processor(false);
     
          
          XQueryCompiler comp = proc.newXQueryCompiler();
     
          XdmNode source = proc.newDocumentBuilder().build(new
StreamSource(inputFile));
     
     
          

          XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp"); 

          XQueryEvaluator eval = exp.load();
          eval.setContextItem(source);        
         
         
          

          XdmItem i =eval.evaluateSingle();
          System.out.println(i.getStringValue()); 

          
          
          XdmValue v = eval.evaluate(); 
          System.out.println(v.size());


//--------------------------------------------------------------------------
-------------------------------------------------------

that is the xml file:


//--------------------------------------------------------------------------
-------------------------------------------------------
encoding="UTF-8"?>

        
            
                29
                123658
                Venda - direta
....

//--------------------------------------------------------------------------
-------------------------------------------------------

The ""http://www.portalfiscal.inf.br/nfe"" return page not found. If I
remove the addres, the program return the correct value. That is way a fink
to ignore the addres. Is that possible?

Regards,





2009/9/11 David Carlisle  




> I have a problem to ignore a xmls attribute in a tag and did not found
> information how to ignore this attribute. Somebody can help me.


A namespace declaration is not an attribute in the XDM model used by
XPath and Xquery. The name of an element includes the namespace.

Most likely the part of teh code that you don't show

= comp.compile("/.../... ...");

is selecting elements in no-namespace, and you should change it to
select elements in the namespace used in your xml source.

either by using a prefix
/p:aa/p:bb

and binding p to the namespace used in the source, or by setting the
default namespace in the xquery to that used in the source.


David


________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs.
________________________________________________________________________





-- 
Sérgio Mariano  






-- 
Sérgio Mariano 




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090915/73c0cba1/attachment.htm

Fwd: xmlns ignore

2009-09-15T15:29:00Z

>           XQueryCompiler comp = proc.newXQueryCompiler();
>
>           XdmNode source = proc.newDocumentBuilder().build(new
> StreamSource(inputFile));
>
>
>
>           XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp");

[snip]

> encoding="UTF-8"?>
> 
>         

Either change your query to include the namespace eg:

declare namespace f = "http://www.portalfiscal.inf.br/nfe";

with

/f:NFe/f:infNFe/f:ide/f:natOp

or declare the namespace on the XQueryCompiler:

http://www.saxonica.com/documentation/dotnetdoc/Saxon/Api/XQueryCompiler.html


-- 
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

RE: [xml-dev] Serialization of XDM

2009-09-15T15:26:43Z

> > There is no standard or even 'community accepted' way of 
> serializing XDM

> What is wrong with JSON for this?

Nearly everything, starting with the fact that the target user community is
likely to want something that can be manipulated using XML tools. 

In fact, what is right with JSON for this?
> 
> The XPath2 data model is a system for *converting* XML into 
> programming language types, or for operating on a kind of XSD 
> PSVI. 

That's not how I would define it (and it's not how the spec defines it). In
fact, I don't really see where conversion into "programming language types"
comes into it at all.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay

Serialization of XDM

2009-09-15T15:10:46Z

but nice idea "keep it XML"
http://kixml.blogspot.com/2009/09/serving-xml_04.html
it can help to combine XQuery&XSLT into one engine

for example how eXist and Saxon perform exchange? xml as string?

On Tue, Sep 15, 2009 at 10:58 AM, Michael Kay  wrote:
>>
>> > is it important that
>> >     $a[1]/foo/bar  is  $a[2]
>>
>
> Someone needs to sit down and write down the requirements and use cases.
>
> One requirement that someone suggested was of the nature of "lossless
> round-tripping". That would imply that given a sequence (N, M), not only
> does (N is M) retain its value after serialization and parsing, but so does
> (N/.. is M/..): which means you can't just serialize the subtrees rooted at
> the nodes in the sequence, you have to serialize the entire document in
> which these nodes are contained.
>
> Perhaps this explains why there is currently no standard in this area.
>
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Fwd: xmlns ignore

2009-09-15T11:44:30Z

Ok now it's correct.
 thank you very much by the help.

2009/9/15 Michael Kay 

>  You clearly didn't understand David's answer.
>
> You can no more "ignore" a namespace declaration in your source document
> than you can ignore the first character of each element name. The namespace
> is an intrinsic part of the names of your elements, and when selecting
> elements by name, you need to be aware of it.
>
> If you want to use unqualified names in your XQuery expression to refer to
> names in a namespace, you can do this by means of a default namespace
> declaration in the XQuery prolog:
>
> declare default element namespace "http://www.portalfiscal.inf.br/nfe"
>
> However, you need to be aware that this affects the interpretation of all
> unprefixed names in your query, not only those in a path expression.
>
> If you don't want to change the source of the query, you can achieve the
> same effect using the Saxon API:
>
> comp.declareNamespace("", "http://www.portalfiscal.inf.br/nfe");
>
> Your example query will then work as written. Note, however, that this is a
> very different thing from "ignoring" the namespace.
>
>
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay
>
>  ------------------------------
> *From:* http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk] *On
> Behalf Of *Sérgio Mariano Dias
> *Sent:* 15 September 2009 14:07
> *To:* http://x-query.com/mailman/listinfo/talk
> *Subject:*  Fwd: xmlns ignore
>
>
> Hi,
>
> This is a complete code test:
>
>
> //---------------------------------------------------------------------------------------------------------------------------------
>
>           Processor proc = new Processor(false);
>
>
>           XQueryCompiler comp = proc.newXQueryCompiler();
>
>           XdmNode source = proc.newDocumentBuilder().build(new
> StreamSource(inputFile));
>
>
>
>           XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp");
>           XQueryEvaluator eval = exp.load();
>           eval.setContextItem(source);
>
>
>
>           XdmItem i =eval.evaluateSingle();
>           System.out.println(i.getStringValue());
>
>
>           XdmValue v = eval.evaluate();
>           System.out.println(v.size());
>
>
> //---------------------------------------------------------------------------------------------------------------------------------
>
> that is the xml file:
>
>
>
> //---------------------------------------------------------------------------------------------------------------------------------
> encoding="UTF-8"?>
> 
>         
>             
>                 29
>                 123658
>                 Venda - direta
> ....
>
>
> //---------------------------------------------------------------------------------------------------------------------------------
>
> The ""http://www.portalfiscal.inf.br/nfe"" return page not found. If I
> remove the addres, the program return the correct value. That is way a fink
> to ignore the addres. Is that possible?
>
> Regards,
>
>
>
>
> 2009/9/11 David Carlisle 
>
>
>>
>> > I have a problem to ignore a xmls attribute in a tag and did not found
>> > information how to ignore this attribute. Somebody can help me.
>>
>> A namespace declaration is not an attribute in the XDM model used by
>> XPath and Xquery. The name of an element includes the namespace.
>>
>> Most likely the part of teh code that you don't show
>>
>> = comp.compile("/.../... ...");
>>
>> is selecting elements in no-namespace, and you should change it to
>> select elements in the namespace used in your xml source.
>>
>> either by using a prefix
>> /p:aa/p:bb
>>
>> and binding p to the namespace used in the source, or by setting the
>> default namespace in the xquery to that used in the source.
>>
>>
>> David
>>
>>
>> ________________________________________________________________________
>> The Numerical Algorithms Group Ltd is a company registered in England
>> and Wales with company number 1249803. The registered office is:
>> Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.
>>
>> This e-mail has been scanned for all viruses by Star. The service is
>> powered by MessageLabs.
>> ________________________________________________________________________
>>
>
>
>
> --
> Sérgio Mariano
>
>
>
>
>
> --
> Sérgio Mariano
>
>
>


-- 
Sérgio Mariano Dias
http://www.dcc.ufmg.br/~mariano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090915/1fe901ce/attachment-0001.htm

Fwd: xmlns ignore

2009-09-15T11:07:15Z

Hi,

This is a complete code test:


//---------------------------------------------------------------------------------------------------------------------------------
          Processor proc = new Processor(false);


          XQueryCompiler comp = proc.newXQueryCompiler();

          XdmNode source = proc.newDocumentBuilder().build(new
StreamSource(inputFile));



          XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp");
          XQueryEvaluator eval = exp.load();
          eval.setContextItem(source);



          XdmItem i =eval.evaluateSingle();
          System.out.println(i.getStringValue());


          XdmValue v = eval.evaluate();
          System.out.println(v.size());

//---------------------------------------------------------------------------------------------------------------------------------

that is the xml file:


//---------------------------------------------------------------------------------------------------------------------------------
encoding="UTF-8"?>

        
            
                29
                123658
                Venda - direta
....

//---------------------------------------------------------------------------------------------------------------------------------

The ""http://www.portalfiscal.inf.br/nfe"" return page not found. If I
remove the addres, the program return the correct value. That is way a fink
to ignore the addres. Is that possible?

Regards,




2009/9/11 David Carlisle 


>
> > I have a problem to ignore a xmls attribute in a tag and did not found
> > information how to ignore this attribute. Somebody can help me.
>
> A namespace declaration is not an attribute in the XDM model used by
> XPath and Xquery. The name of an element includes the namespace.
>
> Most likely the part of teh code that you don't show
>
> = comp.compile("/.../... ...");
>
> is selecting elements in no-namespace, and you should change it to
> select elements in the namespace used in your xml source.
>
> either by using a prefix
> /p:aa/p:bb
>
> and binding p to the namespace used in the source, or by setting the
> default namespace in the xquery to that used in the source.
>
>
> David
>
>
> ________________________________________________________________________
> The Numerical Algorithms Group Ltd is a company registered in England
> and Wales with company number 1249803. The registered office is:
> Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.
>
> This e-mail has been scanned for all viruses by Star. The service is
> powered by MessageLabs.
> ________________________________________________________________________
>



-- 
Sérgio Mariano





-- 
Sérgio Mariano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090915/5063b3b0/attachment-0001.htm

Serialization of XDM

2009-09-15T08:58:05Z

> 
> > is it important that
> >     $a[1]/foo/bar  is  $a[2]
> 

Someone needs to sit down and write down the requirements and use cases.

One requirement that someone suggested was of the nature of "lossless
round-tripping". That would imply that given a sequence (N, M), not only
does (N is M) retain its value after serialization and parsing, but so does
(N/.. is M/..): which means you can't just serialize the subtrees rooted at
the nodes in the sequence, you have to serialize the entire document in
which these nodes are contained.

Perhaps this explains why there is currently no standard in this area.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay

Serialization of XDM

2009-09-15T08:34:58Z

Excellent points.
IMHO, asking for 100% lossless is asking too much.
So if its not 100% where's the line ? A lossy implementation would still 
be very useful
If one were to start writing down intent, use cases and a proposal, what 
would be a good forum for this ?
A web page ? A document via email via these user groups ?
Suggestions welcome.

-David

David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Michael Kay wrote:
>>> is it important that
>>>     $a[1]/foo/bar  is  $a[2]
>>>       
>
> Someone needs to sit down and write down the requirements and use cases.
>
> One requirement that someone suggested was of the nature of "lossless
> round-tripping". That would imply that given a sequence (N, M), not only
> does (N is M) retain its value after serialization and parsing, but so does
> (N/.. is M/..): which means you can't just serialize the subtrees rooted at
> the nodes in the sequence, you have to serialize the entire document in
> which these nodes are contained.
>
> Perhaps this explains why there is currently no standard in this area.
>
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay 
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090915/b599c444/attachment.htm

Serialization of XDM

2009-09-15T03:45:48Z

> is it important that
>     $a[1]/foo/bar  is  $a[2]

well it's inportant if the next process in the pipeline needs to do any
kind of query processing, which is presumably the point of wanting to
pass an xdm instance in the first place.

if the sequence appears to the next process as $input then something as
simple as

count($input//bar)

is going to give different answers depending on whether identity is
preserved.

But in practice it's probably only important to know what the answer is
(whether identity is presrved or not) if you pass xml between processes
by serialising and reparsing then typically you end up with new
identities as far as xdm is concerned but this is rarely a problem in
practice.

so if the specification of the serialisation were to specify that you
were serialising a _copy_ of any XDM instance, so in particular every
item of the sequence always gets a new identity I think that would be OK
for the vast majority of cases.

David

________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

Announce: XQuery, XSLT, XPath and XSL-FO training in US (XQuery talk)

2009-09-15T01:38:28Z

Crane Softwrights Ltd. of Ottawa Canada is pleased to announce a US
east-coast (Washington DC/Falls Church VA area) training delivery of
its XML material in partnership with Mark Logic Corporation of San
Carlos California. Through four separately-subscribed classes over
11 consecutive weekdays in October/November 2009, these intensive
hands-on classes cover working with XML for the OASIS code list
specifications, the OASIS Universal Business Language 2.0 standard,
the W3C XQuery, XSLT and XPath 1.0 and 2.0 standards and the W3C
XSL-FO 1.0 and 1.1 standards:

Practical Code List Implementation (1 day): 2009-10-28
http://www.CraneSoftwrights.com/training/pcli/pclisyl.htm
Abbreviated code: PCLI
Cost: US$550.

Practical Universal Business Language Deployment (2 days): 2009-10-29/30
http://www.CraneSoftwrights.com/training/publd/publdsyl.htm
Abbreviated code: PUBLD
Cost: US$1100.

Practical Transformation Using XSLT, XQuery and XPath (5 days): 2009-11-02/06
http://www.CraneSoftwrights.com/training/ptuxq/ptuxqsyl.htm
Abbreviated code: PTUXQ
Cost: US$2750.

Practical Formatting Using XSL-FO (3 days): 2009-11-09/11
http://www.CraneSoftwrights.com/training/pfux/pfuxsyl.htm
Abbreviated code: PFUX
Cost: US$1650.

Venue:
Homewood Suites, Falls Church, VA (Merrifield) - http://tinyurl.com/lj548u
8130 Porter Road, Merrifield, Virginia, United States 22042
(daily lunch is included in the fee)

Registration for these classes is through Mark Logic's training division:

http://www.MarkLogic.com/services/training.html

and all students must register via email:

mailto:http://x-query.com/mailman/listinfo/talk?subject=Crane%20XML%20training%20Washington

Class size is limited to only 15 students, so please register early
to ensure you get your seat. Seats are allocated in the order of
paid registrations received, not email requests.

Check out all of our training offerings linked from our home page below.

We have plans for future classes and you can tell us if your interest in them:

http://www.CraneSoftwrights.com/forms/interest.php

Thanks!

. . . . . . . . . . Ken

cc: XQuery-Talk, XSL List, Yahoo XSL-FO, Antenna House list, RenderX
list, XML-Doc, XML-L

--
Interested in these classes? http://www.CraneSoftwrights.com/q/i/
Crane Softwrights Ltd. http://www.CraneSoftwrights.com/q/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson: http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview: http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman mailto:http://x-query.com/mailman/listinfo/talk
Male Cancer Awareness Nov'07 http://www.CraneSoftwrights.com/q/bc
Legal business disclaimers: http://www.CraneSoftwrights.com/legal

Serialization of XDM

2009-09-15T00:31:13Z


> On first glance I can imagine the former being fairly easy, but the 
> later much more complex.

The latter isn't a problem as they are different elements in the result,
the $a nodes are _copied_ into their parent as {$a} copy here
doesn't copy node identity.

David

________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

Serialization of XDM

2009-09-14T23:31:09Z

David A. Lee wrote:

  Hi David,

> let $a := 
> return
>    ( $a , $a , $a )
> --->  recognize all 3 are the same

> let $a := 
> return
>    (
> {$a},{$a},{$a})

> --> do we need to know that all 3  are the same ?

  They have the same value, but they are not identical (last
paragraph of the section "3.7.1 Direct Element Constructors"
w3.org/TR/xquery/#id-element-constructor>):

    The result of an element constructor is a new element node,
    with its own node identity. All the attribute and descendant
    nodes of the new element node are also new nodes with their
    own identities, even if they are copies of existing nodes.

  Regards,

-- 
Florent Georges
http://www.fgeorges.org/

Serialization of XDM

2009-09-14T22:16:22Z

> On the other hand, it seems to me that we already have such a 
> serialization format, and it's XQuery itself - or rather the 
> subset that involves literals, sequence "constructor" (i.e. 
> comma operator), and direct element/attribute/namespace/text 
> constructors. It's as portable as it gets - any XQuery 
> processor can immediately parse it.
> And it doesn't seem to be missing anything from XDM, either.

For some use cases it's important that the format be canonical: that is,
there should only be one way of representing a given sequence.

Also, XQuery doesn't have any direct way of representing the type annotation
on a node: the closest you can get is a validate{} instruction, and the
effect of this is very context-dependent.

Incidentally, one deficiency of the Saxon -wrap format that I mentioned
earlier is that it doesn't contain any information about node identity. If
you have a sequence of three elements with the same name and content, it
won't tell you whether the sequence being serialized contains three distinct
nodes, or a single node repeated three times.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay

Serialization of XDM

2009-09-14T21:42:36Z

> 
> I believe this lack of standardization for serialization of 
> XDM is both 
> a "Problem" and a "BIG Problem"
> Before I extend the topic to "suggested solutions" I would love some 
> feedback.
> 
> Q: Does anyone else agree this is a "Problem" or a "Big 
> Problem" ? Am I alone ? Why is this NOT a "Big Problem"?

Saxon has a serialization format for arbitrary XDM sequences: you can see it
by running a query with -wrap option from the command line. Perhaps oddly,
Saxon can generate this format, but there are no tools to parse it and
reconstruct the original sequence.

I seem to remember someone proposing that a format like this should be
standardised in the XQuery WG years ago - if only as a standard way of
representing the results of queries in the test suite. But the fact that
no-one was prepared to do the work suggests that not many people thought it
was a big problem.

> Q: Is there any known work in progress aimed at solving this problem ?

Not that I know of.

> Q: Would it be of value to solve this problem ? (and why not ?)

Yes. Whether the value would exceed the cost is anyone's guess.

> Q: Would there be any interest in tackling this problem in a 
> formal way, such as a W3C recommendation ?

W3C groups are generally having difficulty finding the resources to complete
their existing work programmes, without taking on anything new.

> Q: If there were a recommendation approved would vendors be 
> interested or willing to adopt / implement it ?
> 

If there appeared to be sufficient support behind it, then yes.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay

Serialization of XDM

2009-09-14T20:08:59Z

On Mon, Sep 14, 2009 at 5:48 PM, Evgeny Gazdovsky  wrote:
> We had forgot about JSON! :)

congratulations - right move to forget JSON

> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Serialization of XDM

2009-09-14T20:02:27Z

Still think that JSON may be a variant

Serialization of XDM

2009-09-14T19:36:20Z

What about this document ?

Input:


  


Query:
   
( / , /foo/bar )

If serialized then re-parsed into say :
    declare variable $a as external;     -> stuck into here

is it important that
    $a[1]/foo/bar  is  $a[2]


David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



David Carlisle wrote:
>> On first glance I can imagine the former being fairly easy, but the 
>> later much more complex.
>>     
>
> The latter isn't a problem as they are different elements in the result,
> the $a nodes are _copied_ into their parent as {$a} copy here
> doesn't copy node identity.
>
> David
>
> ________________________________________________________________________
> The Numerical Algorithms Group Ltd is a company registered in England
> and Wales with company number 1249803. The registered office is:
> Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.
>
> This e-mail has been scanned for all viruses by Star. The service is
> powered by MessageLabs. 
> ________________________________________________________________________
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090914/7f6b436e/attachment.htm

Serialization of XDM

2009-09-14T19:12:07Z

Michael Kay wrote:
>
> Incidentally, one deficiency of the Saxon -wrap format that I mentioned
> earlier is that it doesn't contain any information about node identity. If
> you have a sequence of three elements with the same name and content, it
> won't tell you whether the sequence being serialized contains three distinct
> nodes, or a single node repeated three times.
>
>   

Interesting point about Node Identity.  I hadnt thought of that.
Do you have an oppionion of how important it is to preserve node identity ?
Would this be important only at the sequence level or in nested elements 
as well ?
e.g

let $a := 
return
    ( $a , $a , $a )
--->  recognize all 3 are the same

let $a := 
return
    ( {$a},{$a},{$a})

--> do we need to know that all 3  are the same ?

On first glance I can imagine the former being fairly easy, but the 
later much more complex.

David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Serialization of XDM

2009-09-14T18:48:58Z

We had forgot about JSON! :)

Serialization of XDM

2009-09-14T17:06:13Z

Somewhat Ironically, this solves the same problem the same way JSON did 
i.e.
make the serialization format the same as the Code (in Jason's case 
Javascript).
It also has the same problem though.
It requires an XQuery paser to parse.  Suppose you want to send the 
results to something that isnt XQuery, say as an external variable to an 
XSLT or XPATH processor.   Those would require XQuery to pase it.

Its an interesting idea with some merit though ...

David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Pavel Minaev wrote:
> In Mon, Sep 14, 2009 at 8:02 AM, Evgeny Gazdovsky  wrote:
>   
>> Still think that JSON may be a variant
>>     
>
> I don't see how JSON could be any sort of serious contender to
> serialize XDM data, given that the latter is much richer than JSON
> primitives.
>
> On the other hand, it seems to me that we already have such a
> serialization format, and it's XQuery itself - or rather the subset
> that involves literals, sequence "constructor" (i.e. comma operator),
> and direct element/attribute/namespace/text constructors. It's as
> portable as it gets - any XQuery processor can immediately parse it.
> And it doesn't seem to be missing anything from XDM, either.
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090914/8f599b4f/attachment.htm

Serialization of XDM

2009-09-14T15:36:11Z

David A. Lee wrote:

  Hi,

> There is no standard or even 'community accepted' way of
> serializing XDM

  While I used to think this is a problem, more and more I think
it is not (or at most a little problem we can live with.)  XDM
is a convenient (and needed) data structure to work with within a
piece of code (like a stylesheet or a query.)  (BTW this is a
pity that XProc is not based on XDM, IMHO)

  But if you want to pass information between two different
pieces of code (like between a query and a stylesheet) you can
just restrict yourself to nodes.  If you want to pass a sequence
of integers, you can instead pass an element containing elements
the content of which are integers...

  So the simple rule: independent pieces of code set boundaries
between which you can only pass nodes, or even document nodes.
That's what XML means, after all.  If you use compatible
processors, you should then be able to configure them to pass
those nodes in an efficient way, and if not you can serialize and
parse them without loosing any information.

  Just my two cents....

  Regards,

-- 
Florent Georges
http://www.fgeorges.org/

Serialization of XDM

2009-09-14T14:29:30Z

On Mon, Sep 14, 2009 at 1:16 PM, Michael Kay  wrote:
>> On the other hand, it seems to me that we already have such a
>> serialization format, and it's XQuery itself - or rather the
>> subset that involves literals, sequence "constructor" (i.e.
>> comma operator), and direct element/attribute/namespace/text
>> constructors. It's as portable as it gets - any XQuery
>> processor can immediately parse it.
>> And it doesn't seem to be missing anything from XDM, either.
>
> For some use cases it's important that the format be canonical: that is,
> there should only be one way of representing a given sequence.
>
> Also, XQuery doesn't have any direct way of representing the type annotation
> on a node: the closest you can get is a validate{} instruction, and the
> effect of this is very context-dependent.

I wonder if "treat as" operator could be abused for this purpose. The
deserializer would have to be specified as if it always runs the input
sequence through validate{} recursively, naturally...

Of course, there's still the problem of no inline schemas for XQuery,
so it's still not a fully self-contained format. Then again, in
practice, few people use inline schemas for plain XML serialization as
well, so it might not be that much of a problem.

> Incidentally, one deficiency of the Saxon -wrap format that I mentioned
> earlier is that it doesn't contain any information about node identity. If
> you have a sequence of three elements with the same name and content, it
> won't tell you whether the sequence being serialized contains three distinct
> nodes, or a single node repeated three times.

Good point. It's fairly obvious how this could be done with plain
XQuery in general with let/return, but I also can't think of any good
way to define a canonical serialization format for it. One approach
would be to require all nodes that occur more than once in the output
(and only such nodes) to produce a single let-clause, with unspecified
order and unspecified variable names they're bound to, at the
beginning of the query, immediately followed by a single "return"
clause that references them. But this wouldn't be very human-readable
for moderately sized graphs, IMO.

Serialization of XDM

2009-09-14T14:20:02Z

On Mon, Sep 14, 2009 at 1:06 PM, David A. Lee  wrote:
> Somewhat Ironically, this solves the same problem the same way JSON did i.e.
> make the serialization format the same as the Code (in Jason's case
> Javascript).
> It also has the same problem though.
> It requires an XQuery paser to parse.  Suppose you want to send the results
> to something that isnt XQuery, say as an external variable to an XSLT or
> XPATH processor.   Those would require XQuery to pase it.

Just as with JSON, the ability to parse data by eval'ing it in XQuery
is just a side bonus. Of course you'd also want specialized parsers
for this when dealing with such serialized data in anything other than
XQuery (and you'd want it even in XQuery for security reasons). But
there's no shortage of JSON parsers of all kinds, written in different
languages and for different platforms, these days. This could be
similar.

A more major advantage of using XQuery subset as a basis for such a
serialization format is that you don't need to create a specification
from scratch. For the most part, you can directly re-use XQuery BNF
and semantics from the spec.

Also, so long as it is a strict subset, any new dedicated parser could
reuse existing XQuery lexer/parser code with no changes. You'd pretty
only need a new syntax tree rewriter that would output XDM nodes, in
whatever in-memory format a particular implementation wants.

By the way, much as DOM is often maligned, the idea of defining some
common, strictly specified interfaces (in API sense) still has merit,
and having something along those lines for XDM would be nice as well
(and would mesh up well with a common serialization format).

Serialization of XDM

2009-09-14T13:55:51Z

In Mon, Sep 14, 2009 at 8:02 AM, Evgeny Gazdovsky  wrote:
> Still think that JSON may be a variant

I don't see how JSON could be any sort of serious contender to
serialize XDM data, given that the latter is much richer than JSON
primitives.

On the other hand, it seems to me that we already have such a
serialization format, and it's XQuery itself - or rather the subset
that involves literals, sequence "constructor" (i.e. comma operator),
and direct element/attribute/namespace/text constructors. It's as
portable as it gets - any XQuery processor can immediately parse it.
And it doesn't seem to be missing anything from XDM, either.

Serialization of XDM

2009-09-14T11:56:24Z

What about RDF (any not XML notation) and SPARQL/xSPARQL?

Serialization of XDM

2009-09-14T11:48:31Z

I agree with your sentiment
*"if not you can serialize and parse them without loosing any information."

*but not the implication.

This is (partially) true, but the problem with this is there is not 
*standardized* way to do this.
For simple things like say sequences of nodes, you can wrap them in a 
document.
Or for say integers you can wrap them in something like an  tag.
The problem is there are a *million* ways to do it ... which means no 
vendor has agreed, which leaves it up to the programmer to do it.
Which means you have to cooperate on both sides of the equation, and 
since there is no standardized way to do it, different toolkits or 
pieces are almost certain to do it differently, and possibly in ways 
that are difficult to parse.

By "partially" I am refering to the interesting XDM type of attributes,  
which noone can agree at all on how to serialize.
( although you could stick them in an element ... )

So to make this actually work in practice, you must control both sides 
of the partnership, and the serialization in between, and also pass 
"metadata" either by convention or something to make it all work.   A 
programmer, to say take the output of one XQuery, store it in a file, 
then put it back into an external varialbe of another XQuery invocation 
has to do quite a lot of work, the sort of work that XML was designed 
explicitly to *avoid having to do*.

Even defining a standardized schema for 'wrapping' XDM content 
unambiguously in a XML document would be a huge value,
even if the result wasn't pretty, at least it has the potential for 
interchange.

But lets not talk about function items ... I have no clue where to go 
with those ....

David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Florent Georges wrote:
> David A. Lee wrote:
>
>   Hi,
>
>   
>> There is no standard or even 'community accepted' way of
>> serializing XDM
>>     
>
>   While I used to think this is a problem, more and more I think
> it is not (or at most a little problem we can live with.)  XDM
> is a convenient (and needed) data structure to work with within a
> piece of code (like a stylesheet or a query.)  (BTW this is a
> pity that XProc is not based on XDM, IMHO)
>
>   But if you want to pass information between two different
> pieces of code (like between a query and a stylesheet) you can
> just restrict yourself to nodes.  If you want to pass a sequence
> of integers, you can instead pass an element containing elements
> the content of which are integers...
>
>   So the simple rule: independent pieces of code set boundaries
> between which you can only pass nodes, or even document nodes.
> That's what XML means, after all.  If you use compatible
> processors, you should then be able to configure them to pass
> those nodes in an efficient way, and if not you can serialize and
> parse them without loosing any information.
>
>   Just my two cents....
>
>   Regards,
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090914/2d37fe78/attachment.htm

Serialization of XDM

2009-09-14T09:32:10Z

A step towards defining an XDM Serialization would be to knock at the 
edges of the problem.

1) I think people would agree that a Document  should be Serialized (and 
read) in existing standard XML text form.
-- This preserves the most common use case

2) I think it is desirable that a single atomic xs:string be serialized 
as plain string unescaped and unencoded.
-- I think this is a common use case, for example generating plain text, 
or even things like CSV from xquery

It MAY be desirable that any atomic type be serialized as if it were 
first converted to xs:string ... e.g.   xs:integer(1) -> "1"

Now comes the fun part,
Since it is impossible to distinguish a "Sequence of one item" from "One 
item" then we need to serialize  a Sequence of One identically to One.

My guess is this is where the committees started having problems and 
decided to avoid the issue altogether.
If, say a sequence of 1 integer is serialized as "1", but as Kurt 
suggested a sequence notation could be used for sequences, then
say a sequence of 2 integers ( for example) could be  "(1,2)".   this 
means that sequences of 1 and sequences of N>1 are serialized differently !
I suggest this is undesirable.

There is a common practice that avoids this.  That is by using a syntax 
that avoids header and trailing markup so that an single value is 
serialized the same as a sequence of 1 value.  One example is simply 
using a separator such as LF or space.
The terminator being supplied by the wrapper (such as EOF or NUL or 
end-of-element).
This requires (as any markup) a way of escaping or quoting so that the 
seperatator can be contained within the data.  In order to conform to #2 
(simple string output as plain text) that leads out quoting, but does 
require escaping.  Depending on the separator used, this could be 
annoying or inconsequential.
It could also be variable (defined by the environmen).  This is 
essentially CSV ... or its variants ...
That would buy us sequences.  Even sequences of documents !
*IFF* XML parsers would be so nice as to stop reading when they have 
read the closing tag instead of throw an error (I believe this is a 
requirement of a conforming parser, not sure why).  But this can be hacked.
(I'd prefer a hack *inside* the XML parsers but if not then wrappers can 
be written which reuse existing XML parsers , not pretty though).
Certainly a parser would have to know its reading XDM not XML ahead of time.

Well thats a start, its an outline for how to serialize sequences which 
preserves common use cases.
Further work is needed to define the individual item serialization forms 
and decide if preserving type is critical or not.  I.e.  if you write an 
xs:integer is it OK to get back an xs:string ?

David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Kurt Cagle wrote:
> I'm inclined to thing here that there are multiple issues hiding in 
> the question of serialization of XDM. Serialization of turtle node, as 
> Evgeny suggests, points to a non-XML syntax serialization. Similarly, 
> consider a simple sequence of values:
>
> for $a in (1 to 5) return $a * 5
>
> This cannot be serialized in XML notation at all in any meaningful 
> way. Personally I think that this is where output to sequence notation 
> makes sense:
>
> (5,10,15,20,25)
>
> which would also be consistent with an XQuery consumption model.
>
> This is probably something worth talking to Dr. Michael Kay on, as he 
> was the editor of the XQuery/XSLT Serialization spec, but it would 
> seem to me that any XDM serialization should either resolve to a 
> specific type'd entity, a document node, a sequence of type'd entities 
> or a sequence of document nodes. This would hold true just as readily 
> for the RDF/Turtle use case; in an XQuery environment each assertion 
> would be given either as a text node() or, if supported, as a distinct 
> item(), and the results of a SPARQL query would then be a sequence of 
> such nodes.
>
> Kurt Cagle
> Managing Editor
> http://xmlToday.org
>
>
> On Sun, Sep 13, 2009 at 11:56 PM, Evgeny Gazdovsky 
> > wrote:
>
>     What about RDF (any not XML notation) and SPARQL/xSPARQL?
>     _______________________________________________
>     http://x-query.com/mailman/listinfo/talk 
>     http://x-query.com/mailman/listinfo/talk
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090914/afaad0f8/attachment.htm

Serialization of XDM

2009-09-14T02:13:43Z

I'm inclined to thing here that there are multiple issues hiding in the
question of serialization of XDM. Serialization of turtle node, as Evgeny
suggests, points to a non-XML syntax serialization. Similarly, consider a
simple sequence of values:

for $a in (1 to 5) return $a * 5

This cannot be serialized in XML notation at all in any meaningful way.
Personally I think that this is where output to sequence notation makes
sense:

(5,10,15,20,25)

which would also be consistent with an XQuery consumption model.

This is probably something worth talking to Dr. Michael Kay on, as he was
the editor of the XQuery/XSLT Serialization spec, but it would seem to me
that any XDM serialization should either resolve to a specific type'd
entity, a document node, a sequence of type'd entities or a sequence of
document nodes. This would hold true just as readily for the RDF/Turtle use
case; in an XQuery environment each assertion would be given either as a
text node() or, if supported, as a distinct item(), and the results of a
SPARQL query would then be a sequence of such nodes.

Kurt Cagle
Managing Editor
http://xmlToday.org

On Sun, Sep 13, 2009 at 11:56 PM, Evgeny Gazdovsky wrote:

> What about RDF (any not XML notation) and SPARQL/xSPARQL?
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090914/521bcd87/attachment.htm

Serialization of XDM

2009-09-13T23:53:38Z

David, 

in my opinion it is a BIG problem if one feels (like I do) that you are heading for a GREAT idea: advancing from the concept of a byte stream to the concept of an info stream. (info stream = XDM-modelled character stream.) If one remembers what is gained by enabling the interconnection of units emitting/consuming byte streams, one may be surprised to discover that one cannot readily connect by info stream. (That is, unless accepting that all information must be document-wrapped, the XDM subset you spoke of.) Perhaps it is a question of how tightly the units must be coupled. If the work invested in exchange is small compared with internal work (example: consuming/providing web services) it is usually not too much asked to dress up and present oneself as an XML document. XProc accepted this point of view. However, if one aims at a more immediate coupling (shell ...), one arrives at this Big Problem. I think you are right, it *is* a gap, and it *should* be
 addressed by a W3C recommendation. (Also, this would mean the emancipation of XDM from XML, which is overdue.)

One more thought. XQuery is a prince looking like a frog. If the XDM were not what it is and forced XQuery expressions to exchange documents, rather than item sequences, XQuery were a frog looking like a frog. This alone might already make us think... 

With kind regards,
Hans-Juergen




----- Ursprüngliche Mail ----
From: "David A. Lee" 
Subject:  Serialization of XDM

I'm dual-posting this because I think it is relevant to XML in general, 
but perhaps more so to xquery

Summary (postulate)
There is no standard or even 'community accepted' way of serializing XDM

Why is this a problem
XML languages which produce XDM (such as XPath and XQuery) and can 
accept XDM (such as XQuery and XPath through external variables) or 
produce XML are not 'closed' wrt serialization.  For example XQuery can 
produce things which have no standardized representation.  The only way 
to consume and produce these "things" are with proprietary formats, and 
even those only exist in 'in memory' language representations in general. 
The classic example is an XDM Sequence.   There is no standard or even 
common serialization format for Sequences.
Thus an XQuery which produces a sequence (say a sequence of xs:integer) 
cannot have its output fed back into another XQuery in any standard way.
It is true that some implementations of XQuery provide in-memory  
formats for XDM such that the result of that vendor's XQuery can be used 
as input to that vendor's XQuery in the space of that vendor's 
implementation language.   And Some frameworks (such as .NET) provide 
in-memory interfaces for XDM.  But in general once you cross vendors or 
cross languages or exit the realm of languages there is no standardized 
or common way to accomplish interop between XDM producers and consumers.

Why is this a BIG problem
I believe one of the reasons XML has achieved its great success to date 
is due to its standardized Text serialization format.  If XML resided 
purely in the abstract space of "INFOSET" I don't believe it would have 
proliferated to even 1,000,000th as much as it has today.
The fact that newer XML processing languages can produce datum which has 
no text serialization format is a BIG problem, IMHO.
It means that there is no way to exchange data produced by one vendor's 
implementation  to another unless it conforms to the subset of XDM which 
is the XML Document.
Even within *the same vendor* it is almost impossible.   For a tougher 
example, take an XQuery which produces an attribute.  There is no XQuery 
vendor that I know of that can serialize an attribute to a 'file' (or 
network, or string buffer)  and read it back even in its own implementation.
Take even something as simple as text.   Serialize a single atomic text 
value such as "1"  and try to read it back.  Even if a vendor supports 
reading XDM from a file, (which I know of none), what type would it 
assign to "1" ?  xs:string ? xs:integer ?


I believe this lack of standardization for serialization of XDM is both 
a "Problem" and a "BIG Problem"
Before I extend the topic to "suggested solutions" I would love some 
feedback.

Q: Does anyone else agree this is a "Problem" or a "Big Problem" ? Am I 
alone ? Why is this NOT a "Big Problem"?
Q: Is there any known work in progress aimed at solving this problem ?
Q: Would it be of value to solve this problem ? (and why not ?)
Q: Would there be any interest in tackling this problem in a formal way, 
such as a W3C recommendation ?
Q: If there were a recommendation approved would vendors be interested 
or willing to adopt / implement it ?

David A. Lee


------------------------------

_______________________________________________
http://x-query.com/mailman/listinfo/talk
http://x-query.com/mailman/listinfo/talk

End of talk Digest, Vol 77, Issue 19
************************************

Serialization of XDM

2009-09-13T15:06:27Z

I'm dual-posting this because I think it is relevant to XML in general, 
but perhaps more so to xquery



Summary (postulate)
There is no standard or even 'community accepted' way of serializing XDM

Why is this a problem
XML languages which produce XDM (such as XPath and XQuery) and can 
accept XDM (such as XQuery and XPath through external variables) or 
produce XML are not 'closed' wrt serialization.  For example XQuery can 
produce things which have no standardized representation.  The only way 
to consume and produce these "things" are with proprietary formats, and 
even those only exist in 'in memory' language representations in general. 
The classic example is an XDM Sequence.   There is no standard or even 
common serialization format for Sequences.
Thus an XQuery which produces a sequence (say a sequence of xs:integer) 
cannot have its output fed back into another XQuery in any standard way.
It is true that some implementations of XQuery provide in-memory  
formats for XDM such that the result of that vendor's XQuery can be used 
as input to that vendor's XQuery in the space of that vendor's 
implementation language.   And Some frameworks (such as .NET) provide 
in-memory interfaces for XDM.  But in general once you cross vendors or 
cross languages or exit the realm of languages there is no standardized 
or common way to accomplish interop between XDM producers and consumers.

Why is this a BIG problem
I believe one of the reasons XML has achieved its great success to date 
is due to its standardized Text serialization format.  If XML resided 
purely in the abstract space of "INFOSET" I don't believe it would have 
proliferated to even 1,000,000th as much as it has today.
The fact that newer XML processing languages can produce datum which has 
no text serialization format is a BIG problem, IMHO.
It means that there is no way to exchange data produced by one vendor's 
implementation  to another unless it conforms to the subset of XDM which 
is the XML Document.
Even within *the same vendor* it is almost impossible.   For a tougher 
example, take an XQuery which produces an attribute.  There is no XQuery 
vendor that I know of that can serialize an attribute to a 'file' (or 
network, or string buffer)  and read it back even in its own implementation.
Take even something as simple as text.   Serialize a single atomic text 
value such as "1"  and try to read it back.  Even if a vendor supports 
reading XDM from a file, (which I know of none), what type would it 
assign to "1" ?  xs:string ? xs:integer ?


I believe this lack of standardization for serialization of XDM is both 
a "Problem" and a "BIG Problem"
Before I extend the topic to "suggested solutions" I would love some 
feedback.

Q: Does anyone else agree this is a "Problem" or a "Big Problem" ? Am I 
alone ? Why is this NOT a "Big Problem"?
Q: Is there any known work in progress aimed at solving this problem ?
Q: Would it be of value to solve this problem ? (and why not ?)
Q: Would there be any interest in tackling this problem in a formal way, 
such as a W3C recommendation ?
Q: If there were a recommendation approved would vendors be interested 
or willing to adopt / implement it ?



David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Full Text Grammar ambiguous?

2009-09-12T18:00:29Z

> thanks for the observation! This problem was recently posted in the
> W3C's bugzilla system:

Thanks Christian, somehow the W3C Bugzillas search doesn't like me
(the feeling is mutual).

Pretty annoying thing though, I think there are already quite a lot of
(partial) XQFT implementations out there, including our own.

Martin

Full Text Grammar ambiguous?

2009-09-12T16:37:28Z

Martin,

thanks for the observation! This problem was recently posted in the
W3C's bugzilla system:

  http://www.w3.org/Bugs/Public/show_bug.cgi?id=7247

..and is currently discussed by the XQFT task force. A decision how to
solve to this issue will be posted at the same place in near future.

Hope this helps,
Christian

___________________________

Christian Gruen
Universitaet Konstanz
Department of Computer & Information Science
D-78457 Konstanz, Germany
Tel: +49 (0)7531/88-4449, Fax: +49 (0)7531/88-3577
http://www.inf.uni-konstanz.de/~gruen



On Sat, Sep 12, 2009 at 3:27 PM, Martin Probst  wrote:
> Hi all,
>
> maybe I'm just a bit late to the party, but is it possible that the
> full text grammar from XQFT makes the XQuery grammar ambiguous?
>
> Consider the expression:
>  element ftcontains { 'x' }
>
> As far as I can see, it's impossible to tell whether this should be
> parsed as an element constructor for the element named "ftcontains"
> with content 'x', or as a full text matching operation on a child axis
> selection for 'element' against the words 'x'.
>
> Is this a known issue? If yes, how should this be resolved?
>
> Martin
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk

Full Text Grammar ambiguous?

2009-09-12T16:27:53Z

Hi all,

maybe I'm just a bit late to the party, but is it possible that the
full text grammar from XQFT makes the XQuery grammar ambiguous?

Consider the expression:
  element ftcontains { 'x' }

As far as I can see, it's impossible to tell whether this should be
parsed as an element constructor for the element named "ftcontains"
with content 'x', or as a full text matching operation on a child axis
selection for 'element' against the words 'x'.

Is this a known issue? If yes, how should this be resolved?

Martin

Local namespace

2009-09-11T17:23:17Z

> XQuery 1.1 is introducing the "private" function modifier, which will
> allow the definition of functions that aren't exported from a module.

Simply dropping the namespace requirement and mandating that only the
properly namespaced functions are exported might have accomplished the
same thing without additional syntax, right?

>> I have also never really understood why it must be possible to have
>> one module with one target namespace split over multiple files; I
>> think Java has shown that the forced one-to-one relationship between
>> conceptual name (class) and file (.java) is a good thing.
>
> 1) Modules aren't classes, they are more like library APIs.

Yes.

> 2) Java's rules are by no means universally considered a good thing. In
> particular it might be beneficial to split up a 5000 line Java class
> into multiple files, or to group together implementations of the same
> method from multiple implementations of a class.

Well, if you have a single class spanning 5000 lines that might be a
sign of doing it wrong in the first place ;-) The thing about grouping
together tiny, closely related classes works for me in Java using
nested classes, YMMV.

My point was that I generally think its beneficial to have only
exactly one "entity" (i.e. XQuery module file) completely containing
everything thats relevant for one particular namespace, that makes
many things much easier, in particular when IDE support (go to
definition) isn't all that great yet.

> 3) Actually the XQuery spec doesn't say anything about whether a module
> namespace can be split across multiple module files - that's left
> implementation defined.

Yes, but this is how I understood the feature. And at least at some
point of time the test suite kind of enforced that feature.

Martin

Local namespace

2009-09-11T16:40:39Z

Martin Probst wrote:
>> XQuery 1.1 is introducing the "private" function modifier, which will
>> allow the definition of functions that aren't exported from a module.
> 
> Simply dropping the namespace requirement and mandating that only the
> properly namespaced functions are exported might have accomplished the
> same thing without additional syntax, right?

Yes - that's what I originally suggested. However the XQuery working
group felt it was important to be able change a function to be exported
/ not-exported without changing every place that function is called.

John

-- 
John Snelson, Oracle Corporation            http://snelson.org.uk/john
Berkeley DB XML:            http://oracle.com/database/berkeley-db/xml
XQilla:                                  http://xqilla.sourceforge.net

Local namespace

2009-09-11T16:05:35Z

Martin Probst wrote:
> Indeed, you may not define functions or variables in modules that
> don't have the same namespace URI as the module:
> 
> "The name of every variable and function declared in a library module
> must have a namespace URI that is the same as the target namespace of
> the module; otherwise a static error is raised [err:XQST0048]."
> 
> Why that must be, I have never understood. It doesn't save the user
> from hard-to-find typo errors errors as you'd always declare the
> namespace prefix while declaring the module namespace, and then use
> that prefix. And it (IMHO) actively acts against modularization, as
> local variables or functions are highly desirable. In X-Hive / xDB, we
> have never followed that spec rule and allowed local
> functions/variables in modules, those are simply not exported to
> importing modules.

XQuery 1.1 is introducing the "private" function modifier, which will
allow the definition of functions that aren't exported from a module.

> I have also never really understood why it must be possible to have
> one module with one target namespace split over multiple files; I
> think Java has shown that the forced one-to-one relationship between
> conceptual name (class) and file (.java) is a good thing.

1) Modules aren't classes, they are more like library APIs.

2) Java's rules are by no means universally considered a good thing. In
particular it might be beneficial to split up a 5000 line Java class
into multiple files, or to group together implementations of the same
method from multiple implementations of a class.

3) Actually the XQuery spec doesn't say anything about whether a module
namespace can be split across multiple module files - that's left
implementation defined.

John

-- 
John Snelson, Oracle Corporation            http://snelson.org.uk/john
Berkeley DB XML:            http://oracle.com/database/berkeley-db/xml
XQilla:                                  http://xqilla.sourceforge.net

Local namespace

2009-09-11T16:02:13Z

With a big caveat as I have not been involved in any of this, just how
it seems to me.

>  From what I can see in the XQuery 1.0 REC, the "local" namespace
> is aimed to be used only in a main module, and only to locally
> define functions (not variables.)

I think this simply came from the problem that functions without
namespaces, which were allowed in an earlier draft, can collide with
the built in functions or type names, so it was decided to require a
namespace for all user defined functions. To make that a bit easier,
we now have the local: namespace predeclared.

This does not apply to variables as they don't share the same symbol
space as functions, i.e. their names don't collide.

> This is a bit surprising to me,
> as I thought we could define both local:* functions & variables,
> both in main and library modules...

Indeed, you may not define functions or variables in modules that
don't have the same namespace URI as the module:

"The name of every variable and function declared in a library module
must have a namespace URI that is the same as the target namespace of
the module; otherwise a static error is raised [err:XQST0048]."

Why that must be, I have never understood. It doesn't save the user
from hard-to-find typo errors errors as you'd always declare the
namespace prefix while declaring the module namespace, and then use
that prefix. And it (IMHO) actively acts against modularization, as
local variables or functions are highly desirable. In X-Hive / xDB, we
have never followed that spec rule and allowed local
functions/variables in modules, those are simply not exported to
importing modules.

I have also never really understood why it must be possible to have
one module with one target namespace split over multiple files; I
think Java has shown that the forced one-to-one relationship between
conceptual name (class) and file (.java) is a good thing.

Martin

Fwd: xmlns ignore

2009-09-11T14:01:05Z


> I have a problem to ignore a xmls attribute in a tag and did not found
> information how to ignore this attribute. Somebody can help me.

A namespace declaration is not an attribute in the XDM model used by
XPath and Xquery. The name of an element includes the namespace.

Most likely the part of teh code that you don't show

= comp.compile("/.../... ...");

is selecting elements in no-namespace, and you should change it to
select elements in the namespace used in your xml source.

either by using a prefix
/p:aa/p:bb

and binding p to the namespace used in the source, or by setting the
default namespace in the xquery to that used in the source.


David


________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

Fwd: xmlns ignore

2009-09-11T09:45:08Z

Dear all,

I have a problem to ignore a xmls attribute in a tag and did not found
information how to ignore this attribute. Somebody can help me.


I'm using the saxon HE 9.2 to read a xml. My xml have a tag  with  "xmlns="
http://www. ..."", how ignore the "xmlns" when a read the file?

I needed to ignore this attribute because the address return “page no found”
and so my program return nothing. As test, I remove the address from the xml
and the program return the correct  value.


I'm reading the file using this code:

                 Processor proc = new Processor(false);
                 XQueryCompiler comp = proc.newXQueryCompiler();
                 XdmNode source =   proc.newDocumentBuilder().build(new
StreamSource(inputFile));


                 XQueryExecutable exp = comp.compile("/.../... ...");
                 XQueryEvaluator eval = exp.load();
                 eval.setContextItem(source);

                 XdmValue v = eval.evaluate();
                 System.out.println(v.size());


Regards and thank you very much by any help,




---------- Forwarded message ----------
From: Michael Kay 
Date: 2009/9/3
Subject: RE: xmlns ignore
To: http://x-query.com/mailman/listinfo/talk



Short answer: you can't.

This is actually a standard XQuery coding question, nothing specific to
Saxon, so I suggest that you ask it on http://x-query.com/mailman/listinfo/talk

Michael Kay
Saxonica

> -----Original Message-----
> From: http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk]
> Sent: 02 September 2009 16:04
> To: http://x-query.com/mailman/listinfo/talk
> Subject: xmlns ignore
>
> Dear Michael Kay
>
> I'm using the saxon HE 9.2 to read a xml. My xml have a tag
> with  "xmlns="http://www. ..."", how ignore the "xmlns" when
> a read the file?
>
> I'm reading the file using this code:
>
>                 Processor proc = new Processor(false);
>                 XQueryCompiler comp = proc.newXQueryCompiler();
>                 XdmNode source =
> proc.newDocumentBuilder().build(new StreamSource(inputFile));
>
>
>                 XQueryExecutable exp = comp.compile("/.../... ...");
>                 XQueryEvaluator eval = exp.load();
>                 eval.setContextItem(source);
>
>                 XdmValue v = eval.evaluate();
>                 System.out.println(v.size());
>
>
> Regards ,
>
>
>
>




-- 
Sérgio Mariano Dias
http://www.dcc.ufmg.br/~mariano 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090911/c744fc0d/attachment.htm

Schema reporting tools - XQuery use case?

2009-09-10T15:09:58Z

Hello Lisa,

the article was very interesting to me, thank you! Whereas hitherto I only thought of "schema reports", your work shows me that there is a parallel approach one might call "schema querying". (In fact, I borrowed this term from the namespace URI you used in your article.)  

Trying to define the terms: schema reporting transforms schema documents into new documents conveying part of the schema information (e.g. listing valid data paths); the result is a new resource. Schema querying, on the other hand, is a functionality that provides answers for certain kinds of questions referring to the schema. Schema querying tends to provide dynamic information, referring to instance data, whereas schema reporting usually delivers static information. The border is not sharp: the smaller the report's scope, the closer it resembles an answer to a query, and the use of config parameters may link the report to instance data (e.g. by filtering data). Certainly your function 'load-schemas-by-URI' can be regarded as schema reporting (if you add the possibility to collect the retrieved schemas into a single document).

Now I think that the "important use case" I spoke of comprises both, schema reporting and schema querying. And one thing for sure, schema reports can be a convenient basis for implementing schema querying.

With kind regards,
Hans-Juergen




----- Ursprüngliche Mail ----
Von: Xia Li 
An: Hans-Juergen Rennau ; http://x-query.com/mailman/listinfo/talk
CC: http://x-query.com/mailman/listinfo/talk
Gesendet: Mittwoch, den 9. September 2009, 10:54:05 Uhr
Betreff: RE:  Schema reporting tools -  XQuery use case?


As a typical use case, schema information is frequently used to process
GML(Geography Markup Language) instance data. Mostly, the schema
querying part is done with the help of Xerces-J XML schema processing
library in Java. But I do think it is quite useful to have the same
thing done in XML processing language directly instead of having to call
external functions. Ideally, it is best done in XQuery processors and
XPath 2 spec does provide some functions to retrieve schema information.
However, the problem is it is not widely supported feature, and there
barely open source XPath processors that support such functionalities. I
developed an XSLT function library to query schema information. It
essentially contains functions to retrieve a type definition for a given
element. 
    
http://www.ibm.com/developerworks/library/x-schemanode/

It is a much more intriguing programming experience to do this in XSLT
than in Java and I think it can be equally done in XQuery as well.

Lisa

Local namespace

2009-09-10T14:24:52Z

  Hi,

  From what I can see in the XQuery 1.0 REC, the "local" namespace
is aimed to be used only in a main module, and only to locally
define functions (not variables.)  This is a bit surprising to me,
as I thought we could define both local:* functions & variables,
both in main and library modules...

  But maybe I misread the REC.  Any thoughts?

  Regards,

-- 
Florent Georges
http://www.fgeorges.org/

parse form data to XML

2009-09-09T23:02:50Z

i guess XForms in any case better then ordinar HTML forms - so i can
assume that you just underinvestigate this case :)

On Wed, Sep 9, 2009 at 5:32 PM, Jesse Wu wrote:
> Hi Sedov,
>
> I tried XForms but found when the XML model is loaded from a very
> large file with many attributes (vary for each element) for each of
> the element, the screen editing is not practical. May be there is a
> better way to do this using XForms but I could not find a good
> example.
>
> Thanks -Jes
>
> On Wed, Sep 9, 2009 at 12:30 AM, Vyacheslav Sedov
>  wrote:
>> what about XRX? XForms-REST-XQuery
>>
>> On Wed, Sep 9, 2009 at 3:06 AM, Martin Probst wrote:
>>>>    {for $parsed-query-term in $parsed-data
>>>>               let $parse-query-value := substring-after($parsed-query-term,"=")
>>>>               return {xmldb:decode($parse-query-value)}
>>>>               }
>>>
>>> What about
>>>
>>> for $parsed-query-term in $parsed-data
>>> let $parse-query-value := substring-after($parsed-query-term,"=")
>>> let $parameter-name := substring-before($parsed-query-term,
>>> concat("=", $parse-query-value))
>>> return {xmldb:decode($parse-query-value)}
>>>
>>> Alternatively you can simply split again:
>>>
>>> for $parsed-query-term in $parsed-data
>>> let $kv := tokenize($parsed-query-term, '=')
>>> let $key := $kv[1]
>>> let $value := string-join($kv[position()>1],'=')
>>> return {xmldb:decode($value)}
>>>
>>> As the value might contain '=' (I think, or does that need to be
>>> percent encoded?), $kv might contain multiple value parts, so the
>>> string-join is necessary to glue them together again.
>>>
>>> Martin
>>>
>>> _______________________________________________
>>> http://x-query.com/mailman/listinfo/talk
>>> http://x-query.com/mailman/listinfo/talk
>>>
>>
>

"Pruning" an xml "tree" based on nodes' ids

2009-09-09T17:50:19Z

> copy $x := ...
> transform (
>  delete nodes $x//branch[ID != (4, 8)], $x//leaf[ID != 15]
> )
> return $x

Whoops, that should have been

copy $x := ...
transform (
 delete nodes ($x//branch[every $x in (4, 8) satisfies ID != $x],
$x//leaf[ID != 15])
)
return $x

If you compare using != against a list, you need to use
every/satisfies, otherwise the expression will be true if only one of
the values is not equal to the list. Also, parentheses around the
delete nodes part are needed.

Martin

"Pruning" an xml "tree" based on nodes' ids

2009-09-09T16:00:17Z

> I am trying to prune an xml tree based on known ids for the nodes in that
> tree.
> [...]
> For example if I know that I’m in tree id 1 and the branches I
> took in order are 1,4,8,

Are you doing this using copy/transform like the last time?

What about deleting those nodes with the wrong ID?

copy $x := ...
transform (
  delete nodes $x//branch[ID != (4, 8)], $x//leaf[ID != 15]
)
return $x

(Assuming that you'll never need to remove the tree node itself)

Martin

Schema reporting tools - XQuery use case?

2009-09-09T13:39:41Z

i am very seriously thinking about XSLT-based convertor from XSD to
Schematron rules - i have opinion that Schematron power underestimated
in XML world

On Wed, Sep 9, 2009 at 11:55 AM, ricardo
queiros wrote:
> Hi,
> I develop a tool for browsing XML Schema files in a easy way.
> The tool calls http://x-query.com/mailman/listinfo/talk and can be downloaded as open source
> at http://www.dcc.fc.up.pt/schemaDoc/.
> It is based on a XSLT file that transforms the schema file in XHTML code to
> be interpreted in a general browser.
> So the only requirement is a web browser.
> At this moment the tool don't support a few elements of the W3C XML Schema
> specification (e.g. field, key, keyref, redefine, selector, unique elements
> and abstract, substitutionGroup attributtes). It supports Schematron.rules
> embeded in the schema file.
> Best regards.
>
> 2009/9/9 Hans-Juergen Rennau 
>>
>> Hello People,
>>
>> XQuery is _the_ language for agile and efficient evaluation of XML
>> resources (who would contradict me?). But there is another important
>> technology, of course, XML Schema: though its use is often reduced to
>> document validation or data binding, schemas offer a great wealth of static
>> information about (valid) documents. I am sometimes amazed how merrily this
>> information is ignored, especially in projects that have to deal with very
>> complex schemas.
>>
>> Unfortunately, schemas are difficult to read and, worse, difficult to use
>> as input for further processing, as the schema language allows to express
>> the (essentially) same information in so many ways, what with model groups,
>> attribute sets, type derivations, etc. Example: some configuration contains
>> a data path, and one has the appropriate schema files available - how can
>> one implement an automatic check of the path string against that schema?
>>
>> In consequence, I believe creating "schema reporting" tools may be an
>> important use case for XQuery.Here, schema reporting is understood as
>> transforming schema files into a different serialized representation of all
>> or some information the schema contains. (I am _not_ thinking of graphical
>> representations as offered by XML IDEs unless they are accompanied by
>> serialized versions appropriate to serve as input for further processing.)
>> Some examples were: a tree representation of document structure, a list of
>> valid data paths, a mapping of element and attribute names to the governing
>> type, a mapping of type names to data paths.
>>
>> Well - would anyone like to comment on the statement about such schema
>> reporting being an important use case for XQuery?
>>
>> In this context it is of course important to know what is already
>> available (commercial or open source). Would anyone like to speak about
>> available schema reporting tools (in the sense defined above)?
>>
>> Thank you very much,
>> with kind regards -
>>
>> Hans-Juergen Rennau
>>
>>
>>
>>
>>
>> _______________________________________________
>> http://x-query.com/mailman/listinfo/talk
>> http://x-query.com/mailman/listinfo/talk
>
>
>
> --
> Ricardo Queirós
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Schema reporting tools - XQuery use case?

2009-09-09T13:29:58Z

Rick Jelliffe published a number of articles on this, the last one is
http://broadcast.oreilly.com/2009/08/converting-xml-schemas-to-sche-1.html

Best Regards,
George
-- 
George Cristian Bina
 XML Editor, Schema Editor and XSLT Editor/Debugger
http://www.oxygenxml.com

Vyacheslav Sedov wrote:
> i am very seriously thinking about XSLT-based convertor from XSD to
> Schematron rules - i have opinion that Schematron power underestimated
> in XML world
> 
> On Wed, Sep 9, 2009 at 11:55 AM, ricardo
> queiros wrote:
>> Hi,
>> I develop a tool for browsing XML Schema files in a easy way.
>> The tool calls http://x-query.com/mailman/listinfo/talk and can be downloaded as open source
>> at http://www.dcc.fc.up.pt/schemaDoc/.
>> It is based on a XSLT file that transforms the schema file in XHTML code to
>> be interpreted in a general browser.
>> So the only requirement is a web browser.
>> At this moment the tool don't support a few elements of the W3C XML Schema
>> specification (e.g. field, key, keyref, redefine, selector, unique elements
>> and abstract, substitutionGroup attributtes). It supports Schematron.rules
>> embeded in the schema file.
>> Best regards.
>>
>> 2009/9/9 Hans-Juergen Rennau 
>>> Hello People,
>>>
>>> XQuery is _the_ language for agile and efficient evaluation of XML
>>> resources (who would contradict me?). But there is another important
>>> technology, of course, XML Schema: though its use is often reduced to
>>> document validation or data binding, schemas offer a great wealth of static
>>> information about (valid) documents. I am sometimes amazed how merrily this
>>> information is ignored, especially in projects that have to deal with very
>>> complex schemas.
>>>
>>> Unfortunately, schemas are difficult to read and, worse, difficult to use
>>> as input for further processing, as the schema language allows to express
>>> the (essentially) same information in so many ways, what with model groups,
>>> attribute sets, type derivations, etc. Example: some configuration contains
>>> a data path, and one has the appropriate schema files available - how can
>>> one implement an automatic check of the path string against that schema?
>>>
>>> In consequence, I believe creating "schema reporting" tools may be an
>>> important use case for XQuery.Here, schema reporting is understood as
>>> transforming schema files into a different serialized representation of all
>>> or some information the schema contains. (I am _not_ thinking of graphical
>>> representations as offered by XML IDEs unless they are accompanied by
>>> serialized versions appropriate to serve as input for further processing.)
>>> Some examples were: a tree representation of document structure, a list of
>>> valid data paths, a mapping of element and attribute names to the governing
>>> type, a mapping of type names to data paths.
>>>
>>> Well - would anyone like to comment on the statement about such schema
>>> reporting being an important use case for XQuery?
>>>
>>> In this context it is of course important to know what is already
>>> available (commercial or open source). Would anyone like to speak about
>>> available schema reporting tools (in the sense defined above)?
>>>
>>> Thank you very much,
>>> with kind regards -
>>>
>>> Hans-Juergen Rennau
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> http://x-query.com/mailman/listinfo/talk
>>> http://x-query.com/mailman/listinfo/talk
>>
>>
>> --
>> Ricardo Queirós
>>
>> _______________________________________________
>> http://x-query.com/mailman/listinfo/talk
>> http://x-query.com/mailman/listinfo/talk
>>
> 
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk

Schema reporting tools - XQuery use case?

2009-09-09T13:26:44Z

The commercial Saxon 9.2 EE (previous Saxon-SA) XML Schema processor has 
a command line switch that outputs an XML file with the schema 
information in a format suitable for further processing.
For details see the -scmout:filename option
http://www.saxonica.com/documentation9.2/schema-processing/commandline.html

Best Regards,
George
-- 
George Cristian Bina
 XML Editor, Schema Editor and XSLT Editor/Debugger
http://www.oxygenxml.com

Hans-Juergen Rennau wrote:
> Hello People,
> 
> XQuery is _the_ language for agile and efficient evaluation of XML resources (who would contradict me?). But there is another important technology, of course, XML Schema: though its use is often reduced to document validation or data binding, schemas offer a great wealth of static information about (valid) documents. I am sometimes amazed how merrily this information is ignored, especially in projects that have to deal with very complex schemas.
> 
> Unfortunately, schemas are difficult to read and, worse, difficult to use as input for further processing, as the schema language allows to express the (essentially) same information in so many ways, what with model groups, attribute sets, type derivations, etc. Example: some configuration contains a data path, and one has the appropriate schema files available - how can one implement an automatic check of the path string against that schema?
> 
> In consequence, I believe creating "schema reporting" tools may be an important use case for XQuery.Here, schema reporting is understood as transforming schema files into a different serialized representation of all or some information the schema contains. (I am _not_ thinking of graphical representations as offered by XML IDEs unless they are accompanied by serialized versions appropriate to serve as input for further processing.) Some examples were: a tree representation of document structure, a list of valid data paths, a mapping of element and attribute names to the governing type, a mapping of type names to data paths.
> 
> Well - would anyone like to comment on the statement about such schema reporting being an important use case for XQuery?
> 
> In this context it is of course important to know what is already available (commercial or open source). Would anyone like to speak about available schema reporting tools (in the sense defined above)?
> 
> Thank you very much,
> with kind regards -
> 
> Hans-Juergen Rennau
> 
> 
>       
> 
> 
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk

Schema reporting tools - XQuery use case?

2009-09-09T11:54:05Z

As a typical use case, schema information is frequently used to process
GML(Geography Markup Language) instance data. Mostly, the schema
querying part is done with the help of Xerces-J XML schema processing
library in Java. But I do think it is quite useful to have the same
thing done in XML processing language directly instead of having to call
external functions. Ideally, it is best done in XQuery processors and
XPath 2 spec does provide some functions to retrieve schema information.
However, the problem is it is not widely supported feature, and there
barely open source XPath processors that support such functionalities. I
developed an XSLT function library to query schema information. It
essentially contains functions to retrieve a type definition for a given
element. 
     
http://www.ibm.com/developerworks/library/x-schemanode/

It is a much more intriguing programming experience to do this in XSLT
than in Java and I think it can be equally done in XQuery as well.

Lisa


-----Original Message-----
From: http://x-query.com/mailman/listinfo/talk [mailto:http://x-query.com/mailman/listinfo/talk] On
Behalf Of Hans-Juergen Rennau
Sent: Tuesday, September 08, 2009 11:15 PM
To: http://x-query.com/mailman/listinfo/talk
Cc: http://x-query.com/mailman/listinfo/talk
Subject:  Schema reporting tools - XQuery use case?

Hello People,

XQuery is _the_ language for agile and efficient evaluation of XML
resources (who would contradict me?). But there is another important
technology, of course, XML Schema: though its use is often reduced to
document validation or data binding, schemas offer a great wealth of
static information about (valid) documents. I am sometimes amazed how
merrily this information is ignored, especially in projects that have to
deal with very complex schemas.

Unfortunately, schemas are difficult to read and, worse, difficult to
use as input for further processing, as the schema language allows to
express the (essentially) same information in so many ways, what with
model groups, attribute sets, type derivations, etc. Example: some
configuration contains a data path, and one has the appropriate schema
files available - how can one implement an automatic check of the path
string against that schema?

In consequence, I believe creating "schema reporting" tools may be an
important use case for XQuery.Here, schema reporting is understood as
transforming schema files into a different serialized representation of
all or some information the schema contains. (I am _not_ thinking of
graphical representations as offered by XML IDEs unless they are
accompanied by serialized versions appropriate to serve as input for
further processing.) Some examples were: a tree representation of
document structure, a list of valid data paths, a mapping of element and
attribute names to the governing type, a mapping of type names to data
paths.

Well - would anyone like to comment on the statement about such schema
reporting being an important use case for XQuery?

In this context it is of course important to know what is already
available (commercial or open source). Would anyone like to speak about
available schema reporting tools (in the sense defined above)?

Thank you very much,
with kind regards -

Hans-Juergen Rennau


      


_______________________________________________
http://x-query.com/mailman/listinfo/talk
http://x-query.com/mailman/listinfo/talk

parse form data to XML

2009-09-09T10:32:05Z

Hi Sedov,

I tried XForms but found when the XML model is loaded from a very
large file with many attributes (vary for each element) for each of
the element, the screen editing is not practical. May be there is a
better way to do this using XForms but I could not find a good
example.

Thanks -Jes

On Wed, Sep 9, 2009 at 12:30 AM, Vyacheslav Sedov
 wrote:
> what about XRX? XForms-REST-XQuery
>
> On Wed, Sep 9, 2009 at 3:06 AM, Martin Probst wrote:
>>>    {for $parsed-query-term in $parsed-data
>>>               let $parse-query-value := substring-after($parsed-query-term,"=")
>>>               return {xmldb:decode($parse-query-value)}
>>>               }
>>
>> What about
>>
>> for $parsed-query-term in $parsed-data
>> let $parse-query-value := substring-after($parsed-query-term,"=")
>> let $parameter-name := substring-before($parsed-query-term,
>> concat("=", $parse-query-value))
>> return {xmldb:decode($parse-query-value)}
>>
>> Alternatively you can simply split again:
>>
>> for $parsed-query-term in $parsed-data
>> let $kv := tokenize($parsed-query-term, '=')
>> let $key := $kv[1]
>> let $value := string-join($kv[position()>1],'=')
>> return {xmldb:decode($value)}
>>
>> As the value might contain '=' (I think, or does that need to be
>> percent encoded?), $kv might contain multiple value parts, so the
>> string-join is necessary to glue them together again.
>>
>> Martin
>>
>> _______________________________________________
>> http://x-query.com/mailman/listinfo/talk
>> http://x-query.com/mailman/listinfo/talk
>>
>

Schema reporting tools - XQuery use case?

2009-09-09T09:55:02Z

Hi,
I develop a tool for browsing XML Schema files in a easy way.
The tool calls http://x-query.com/mailman/listinfo/talk and can be downloaded as open source at
http://www.dcc.fc.up.pt/schemaDoc/.
It is based on a XSLT file that transforms the schema file in XHTML code to
be interpreted in a general browser.
So the only requirement is a web browser.

At this moment the tool don't support a few elements of the W3C XML Schema
specification (e.g. *field, key, keyref, redefine, selector, unique *elements
and *abstract, substitutionGroup *attributtes). It supports Schematron.rules
embeded in the schema file.

Best regards.

2009/9/9 Hans-Juergen Rennau 

> Hello People,
>
> XQuery is _the_ language for agile and efficient evaluation of XML
> resources (who would contradict me?). But there is another important
> technology, of course, XML Schema: though its use is often reduced to
> document validation or data binding, schemas offer a great wealth of static
> information about (valid) documents. I am sometimes amazed how merrily this
> information is ignored, especially in projects that have to deal with very
> complex schemas.
>
> Unfortunately, schemas are difficult to read and, worse, difficult to use
> as input for further processing, as the schema language allows to express
> the (essentially) same information in so many ways, what with model groups,
> attribute sets, type derivations, etc. Example: some configuration contains
> a data path, and one has the appropriate schema files available - how can
> one implement an automatic check of the path string against that schema?
>
> In consequence, I believe creating "schema reporting" tools may be an
> important use case for XQuery.Here, schema reporting is understood as
> transforming schema files into a different serialized representation of all
> or some information the schema contains. (I am _not_ thinking of graphical
> representations as offered by XML IDEs unless they are accompanied by
> serialized versions appropriate to serve as input for further processing.)
> Some examples were: a tree representation of document structure, a list of
> valid data paths, a mapping of element and attribute names to the governing
> type, a mapping of type names to data paths.
>
> Well - would anyone like to comment on the statement about such schema
> reporting being an important use case for XQuery?
>
> In this context it is of course important to know what is already available
> (commercial or open source). Would anyone like to speak about available
> schema reporting tools (in the sense defined above)?
>
> Thank you very much,
> with kind regards -
>
> Hans-Juergen Rennau
>
>
>
>
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>



-- 
Ricardo Queirós
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090909/a74384db/attachment.htm

"Pruning" an xml "tree" based on nodes' ids

2009-09-09T09:44:18Z

All,

I am trying to prune an xml tree based on known ids for the nodes in that tree.

For example in the following code:


                
                                
                                                
                                                                
                                                                
                                                                
                                                
                                
                                
                                                
                                                                
                                                
                                
                
                
                                
                                                
                                                                
                                                                                
                                                                
                                                
                                                
                                                                
                                                                                
                                                                
                                                                
                                                                                
                                                                
                                                
                                
                


If I know that I am ultimately after leaf with the ID of 15.  However, I also need the ancestor nodes that led me to that leaf.  I will know those IDs as well.  For example if I know that I'm in tree id 1 and the branches I took in order are 1,4,8, how can show a pruned xml tree where the final result is:


                
                                
                                                
                                                                
                                                
                                
                



I have been able to prune in to this:


                
                                
                                                
                                                                


                                                
                                
                                
                                                
                                                                
                                                
                                
                


But the problems comes in the fact that I don't know how many levels of branches there will be before I finally reach the leaf that I'm after.  It seems like I need to work backwards starting at the leaf and prune based on the ids that I have, but I can really see a way to do that.

Any suggestions would be greatly appreciated.

Regards,

Bob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090909/0688c0bf/attachment-0001.htm

parse form data to XML

2009-09-09T09:30:03Z

what about XRX? XForms-REST-XQuery

On Wed, Sep 9, 2009 at 3:06 AM, Martin Probst wrote:
>>    {for $parsed-query-term in $parsed-data
>>               let $parse-query-value := substring-after($parsed-query-term,"=")
>>               return {xmldb:decode($parse-query-value)}
>>               }
>
> What about
>
> for $parsed-query-term in $parsed-data
> let $parse-query-value := substring-after($parsed-query-term,"=")
> let $parameter-name := substring-before($parsed-query-term,
> concat("=", $parse-query-value))
> return {xmldb:decode($parse-query-value)}
>
> Alternatively you can simply split again:
>
> for $parsed-query-term in $parsed-data
> let $kv := tokenize($parsed-query-term, '=')
> let $key := $kv[1]
> let $value := string-join($kv[position()>1],'=')
> return {xmldb:decode($value)}
>
> As the value might contain '=' (I think, or does that need to be
> percent encoded?), $kv might contain multiple value parts, so the
> string-join is necessary to glue them together again.
>
> Martin
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

parse form data to XML

2009-09-09T02:06:41Z

>    {for $parsed-query-term in $parsed-data
>               let $parse-query-value := substring-after($parsed-query-term,"=")
>               return {xmldb:decode($parse-query-value)}
>               }

What about

for $parsed-query-term in $parsed-data
let $parse-query-value := substring-after($parsed-query-term,"=")
let $parameter-name := substring-before($parsed-query-term,
concat("=", $parse-query-value))
return {xmldb:decode($parse-query-value)}

Alternatively you can simply split again:

for $parsed-query-term in $parsed-data
let $kv := tokenize($parsed-query-term, '=')
let $key := $kv[1]
let $value := string-join($kv[position()>1],'=')
return {xmldb:decode($value)}

As the value might contain '=' (I think, or does that need to be
percent encoded?), $kv might contain multiple value parts, so the
string-join is necessary to glue them together again.

Martin

Schema reporting tools - XQuery use case?

2009-09-09T00:14:37Z

Hello People,

XQuery is _the_ language for agile and efficient evaluation of XML resources (who would contradict me?). But there is another important technology, of course, XML Schema: though its use is often reduced to document validation or data binding, schemas offer a great wealth of static information about (valid) documents. I am sometimes amazed how merrily this information is ignored, especially in projects that have to deal with very complex schemas.

Unfortunately, schemas are difficult to read and, worse, difficult to use as input for further processing, as the schema language allows to express the (essentially) same information in so many ways, what with model groups, attribute sets, type derivations, etc. Example: some configuration contains a data path, and one has the appropriate schema files available - how can one implement an automatic check of the path string against that schema?

In consequence, I believe creating "schema reporting" tools may be an important use case for XQuery.Here, schema reporting is understood as transforming schema files into a different serialized representation of all or some information the schema contains. (I am _not_ thinking of graphical representations as offered by XML IDEs unless they are accompanied by serialized versions appropriate to serve as input for further processing.) Some examples were: a tree representation of document structure, a list of valid data paths, a mapping of element and attribute names to the governing type, a mapping of type names to data paths.

Well - would anyone like to comment on the statement about such schema reporting being an important use case for XQuery?

In this context it is of course important to know what is already available (commercial or open source). Would anyone like to speak about available schema reporting tools (in the sense defined above)?

Thank you very much,
with kind regards -

Hans-Juergen Rennau

parse form data to XML

2009-09-08T16:42:05Z



Probably not the most elegant way ...

==========
declare variable $s := 
"v1=v1&a1=a1&v2=v2&a2=a2&v3=v3&a3=a3" ;

let $a := tokenize($s , "&" )
for $i in 0 to xs:integer(count($a) div 2 - 1)
   return { 
tokenize( $a[($i * 2) + 2 ] , "=" )[2] }

==========



David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Jesse Wu wrote:
> Sorry for the newbie question, I am still in the process of learning XQuery.
> What will be the the best way to get HTML form post data into a XML.
> For example:
>
> get-data returns
>
> v1=v1&a1=a1&v2=v2&a2=a2&v3=v3&a3=a3& .... v10=v10&a10=a10
>
> need to get it into XML as
>
> v1
> v2
> v3
> ....
>
> v10
>
> tried to use tokenize then string-before and after =, but could not
> figure out how to get the loop to do 2 elements for each row.
>
>     {for $parsed-query-term in $parsed-data
>                let $parse-query-value := substring-after($parsed-query-term,"=")
>                return {xmldb:decode($parse-query-value)}
>                }
>
> Thank you or your help. -Jes
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

parse form data to XML

2009-09-08T16:22:08Z

Sorry for the newbie question, I am still in the process of learning XQuery.
What will be the the best way to get HTML form post data into a XML.
For example:

get-data returns

v1=v1&a1=a1&v2=v2&a2=a2&v3=v3&a3=a3& .... v10=v10&a10=a10

need to get it into XML as

v1
v2
v3
....

v10

tried to use tokenize then string-before and after =, but could not
figure out how to get the loop to do 2 elements for each row.

    {for $parsed-query-term in $parsed-data
               let $parse-query-value := substring-after($parsed-query-term,"=")
               return {xmldb:decode($parse-query-value)}
               }

Thank you or your help. -Jes

(no subject)

2009-09-07T13:50:07Z

what about remote job?

On Mon, Sep 7, 2009 at 12:42 PM, Marcus Clemens wrote:
> Hi
>
>
>
> Marklogic are looking to hire a couple of permanent consultants . you will
> be based in the UK and there will be some travel to Europe and the USA
>
>
>
> Please send a cv if this is of any interest
>
>
>
> Kind regards
>
>
>
> Marcus Clemens
>
> Director & Senior Consultant
>
> Mercator IT Ltd
>
>
>
> Tel: 01892 611161
>
> Fax: 01892 660185
>
> Email: http://x-query.com/mailman/listinfo/talk
>
> Cooks Corner Business Park, The Old Saw Mill, London Road, Crowborough TN6
> 1TQ
>
>
>
> Registered in England no: 05755983
>
> Registered office: 117 Dartford Road, Dartford, Kent DA1 3EN
>
>
>
> This email may contain privileged/confidential information and is for the
> intended addressee only.  If you have received this message in error then
> you must not use, retain, disseminate or otherwise deal with it.  Please
> notify the sender by return email and destroy.  The views of the author may
> not necessarily constitute the views of Mercator IT Solutions Ltd.  Nothing
> in this email shall bind Mercator IT Solutions Ltd in any contract or
> obligation.
>
>
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

(no subject)

2009-09-07T10:42:03Z

Hi 

 

Marklogic are looking to hire a couple of permanent consultants . you
will be based in the UK and there will be some travel to Europe and the
USA 

 

Please send a cv if this is of any interest 

 

Kind regards 

 

Marcus Clemens

Director & Senior Consultant

Mercator IT Ltd

 

Tel: 01892 611161

Fax: 01892 660185

Email: http://x-query.com/mailman/listinfo/talk  

Cooks Corner Business Park, The Old Saw Mill, London Road, Crowborough
TN6 1TQ

 

Registered in England no: 05755983

Registered office: 117 Dartford Road, Dartford, Kent DA1 3EN

 

This email may contain privileged/confidential information and is for
the intended addressee only.  If you have received this message in error
then you must not use, retain, disseminate or otherwise deal with it.
Please notify the sender by return email and destroy.  The views of the
author may not necessarily constitute the views of Mercator IT Solutions
Ltd.  Nothing in this email shall bind Mercator IT Solutions Ltd in any
contract or obligation.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090907/0a4361eb/attachment.htm

Tool development: by Perl-wrapped XQuery

2009-09-06T10:19:27Z

Hello, Hans-Juergen

I do believe you are understanding my perspective.  It give me hope that 
perhaps it is not a singular one and I'm not utterly and entirely insane.

You've made some interesting comments !

> 2. (but also) XML Schemas (!) 
> => xsd info commands
>   
I had never thought of this !
> There were no "schemas" to describe byte streams, but we have them to describe item streams, and I think here is a great demand of standard tools unlocking their wealth of information. For example:
>
> get-simple-type myElementName | simple-type-pattern
> => yielding the pattern constraining an element (e.g.  "\D\D\D\d+"), or
>
> get-paths myFile | invalid-paths myTargetNamespacePrefix
> => yielding any paths not compatible with a certain schema.
>   
Wow those are great ideas.  I need to think about what they mean and how 
one would make use of them.
The last one could be implemented by iteration over  "xvalidate"  .. 
.but building this into something like a "find" command or even making 
type validity and information at a very low level as apposed to just a 
command level is interesting.   Also considering that a document (or an 
element) can have multiple types simultaneously brings on a level of 
complexity.   What do we do with these commands ?

> I used to regard XProc as *the* enabler, *the* integrator. Now I get a feeling that one might look at XProc and xmlsh as sister technologies resembling XSLT and XQuery: both built on the same basis, the XDM and the XPath language; the former a little more powerful and feature-rich, with the cost of rather heavy syntax, the latter very slender and flexible.
>
> With kind regards,
> Hans-Juergen
>   
I disagree that xproc is more "feature-rich" then xmlsh but we can leave 
that debate for another day.
The important thing, IMHO, is that having these multiple languages and 
tools demonstrate the importance of integration technologies, and that 
there is more then one way to do it.  Once we know there are 2 ways to 
do something we can discover there are 10 or 1000 ... and as you 
indicated this opens up a really interesting area that is really at a 
different level of thought then any one particular technology, but 
rather  how a system based on XML concepts can behave, not limited to 
one technology.  I think this is an exciting phase in XML technologies.

-David

-- 
David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Tool development: by Perl-wrapped XQuery

2009-09-05T23:09:25Z

Hello David,

gradually I start to get a feeling for your perspective. Pulling "an efficient mechanism for interconnecting them" into focus you move across the boundaries of the XQuery spec's field of vision. Your criticism that design is not only function and module design gains power. Vaguely but vividly this reminds me of the SOAP versus REST controversy: with SOAP clinging to an artificial complication ("I am calling something...") and REST arriving at the more fundamental abstractions of resource and address, enabling a new ease of access and of combining existing resources. Modules are a category emerging only *after* I have called a query, reusable inside queries, whereas the outside is, to put it mildly, bigger and more important.

The great primitives of Unix - the fundamental tools - target the basic entities of an OS environment: (byte) streams, files, directories, users, ... I wonder what new targets emerge with the transition to xmlsh? This is a very important question, relevant as much to xmlsh as to tool development in general - which tools should occupy the top of the community's agenda? ! It occurs to me that there are (at least) two targets of prime importance:

1. (of course) item streams (or 'sequences', 'infosets', 'XDM instances)
=> XDM info commands (or 'services', 'tools', ...)

2. (but also) XML Schemas (!)
=> xsd info commands

There were no "schemas" to describe byte streams, but we have them to describe item streams, and I think here is a great demand of standard tools unlocking their wealth of information. For example:

get-simple-type myElementName | simple-type-pattern
=> yielding the pattern constraining an element (e.g. "\D\D\D\d+"), or

get-paths myFile | invalid-paths myTargetNamespacePrefix
=> yielding any paths not compatible with a certain schema.

Reading
'XQuery is useful and 'important' at the single expression level, all the way up to a full "program"'

and remembering that XQuery is a fully composable language one feels that it is very important to think about the INTEGRATION of XQuery into software and processes in all its aspects. (Especially because XQuery is an Information Processing Language, but not a general purpose programming language!) One could even argue that the still small acceptance of XQuery is related to still insufficient integration.

I used to regard XProc as *the* enabler, *the* integrator. Now I get a feeling that one might look at XProc and xmlsh as sister technologies resembling XSLT and XQuery: both built on the same basis, the XDM and the XPath language; the former a little more powerful and feature-rich, with the cost of rather heavy syntax, the latter very slender and flexible.

With kind regards,
Hans-Juergen

----- Ursprüngliche Mail ----
Von: David A. Lee
An: Hans-Juergen Rennau
CC: http://x-query.com/mailman/listinfo/talk
Gesendet: Samstag, den 5. September 2009, 04:57:12 Uhr
Betreff: Re: Tool development: by Perl-wrapped XQuery

Hello Hans-Juergen

I do agree with a lot of your comments ...
However to add to this one There's another side to this.
> When it comes to the other dimension of scaling - the number of processing steps (rather than the number of units processed in a given step) - it is not at all per se better to have many small programs. As a rule of thumb I suggest "divide the processing if and only if reuseable steps result". So, for example, generic preprocessings (as the one
The third aspect you don't mention is ease of development. I personally find its *vastly easier* to author, debug, and later maintain an xml translation that is decomposed into say 5 steps in individual files with concrete results in each "step". I can run each xquery /xslt individually and output the results (either command line or in a GUI) and debug each one separately. Then put them together and debug the results. Even if *none* of the steps are ever "reusable" either in the past or the future to me this is a very powerful advantage of "divide and conquer". If the end result can be run as efficiently as if they were all in one big program I see that as a huge win.

And as such I disagree with this comment

"Speaking of XQuery - the design of modules and functions is important, not the creation of little programs. "

I suggest that is inventing/imposing an unnecessary distinction between "program" and "modules" and presupposing what is "important".
It may not have been 'important' in the authors of xquery's mind, but thats the strength of reusable software - to grow beyond the original intent.
XQuery is useful and 'important' at the single expression level, all the way up to a full "program" whatever that is.
Just because xquery supports a module implementation does not mean thats the only valuable way of composing larger programs that use xquery as s component.

This is what Unix shell programming taught us 40 years ago, and what programs like XProc are teaching us today..
Its also what frameworks like .NET that support things like inheritance across languages teaches us.

That the distinction between "program" and "module" is fuzzy. And that by making 'little programs' that do one thing only, but do it well,
and providing an efficient mechanism for interconnecting them, you can produce exponentially complex *behavior* without exponentially complex *code*.
The distinction between 'program' and 'module' is subjective and has overlapping meanings in different contexts.
Is "ls" a program or a module ? How about "more" ? then what about 'lsmore' implemented as "ls | more" ? It is this artificial distinction between "module" and "program" itself that leads to what I called "monolithic" programs.

There's a long ways we could go with this. Imagine a language that could 'inherit' a single template from xslt in the context of another language ?
( such as .NET allows across disparate languages) .
And of course taking it too far you can produce exponentially inefficient and complex code that does nothing but blow up the cpu :)

Were beginning to see a more of this (xquery in small pieces), such as using tiny bits of XQuery inside other programs like "XQuery in the browser". Little bits of XQuery in XML data bases. XQuery being used as part of the business logic in a CRM system (yes this exists) coexisting with other languages.
Are these "programs" or "modules" or even less just single expressions ? It doesn't matter.

What matters is can you easily and efficiently hook them up and use them in ways the original authors may have never dreamed.
But in my opinion they are definitely 'important'

David A. Lee
http://x-query.com/mailman/listinfo/talk http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Tool development: by Perl-wrapped XQuery

2009-09-05T08:57:12Z

Hello Hans-Juergen

I do agree with a lot of your comments ...
However to add to this one There's another side to this.
> When it comes to the other dimension of scaling - the number of processing steps (rather than the number of units processed in a given step) - it is not at all per se better to have many small programs. As a rule of thumb I suggest "divide the processing if and only if reuseable steps result". So, for example, generic preprocessings (as the one
The third aspect you don't mention is ease of development. I
personally find its *vastly easier* to author, debug, and later maintain
an xml translation that is decomposed into say 5 steps in individual
files with concrete results in each "step". I can run each xquery
/xslt individually and output the results (either command line or in a
GUI) and debug each one separately. Then put them together and debug
the results. Even if *none* of the steps are ever "reusable" either in
the past or the future to me this is a very powerful advantage of
"divide and conquer". If the end result can be run as efficiently as
if they were all in one big program I see that as a huge win.

And as such I disagree with this comment

"Speaking of XQuery - the design of modules and functions is important,
not the creation of little programs. "

I suggest that is inventing/imposing an unnecessary distinction between
"program" and "modules" and presupposing what is "important".
It may not have been 'important' in the authors of xquery's mind, but
thats the strength of reusable software - to grow beyond the original
intent.
XQuery is useful and 'important' at the single expression level, all the
way up to a full "program" whatever that is.
Just because xquery supports a module implementation does not mean thats
the only valuable way of composing larger programs that use xquery as s
component.

This is what Unix shell programming taught us 40 years ago, and what
programs like XProc are teaching us today.
Its also what frameworks like .NET that support things like inheritance
across languages teaches us.

That the distinction between "program" and "module" is fuzzy. And that
by making 'little programs' that do one thing only, but do it well,
and providing an efficient mechanism for interconnecting them, you can
produce exponentially complex *behavior* without exponentially complex
*code*.
The distinction between 'program' and 'module' is subjective and has
overlapping meanings in different contexts.
Is "ls" a program or a module ? How about "more" ? then what about
'lsmore' implemented as "ls | more" ?
It is this artificial distinction between "module" and "program" itself
that leads to what I called "monolithic" programs.

There's a long ways we could go with this. Imagine a language that
could 'inherit' a single template from xslt in the context of another
language ?
( such as .NET allows across disparate languages) .
And of course taking it too far you can produce exponentially
inefficient and complex code that does nothing but blow up the cpu :)

Were beginning to see a more of this (xquery in small pieces), such as
using tiny bits of XQuery inside other programs like "XQuery in the
browser". Little bits of XQuery in XML data bases. XQuery being used
as part of the business logic in a CRM system (yes this exists)
coexisting with other languages.
Are these "programs" or "modules" or even less just single expressions ?
It doesn't matter.

What matters is can you easily and efficiently hook them up and use them
in ways the original authors may have never dreamed.
But in my opinion they are definitely 'important'

David A. Lee
http://x-query.com/mailman/listinfo/talk
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Tool development: by Perl-wrapped XQuery

2009-09-04T22:24:34Z

Hello David,

yes, I agree that it is desirable to be free in one's choice how to shape the units of functionality, large or small. Still, I would like to make a couple of points.

You wrote:
"... so they have instead built 'monolithic' programs that do all the work within 1 program (xquery, xslt etc) and take a list of filenames input and maybe filenames output to process. ... I suggest that programming model is no longer necessary, and was arrived at due to the poor performance of doing it any other way,"

Sometimes, perhaps even often, this may be the case. But one should not forget that XQuery's aggregative potential teaches us to think in terms of sequences rather than in terms of items. This sweeping approach has nothing to to with "monolithic", just as nobody would call XPath expressions monolithic.

When it comes to the other dimension of scaling - the number of processing steps (rather than the number of units processed in a given step) - it is not at all per se better to have many small programs. As a rule of thumb I suggest "divide the processing if and only if reuseable steps result". So, for example, generic preprocessings (as the one Michael Kay recently recommended in the case of Excel data) are fine examples where dividing is desirable. But if the dividing produces pieces where the second depends on the first but the first serves no other purpose than prepare the second - then the dividing is bad and an unnecessary increase of complexity. Speaking of XQuery - the design of modules and functions is important, not the creation of little programs. XQuery is a powerful language suited for writing highly complex programs. Doing this does not mean following a "monolithic" programming model.

This being said, two final remarks. First, xmlsh seems so important to me because it aims to overcome a sheer anachronism of conventional Unix shells - their inability to deliver structured information. Second, I regard XProc as the sorely needed integration technology which may greatly enhance the value of the various languages (XQuery, ...); even more, it may pave the way to a new paradigm of data processing, in which concepts like "step" and "ports" become so natural to us as "documents" and "objects".

With kind regards,
Hans-Juergen

----- Ursprüngliche Mail ----
Von: David A. Lee
An: Hans-Juergen Rennau
CC: http://x-query.com/mailman/listinfo/talk; http://x-query.com/mailman/listinfo/talk
Gesendet: Freitag, den 4. September 2009, 10:10:37 Uhr
Betreff: Re: Tool development: by Perl-wrapped XQuery

I agree that if you stick to use case #1 (only call xquery once or few
times) and all your input and output are files (or file names) then
pretty much any scripting language that lets you launch a subprocess is
equivalent and you will have very little performance hit.
Historically, I suggest many people use that use case primarily because
its horrendously expensive not to (in most languages) so they have
instead built 'monolithic' programs that do all the work within 1
program (xquery, xslt etc) and take a list of filenames input and maybe
filenames output to process.
This does work.

My point is that I suggest that programming model is no longer
necessary, and was arrived at due to the poor performance of doing it
any other way,
not because its a desirable way of coding.
If you use a language that can call xquery *efficiently* you can design
in a more modular way and use 'itty bitty' xquery programs run thousands
of times instead of one huge program run once, just as efficiently, and
in my opinion much easier to develop, debug, and understand. It frees
you to design and develop in modular ways, using languages of your
choice for different operations instead of forcing you into a single
monolithic application.
Of course it is a personal *opinion* that that is a 'better' way of
doing things :) I certainly enjoy it more.
Others may prefer writing one huge program that does everything.
This exact same philosophical (and technical) debate is what
distinguished unix from the mainframe model of programming. I find it
fascinating that the same fundamental design issues and discussions
evolved again 40 years later.

David A. Lee
>

Tool development: by Perl-wrapped XQuery

2009-09-04T15:24:47Z

David A. Lee schrieb:

>>> * Encourage scripting languages developers to embed these XML
>>> languages directly into the scripting languages (say perl).
>> More precisely, write bindings to implementations in C/C++.
>>
> Only if the scripting language itself is written in C/C++ or has easy
> bindings to it.  If its in another language then bindings in that
> language are useful.

True. I only thought of Perl/PHP/Python/Ruby - all in C. But Jython and
JRuby were mentioned, and I could mention Quercus - all in Java.

> But equally important, IMHO,  is to be able to "glue" the data from
> one module to another efficiently.  That needs common in-memory
> formats.

That's the downside of issuing specifications and waiting for the
implementations to happen. Thank God XML does have a standard
serialization, so any document can migrate from one implementation
to another :-)

> That may mean, for example, that the language runtime define a
> standard XML in-memory representation and every module can use it
> natively. So that you can have say variables of XML Documents and not
> have to transform them to each module's internal representation
> (rebuild the document).
>
> To do this well I suggest is more then just writing bindings.  Its a
> language level problem as well as a binding problem.

Isn't .NET a solution to that problem?

-- 
Michael Ludwig

Tool development: by Perl-wrapped XQuery

2009-09-04T15:12:03Z

> If you can find one vendor/ implementation that has all the pices its a tad
> easier (document types match !) but its still a bit of hell.

That is where I'd say at least in Java land, you can get everything to
accept SAX or DOM. It might not be the absolute best choice (e.g.
Saxon and its TinyTree), but it will work.

I actually can't come up with a tool I needed to use in Java that
didn't support DOM. Can you?

> I think you'll find it about as painful to to use jruby or jython to say
> call  xinclude , then rng validate then xquery then pass its results to xslt
> as writing it in pure java.

Indeed, the problem is that these APIs are extremely verbose and have
an erratic programming model. Parsing a document in Java is a 20 line+
endeavor, and if you want to get it right you'll have to set magic
parser features using undocumented strings no one knows where to find.

But this is the kind of thing you can do right if you don't need the
flexibility to supply your own DocumentBuilderFactory (or whatever),
just by mandating a default.

> In xmlsh its as simple as   xinclude file | rngvalidate schema | xquery -f
> schema | xslt -f stylesheet
>
> To get these kind of languages (jruby etc) into shape where XML is actually
> *easy to use* is going to require work.  Possibly at the language level to
> make it really easy.

Yes, but only a reasonable amount of work. The above code would
actually be valid JRuby syntax, so you wouldn't need any language
extensions, just some library routines.

> I talked to a Ruby programmer at work after a demonstration on XQuery and
> his answer was "I can do that in ruby ... why would I want to learn xquery ?
>  And ruby can do a LOT more then just xml processing " ... After seeing some
> of his code I had to admit he had a point.  I still have been unable to
> successfully convince him the value of xquery over Ruby's XML API's.

Ruby has this nice feature of being very amenable to domain specific
languages, and some of the XML APIs use this nicely. However the
built-in XML support in Ruby is plain horrible, REXML is a huge can of
bugs.

And of course with imperative code in Ruby you loose all the
declarative/optimizable/... niceness of XQuery and XSLT.

Martin

Tool development: by Perl-wrapped XQuery

2009-09-04T14:59:53Z

Jakob Fix schrieb:
> Talking of tools, what is the experience of people on this list with
> xmlstarlet (http://xmlstar.sourceforge.net/) which seems to me like a
> close contender to xmlsh (but haven't looked closely)?

It is built on LibXML2 and LibXSLT by Daniel Veillard. It is like a bag
of tricks. It allows you to code shortcuts for XSLT on the command line.
But you're limited to a subset of XSLT and the quirks of the command
line (notably, quotes).

I still use "xmlstarlet list" when I need a directory listing in XML
because it's convenient and I happen to know about it.

-- 
Michael Ludwig

Tool development: by Perl-wrapped XQuery

2009-09-04T14:50:13Z

David A. Lee schrieb:
>> Now my questions:
>> - would you recommend alternatives for rapid tool development?
>> - if taking a similar approach - would you like to recommend special
>> details, perhaps the script language actually used, or other aspect?
>>
> In my opinion the problem with doing this in perl is that unless the
> xquery implementation itself is in perl or atleast runs within the
> same process you will run into horrible performance problems.

Of course. JVM invocation overhead, IPC overhead. No persistent process
supported by indexes to speed up queries, all possible optimizations
forfeited. But Hans-Jürgen's approach is convenient and sufficient where
performance is not paramount.

In Perl, reasonable XQuery integration can currently be achieved by
using Oracle Berkeley DB XML. Of course, while the interface is in Perl,
the implementation is in C++.

If there were Perl bindings for XQilla XQuery in Perl would be possible
within the process but without the Berkeley database.

But the main speed factor for XQuery is simply the indexes available in
the database. How could it be otherwise? So you need a server process
built on indexes. That's usually C/C++ or Java, like most servers;
scripting APIs are a nice extra.

An alternative is to have some persistent XQuery process running and
having Perl/PHP/Python processes interface to it via HTTP/REST/WHATEVER.

> * Use a scripting language that already is 'in process' with all the
> XML core languages you want to use (xquery, xslt etc)
>  -> examples XProc, xmlsh
>
> * Encourage scripting languages developers to embed these XML
> languages directly into the scripting languages (say perl).

More precisely, write bindings to implementations in C/C++.

-- 
Michael Ludwig

Tool development: by Perl-wrapped XQuery

2009-09-04T14:36:10Z

Talking of tools, what is the experience of people on this list with
xmlstarlet (http://xmlstar.sourceforge.net/) which seems to me like a
close contender to xmlsh (but haven't looked closely)?

cheers,
Jakob.



On Fri, Sep 4, 2009 at 13:09, David A. Lee wrote:
>
>
>
>>
>> Now my questions: - would you recommend alternatives for rapid tool
>> development? - if taking a similar approach - would you like to recommend
>> special details, perhaps the script language actually used, or other aspect?
>>
>
> In my opinion the problem with doing this in perl is that unless the xquery
> implementation itself is in perl or atleast runs within the same process you
> will run into horrible performance problems.   See my (with Norm Walsh's)
> paper :
>
> http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html
>
> What we found is that for our test cases there is a 100-200x (yes 10000 % -
> 20000 % ) performance penalty of using a scripting language to call xml
> processing programs.   This *can* be optimized but the exact use cases of
> using a off-the-shelf scripting language to do this kind of thing is
> typically by the audience of people who do not want to spend the extra
> effort to optimize it, or who are not experts in the type of software
> development/languages required to  do it, or both. i.e its exactly why they
> are using scripting - so they don't have to do all that extra work.
>
> This is the primary reason xmlsh was invented instead of re-using an
> existing scripting language.   I took a "toy" program in a scripting
> language it worked great.
> But when I loaded up all the files I needed it to run it died a horrible
> death.   This is what I call "The Brick Wall" and why scripting XML
> processes fail so many of us.   The presentation cited above has some good
> charts and figures as well as the full test case code.
>
> This is why I suggest either (both)
>
> * Use a scripting language that already is 'in process' with all the XML
> core languages you want to use (xquery, xslt etc)
> -> examples XProc, xmlsh
>
> * Encourage scripting languages developers to embed these XML languages
> directly into the scripting languages (say perl).
> -> This is hard work and may in fact involve re-implementing many of the
> core tools from scratch.
> -> Some of the work is done but is incomplete ... I've seen references to
> XSLT implementions native in perl where the author quoted  something like
>  "This isnt a complete implementation of XSLT 1.0 but it works pretty good
> for me".
>
>
> --
> David A. Lee
> http://x-query.com/mailman/listinfo/talk  http://www.calldei.com
> http://www.xmlsh.org
> 812-482-5224
>
>
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Tool development: by Perl-wrapped XQuery

2009-09-04T14:29:03Z

> This is why I suggest either (both)
>
> * Use a scripting language that already is 'in process' with all the XML
> core languages you want to use (xquery, xslt etc)
> -> examples XProc, xmlsh
>
> * Encourage scripting languages developers to embed these XML languages
> directly into the scripting languages (say perl).

While XProc is probably the cleanest solution, another approach in
particular for ad-hoc scripting might be to use an existing scripting
language that can run on top of the JVM (1), e.g. jython or jruby.
This gives you the flexibility of handy scripting but doesn't incur
the overhead of re-parsing huge documents. And of course on the JVM
you'll find implementations for everything-XML, from XSLT via XQuery
and XML databases to various validation tools etc., all nicely
co-existing based on the DOM and SAX (2).

Regards
Martin

(1) this might also be possible on other platforms, e.g. the CLR, but
I have no clue about those
(2) while I'm the first to bash the DOM for its horrible programming
interface, it's role as a common ground for all XML tools is
wonderful.

Tool development: by Perl-wrapped XQuery

2009-09-04T14:10:37Z

I agree that if you stick to use case #1 (only call xquery once or few 
times) and all your input and output are files (or file names) then 
pretty much any scripting language that lets you launch a subprocess is 
equivalent and you will have very little performance hit.  
Historically, I suggest many people use that use case primarily because 
its horrendously expensive not to (in most languages) so they have 
instead built 'monolithic' programs that do all the work within 1 
program (xquery, xslt etc) and take a list of filenames input and maybe 
filenames output to process.
This does work.

My point is that I suggest that programming model is no longer 
necessary, and was arrived at due to the poor performance of doing it 
any other way,
not because its a desirable way of coding.
If you use a language that can call xquery *efficiently* you can design 
in a more modular way and use 'itty bitty' xquery programs run thousands 
of times instead of one huge program run once,  just as efficiently, and 
in my opinion much easier to develop, debug, and understand.  It frees 
you to design and develop in modular ways, using languages of your 
choice for different operations instead of forcing you into a single 
monolithic application.
Of course it is a personal *opinion* that that is a 'better' way of 
doing things :)  I certainly enjoy it more.
Others may prefer writing one huge program that does everything.  
This exact same philosophical (and technical) debate is what 
distinguished unix from the mainframe model of programming.  I find it 
fascinating that the same fundamental design issues and discussions 
evolved again 40 years later.



David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Hans-Juergen Rennau wrote:
> Hello David,
>
> scripting and tool devlopment using script-wrapping are akin, but not quite the same. It seems to me important to distinguish clearly between two usecases:
>
> 1. the tool's functionality is provided by executing XQuery once (or only a few times)
> 2. the tool's functionality is provided by executing XQuery very often (as in your tests, hundreds of times)
>
> Almost all tools I have developed so far fall into category 1. Obviously there is NO PERFORMANCE PENALTY worth mentioning in this case: I believe the difference between calling the query via its Perl script or alternatively calling it directly from the command line would not be perceptible for a human being.
>
> It is not coincidence that category 1 can VERY often be used - rather, it is exactly the sequence philosophy which makes it a common case!  Example: if you have a query processing a document received as context item, then one can modify the query to process any number of input documents in succession, producing an aggregation of results by just adding 4 (!) lines:
>
> declare variable $fileNames external;
> declare variable $docs := for $f in tokenize(fileNames, "\s*") return doc($f);
> {for $d in $docs return
>    - now comes the original query with this modification: prepend $d before any path starting with / or // -
> }
>
> All that is left to be done is pass the names of the input documents as a space separated string to the query. Which is a trivial task for the little Perl section preceding the invocation.
>
> If one ignores this dependence on usecase, one arrives at the situation described by Michael Kay on http://www.saxonica.com/questions.html, last question: "Is it true that Altova XML is three times as fast as Saxon?"
>
> I never cease to be amazed by exactly this quality of XQuery - its ability to integrate vast sets of input resources with almost no increased effort as compared to the processing of a single resource.
>
> With kind regards,
> Hans-Juergen Rennau
>
>
>
>
> ----- Ursprüngliche Mail ----
> Von: David A. Lee 
> An: Hans-Juergen Rennau 
> CC: http://x-query.com/mailman/listinfo/talk; http://x-query.com/mailman/listinfo/talk
> Gesendet: Freitag, den 4. September 2009, 04:09:21 Uhr
> Betreff: Re:  Tool development: by Perl-wrapped XQuery
>
>
> In my opinion the problem with doing this in perl is that unless the xquery implementation itself is in perl > or atleast runs within the same process you will run into horrible performance problems.   See my (with Norm Walsh's) paper :
>
> http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html
>
> What we found is that for our test cases there is a 100-200x (yes 10000 % - 20000 % ) performance penalty of using a scripting language to call xml processing programs.   This *can* be optimized but the exact use cases of using a off-the-shelf scripting language to do this kind of thing is typically by the audience of people who do not want to spend the extra effort to optimize it, or who are not experts in the type of software development/languages required to  do it, or both. i.e its exactly why they are using scripting - so they don't have to do all that extra work.
>
> This is the primary reason xmlsh was invented instead of re-using an existing scripting language.   I took a "toy" program in a scripting language it worked great.
> But when I loaded up all the files I needed it to run it died a horrible death.   This is what I call "The Brick Wall" and why scripting XML processes fail so many of us.   The presentation cited above has some good charts and figures as well as the full test case code.
>
> This is why I suggest either (both)
>
> * Use a scripting language that already is 'in process' with all the XML core languages you want to use (xquery, xslt etc)
> -> examples XProc, xmlsh
>
> * Encourage scripting languages developers to embed these XML languages directly into the scripting languages (say perl).
> -> This is hard work and may in fact involve re-implementing many of the core tools from scratch.
> -> Some of the work is done but is incomplete ... I've seen references to XSLT implementions native in perl where the author quoted  something like
>   "This isnt a complete implementation of XSLT 1.0 but it works pretty good for me".
>
>
> -- David A. Lee
>
>
>       
>
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

Tool development: by Perl-wrapped XQuery

2009-09-04T11:39:10Z

Hans-Juergen Rennau wrote:
> As Daniela Florescu emphasized, XQuery is an information processing language, rather than a query language, and this makes it a great implementation language for tool development. But two problems discourage the use of "raw queries" - without frontent or wrapping of any sort - as tools:
> 
> a) no default parameters
> b) no standard-based access to system commands, namely no file discovery

I think that these are very good examples of functions that need to be 
added to EXPath.

> a) is in conflict with the requirement of powerful tools to offer a great number of options; and b) forces the caller to specifiy all input files - possibly hundreds or more - explicitly. (Or to provide secondary input files containing the file names, making things complicated..)
> 
> So David A. Lee recently wrote:
> "I had to write a bunch of "framework" code in Java just to arrange to *call* xquery ( you know, find the input files, get the xquery files, setup the parameters, call it, serialize the results etc)."
> 
> David's contribution drew my attention to xmlsh which seems to me a fascinating approach. But I feel that (at least presently) tool development per XQuery needs a lightweight approach to wrapping. (No?) I have taken to an extremely simple technique of wrapping queries in Perl, thus overcoming the problems mentioned above: 
> 
> a) get the powerful command line interface of Perl scripts (options and arguments, defaulting)
> b) leave the preliminary file discovery to Perl (e.g. per glob)
> 
> The queries themselves I insert as here documents right into the Perl code, one function per query, like this:
> 
> sub xq_foo {       
>     return << 'EOF';
> (: here comes query 'foo', possibly thousands of lines ...  :)
> EOF
> }
> 
> This is significant for keeping things simple: one Perl script can infact be a library with dozens of queries of unlimited complexity. (This way the tool software can be structured into a small number of deliverables each of which is a whole application, rather than a script in the conventional sense.) The approach is rounded off by letting Perl select the query, construct the command line invocation of your XQuery processor (e.g. Saxon), calling the processor and returning the output on stdout. Having written a couple of such Perl-XQuery-hybrids you recognize a uniform pattern of how to organize this little translation of Perl input into XQuery invocation, including input checks, so that from the third specimen onwards you have, in terms of effort, virtually no overhead, just write query after query INTO your Perl frame, and add invocation branch after invocation branch.
> 
> Now my questions: 
> - would you recommend alternatives for rapid tool development?

I can't fault what you're doing right now - but it would be nice to be 
able to handle this in pure XQuery.

> - if taking a similar approach - would you like to recommend special details, perhaps the script language actually used, or other aspect?

I think that pure functional XQuery is always going to have issues with 
tool writing, since it believes that the world doesn't change during the 
time a query is executed (and goes to great lengths to make this seem 
so). This is clearly a good use case for XQuery Scripting Extensions, 
which introduces a notion of time passing and the world changing whilst 
still remaining largely functional.

http://www.w3.org/TR/xquery-sx-10/

John

-- 
John Snelson, Oracle Corporation            http://snelson.org.uk/john
Berkeley DB XML:            http://oracle.com/database/berkeley-db/xml
XQilla:                                  http://xqilla.sourceforge.net

Tool development: by Perl-wrapped XQuery

2009-09-04T10:59:19Z

Hello David,

scripting and tool devlopment using script-wrapping are akin, but not quite the same. It seems to me important to distinguish clearly between two usecases:

1. the tool's functionality is provided by executing XQuery once (or only a few times)
2. the tool's functionality is provided by executing XQuery very often (as in your tests, hundreds of times)

Almost all tools I have developed so far fall into category 1. Obviously there is NO PERFORMANCE PENALTY worth mentioning in this case: I believe the difference between calling the query via its Perl script or alternatively calling it directly from the command line would not be perceptible for a human being.

It is not coincidence that category 1 can VERY often be used - rather, it is exactly the sequence philosophy which makes it a common case! Example: if you have a query processing a document received as context item, then one can modify the query to process any number of input documents in succession, producing an aggregation of results by just adding 4 (!) lines:

declare variable $fileNames external;
declare variable $docs := for $f in tokenize(fileNames, "\s*") return doc($f);
{for $d in $docs return
- now comes the original query with this modification: prepend $d before any path starting with / or // -
}

All that is left to be done is pass the names of the input documents as a space separated string to the query. Which is a trivial task for the little Perl section preceding the invocation.

If one ignores this dependence on usecase, one arrives at the situation described by Michael Kay on http://www.saxonica.com/questions.html, last question: "Is it true that Altova XML is three times as fast as Saxon?"

I never cease to be amazed by exactly this quality of XQuery - its ability to integrate vast sets of input resources with almost no increased effort as compared to the processing of a single resource.

With kind regards,
Hans-Juergen Rennau

----- Ursprüngliche Mail ----
Von: David A. Lee
An: Hans-Juergen Rennau
CC: http://x-query.com/mailman/listinfo/talk; http://x-query.com/mailman/listinfo/talk
Gesendet: Freitag, den 4. September 2009, 04:09:21 Uhr
Betreff: Re: Tool development: by Perl-wrapped XQuery

In my opinion the problem with doing this in perl is that unless the xquery implementation itself is in perl > or atleast runs within the same process you will run into horrible performance problems. See my (with Norm Walsh's) paper :

http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html

What we found is that for our test cases there is a 100-200x (yes 10000 % - 20000 % ) performance penalty of using a scripting language to call xml processing programs. This *can* be optimized but the exact use cases of using a off-the-shelf scripting language to do this kind of thing is typically by the audience of people who do not want to spend the extra effort to optimize it, or who are not experts in the type of software development/languages required to do it, or both. i.e its exactly why they are using scripting - so they don't have to do all that extra work.

This is the primary reason xmlsh was invented instead of re-using an existing scripting language. I took a "toy" program in a scripting language it worked great.
But when I loaded up all the files I needed it to run it died a horrible death. This is what I call "The Brick Wall" and why scripting XML processes fail so many of us. The presentation cited above has some good charts and figures as well as the full test case code.

This is why I suggest either (both)

* Use a scripting language that already is 'in process' with all the XML core languages you want to use (xquery, xslt etc)
-> examples XProc, xmlsh

* Encourage scripting languages developers to embed these XML languages directly into the scripting languages (say perl).
-> This is hard work and may in fact involve re-implementing many of the core tools from scratch.
-> Some of the work is done but is incomplete ... I've seen references to XSLT implementions native in perl where the author quoted something like
"This isnt a complete implementation of XSLT 1.0 but it works pretty good for me".

-- David A. Lee

Tool development: by Perl-wrapped XQuery

2009-09-04T10:11:56Z

On Fri, Sep 4, 2009 at 5:03 AM, David A. Lee wrote:
> But equally important, IMHO,  is to be able to "glue" the data from one
> module to another efficiently.  That needs common in-memory formats.
> And a way in the language to make calling these modules and passing data
> to/from them easy in a way thats 'natural' for that language.
> That may mean, for example, that the language runtime define a standard XML
> in-memory representation and every module can use it natively.
> So that you can have say variables of XML Documents and not have to
> transform them to each module's internal representation (rebuild the
> document).

This is a very important point. I am mostly dealing with XSLT, XQuery,
and the rest of the stack from .NET perspective. There, we actually do
have a standard representation of XML - IXPathNavigable factory
interface and the associated XPathNavigator cursor class. In fact, it
is specifically designed in a way that makes a straightforward mapping
to XDM possible (to quote MSDN: "The XPathNavigator class in the
System.Xml.XPath namespace is an abstract class which defines a cursor
model for navigating and editing XML information items as instances of
the XQuery 1.0 and XPath 2.0 Data Model."). All .NET XML APIs provide
support for those interfaces one way or another - all in-memory XML
tree representations (and there are three of them in the standard
library) can be treated as IXPathNavigable, .NET stock XSLT 1.0
implementation takes an IXPathNavigable as a source, stock XML Schema
validator also works with it, and so on. The interface is also simple
enough that I can take virtually any data source (e.g., an object
graph), easily expose it as an XDM instance, and feed that to any
XML-processing pipeline (usually XSLT) as an input, with no need to
copy data back and forth.

So, all stock components integrate really well together, with no
representation mismatch (and yet representation itself being nicely
abstracted away), but they are dated (XSLT 1.0... 'nuff said). And
third-party components often don't integrate that well. For example,
there's no XSLT 2.0 implementation for .NET that works directly over
IXPathNavigable input trees (without making an extra copy), or that
provides output in a form for which an XPathNavigator can be
immediately provided without copies. And the first and only XQuery
implementation which supports all of the above that I'm aware of is
the recently released XQSharp.

Tool development: by Perl-wrapped XQuery

2009-09-04T09:39:50Z

>
> That's the downside of issuing specifications and waiting for the
> implementations to happen. Thank God XML does have a standard
> serialization, so any document can migrate from one implementation
> to another :-)
>
I wish that were true.  Its true for "Documents" but not for everything 
XML tools want to process and consume.
My current peeve is sequences.
There is *NO* standard representation of sequences period.  (please 
please tell me I'm wrong !)
A real example:
    toolA : produces a sequence of nodes
    toolB : accepts a sequence of nodes

There is no standard way of glueing these together in any language or 
serialization format that I know of.  Its all vendor specific.
And you cant even rely on the worse case of text serialization
. 
A typical part-solution/hack is to wrap the sequences in a document, 
then unwrap them on the target.  This works sometimes.
It fails miserably for things like sequences of attributes,  or even 
sequences of integers.



>
> Isn't .NET a solution to that problem?
>
It is a nice framework.  But someone still has to write all the pieces 
and agree to as common representation for all the data interfaces.
And the common .NET languages are about at the same level as Java ... 
although there is a windows .net scripting language I wouldn't call it 
"simple".
So I dont agree that .NET is a "solution".   Its a technology framework 
not a solution in itself.
Saxon has a great .NET binding (runs the JVM directly within .NET but 
exposes pure .NET interfaces).
But its at the same level of complexity to use as java and doesnt have 
*all* the pieces.  Its got lots, but not all.

I'd love to see a comprehensive XML tools framework in .NET which is 
actually as simple to use as scripting.  I think it could be done.
But to be 'really simple' , IMHO, would require a .NET implementation of 
a scripting syntax.

ruby.net anyone :) ?


-David

-- 
David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Tool development: by Perl-wrapped XQuery

2009-09-04T09:03:21Z


>
>> * Use a scripting language that already is 'in process' with all the
>> XML core languages you want to use (xquery, xslt etc)
>>  -> examples XProc, xmlsh
>>
>> * Encourage scripting languages developers to embed these XML
>> languages directly into the scripting languages (say perl).
>
> More precisely, write bindings to implementations in C/C++.
>
Only if the scripting language itself is written in C/C++ or has easy 
bindings to it.  If its in another language then bindings in that 
language are useful.

But equally important, IMHO,  is to be able to "glue" the data from one 
module to another efficiently.  That needs common in-memory formats.
And a way in the language to make calling these modules and passing data 
to/from them easy in a way thats 'natural' for that language.
That may mean, for example, that the language runtime define a standard 
XML in-memory representation and every module can use it natively.
So that you can have say variables of XML Documents and not have to 
transform them to each module's internal representation (rebuild the 
document).

To do this well I suggest is more then just writing bindings.  Its a 
language level problem as well as a binding problem.

-D






David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Tool development: by Perl-wrapped XQuery

2009-09-04T08:51:15Z

xmlstar looks very useful !!!

Taking a quick look though it seems it will suffer exactly the same 
performance problems as most other scripting languages because it has no 
way to "glue" the commands together in process.  So you end up having a 
process execution for each command and full text serialization in and out.
( also missing the one key thing this list is about aka "xquery" ).


David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Jakob Fix wrote:
> Talking of tools, what is the experience of people on this list with
> xmlstarlet (http://xmlstar.sourceforge.net/) which seems to me like a
> close contender to xmlsh (but haven't looked closely)?
>
> cheers,
> Jakob.
>
>
>
> On Fri, Sep 4, 2009 at 13:09, David A. Lee wrote:
>   
>>
>>     
>>> Now my questions: - would you recommend alternatives for rapid tool
>>> development? - if taking a similar approach - would you like to recommend
>>> special details, perhaps the script language actually used, or other aspect?
>>>
>>>       
>> In my opinion the problem with doing this in perl is that unless the xquery
>> implementation itself is in perl or atleast runs within the same process you
>> will run into horrible performance problems.   See my (with Norm Walsh's)
>> paper :
>>
>> http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html
>>
>> What we found is that for our test cases there is a 100-200x (yes 10000 % -
>> 20000 % ) performance penalty of using a scripting language to call xml
>> processing programs.   This *can* be optimized but the exact use cases of
>> using a off-the-shelf scripting language to do this kind of thing is
>> typically by the audience of people who do not want to spend the extra
>> effort to optimize it, or who are not experts in the type of software
>> development/languages required to  do it, or both. i.e its exactly why they
>> are using scripting - so they don't have to do all that extra work.
>>
>> This is the primary reason xmlsh was invented instead of re-using an
>> existing scripting language.   I took a "toy" program in a scripting
>> language it worked great.
>> But when I loaded up all the files I needed it to run it died a horrible
>> death.   This is what I call "The Brick Wall" and why scripting XML
>> processes fail so many of us.   The presentation cited above has some good
>> charts and figures as well as the full test case code.
>>
>> This is why I suggest either (both)
>>
>> * Use a scripting language that already is 'in process' with all the XML
>> core languages you want to use (xquery, xslt etc)
>> -> examples XProc, xmlsh
>>
>> * Encourage scripting languages developers to embed these XML languages
>> directly into the scripting languages (say perl).
>> -> This is hard work and may in fact involve re-implementing many of the
>> core tools from scratch.
>> -> Some of the work is done but is incomplete ... I've seen references to
>> XSLT implementions native in perl where the author quoted  something like
>>  "This isnt a complete implementation of XSLT 1.0 but it works pretty good
>> for me".
>>
>>
>> --
>> David A. Lee
>> http://x-query.com/mailman/listinfo/talk  http://www.calldei.com
>> http://www.xmlsh.org
>> 812-482-5224
>>
>>
>>
>> _______________________________________________
>> http://x-query.com/mailman/listinfo/talk
>> http://x-query.com/mailman/listinfo/talk
>>
>>     
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://x-query.com/pipermail/talk/attachments/20090904/c0df45fc/attachment.htm

Tool development: by Perl-wrapped XQuery

2009-09-04T08:46:52Z


> While XProc is probably the cleanest solution, another approach in
> particular for ad-hoc scripting might be to use an existing scripting
> language that can run on top of the JVM (1), e.g. jython or jruby.
> This gives you the flexibility of handy scripting but doesn't incur
> the overhead of re-parsing huge documents. And of course on the JVM
> you'll find implementations for everything-XML, from XSLT via XQuery
> and XML databases to various validation tools etc., all nicely
> co-existing based on the DOM and SAX (2).
>
> Regards
> Martin
>   

jypthon or jruby solve the performance problem but they dont solve the 
complexity problem.
The fact is these XML API technologies are a royal pain to call and 
worse to glue to together.
If you can find one vendor/ implementation that has all the pices its a 
tad easier (document types match !) but its still a bit of hell.
Then add technologies that VendorA doesnt do and try to wrap VendorB's 
data type and push it through VendorA's API.
I think you'll find it about as painful to to use jruby or jython to say 
call  xinclude , then rng validate then xquery then pass its results to 
xslt as writing it in pure java.
In xproc this is very clean, but admittedly verbose.  (guess about 40 
lines of xproc code).
In xmlsh its as simple as 
    xinclude file | rngvalidate schema | xquery -f schema | xslt -f 
stylesheet

To get these kind of languages (jruby etc) into shape where XML is 
actually *easy to use* is going to require work.  
Possibly at the language level to make it really easy.
If this is done to more scripting languages (making it both efficient 
and simple to use) rather then just "possible" I suspect adoption of the 
core technologies would skyrocket.  

Thats what we need, IMHO, is more *simplicity* of using and intemixing 
the core xml tools in a way that doesn't incur huge performance costs.
But if they dont include the core technologies such as XQuery , XSLT 
(2.0 ideally) , xinclude , validation ... in such a way that they can be 
easily intermixed accessed then people wont use them.  The reason people 
are using scripting languages is for ease of use.
I talked to a Ruby programmer at work after a demonstration on XQuery 
and his answer was "I can do that in ruby ... why would I want to learn 
xquery ?  And ruby can do a LOT more then just xml processing " ... 
After seeing some of his code I had to admit he had a point.  I still 
have been unable to successfully convince him the value of xquery over 
Ruby's XML API's.  Although he's never had to do anything *complicated* 
... but because of that he cant envision the need for a different 
language.   He's at the stage I was when I first started XML Programming 
in java ... I would look at XSLT programs "Why would I want to write 
something that complicated when I can just use the easy DOM calls in 
java.   It wasn't until several years of having to do *difficult* 
transformations that I finally realized how big my java programs were 
getting.



David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Tool development: by Perl-wrapped XQuery

2009-09-04T08:09:21Z



>
> Now my questions: 
> - would you recommend alternatives for rapid tool development? 
> - if taking a similar approach - would you like to recommend special details, perhaps the script language actually used, or other aspect?
>   
In my opinion the problem with doing this in perl is that unless the 
xquery implementation itself is in perl or atleast runs within the same 
process you will run into horrible performance problems.   See my (with 
Norm Walsh's) paper :

http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html

What we found is that for our test cases there is a 100-200x (yes 10000 
% - 20000 % ) performance penalty of using a scripting language to call 
xml processing programs.   This *can* be optimized but the exact use 
cases of using a off-the-shelf scripting language to do this kind of 
thing is typically by the audience of people who do not want to spend 
the extra effort to optimize it, or who are not experts in the type of 
software development/languages required to  do it, or both. i.e its 
exactly why they are using scripting - so they don't have to do all that 
extra work.

This is the primary reason xmlsh was invented instead of re-using an 
existing scripting language.   I took a "toy" program in a scripting 
language it worked great.
But when I loaded up all the files I needed it to run it died a horrible 
death.   This is what I call "The Brick Wall" and why scripting XML 
processes fail so many of us.   The presentation cited above has some 
good charts and figures as well as the full test case code.

This is why I suggest either (both)

* Use a scripting language that already is 'in process' with all the XML 
core languages you want to use (xquery, xslt etc)
 -> examples XProc, xmlsh

* Encourage scripting languages developers to embed these XML languages 
directly into the scripting languages (say perl).
-> This is hard work and may in fact involve re-implementing many of the 
core tools from scratch.
-> Some of the work is done but is incomplete ... I've seen references 
to XSLT implementions native in perl where the author quoted  something like
   "This isnt a complete implementation of XSLT 1.0 but it works pretty 
good for me".


-- 
David A. Lee
http://x-query.com/mailman/listinfo/talk  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

Tool development: by Perl-wrapped XQuery

2009-09-04T00:08:09Z

As Daniela Florescu emphasized, XQuery is an information processing language, rather than a query language, and this makes it a great implementation language for tool development. But two problems discourage the use of "raw queries" - without frontent or wrapping of any sort - as tools:

a) no default parameters
b) no standard-based access to system commands, namely no file discovery

a) is in conflict with the requirement of powerful tools to offer a great number of options; and b) forces the caller to specifiy all input files - possibly hundreds or more - explicitly. (Or to provide secondary input files containing the file names, making things complicated..)

So David A. Lee recently wrote:
"I had to write a bunch of "framework" code in Java just to arrange to *call* xquery ( you know, find the input files, get the xquery files, setup the parameters, call it, serialize the results etc)."

David's contribution drew my attention to xmlsh which seems to me a fascinating approach. But I feel that (at least presently) tool development per XQuery needs a lightweight approach to wrapping. (No?) I have taken to an extremely simple technique of wrapping queries in Perl, thus overcoming the problems mentioned above: 

a) get the powerful command line interface of Perl scripts (options and arguments, defaulting)
b) leave the preliminary file discovery to Perl (e.g. per glob)

The queries themselves I insert as here documents right into the Perl code, one function per query, like this:

sub xq_foo {       
    return << 'EOF';
(: here comes query 'foo', possibly thousands of lines ...  :)
EOF
}

This is significant for keeping things simple: one Perl script can infact be a library with dozens of queries of unlimited complexity. (This way the tool software can be structured into a small number of deliverables each of which is a whole application, rather than a script in the conventional sense.) The approach is rounded off by letting Perl select the query, construct the command line invocation of your XQuery processor (e.g. Saxon), calling the processor and returning the output on stdout. Having written a couple of such Perl-XQuery-hybrids you recognize a uniform pattern of how to organize this little translation of Perl input into XQuery invocation, including input checks, so that from the third specimen onwards you have, in terms of effort, virtually no overhead, just write query after query INTO your Perl frame, and add invocation branch after invocation branch.

Now my questions: 
- would you recommend alternatives for rapid tool development? 
- if taking a similar approach - would you like to recommend special details, perhaps the script language actually used, or other aspect?

Any feedback would be highly welcome!

With kind regards,
Hans-Juergen Rennau

size of XQuery developer community

2009-09-03T17:00:32Z

On Thu, Sep 3, 2009 at 8:21 AM, Gary Lewis wrote:
> ... but the various implementors might want to consider what they
> have in common and then do something (eg, create an xquery learning
> resource site) that would benefit all xquery developers regardless of
> the flavor of xquery they use.

I think the XQuery Wikibook http://en.wikibooks.org/wiki/XQuery is an
ideal starting point.  It's got a great base of material already.
It's currently very eXist-centric, but there's no reason that there
couldn't be articles about other implementations.

- Joe

[ANN] XQSharp 1.0 (XQuery for the .NET Framework)

2009-09-03T14:57:01Z

The XQSharp team are pleased to announce the long-awaited release of 
XQSharp 1.0

XQSharp is an XQuery processor for the .NET Framework.  It is developed 
by Clinical & Biomedical Computing Limited, a member of the W3C.

XQSharp 1.0 is the first complete release of XQSharp containing our .NET 
API, which has been designed to work in conjunction with the classes in 
the System.Xml namespace.  The full API Reference can be viewed at 
http://www.xqsharp.com/xqsharp/doc/.

Key Features:
* Up-to-date and Standards Compliant (exemplary performance in the 
XQuery Test Suite)
* Schema-Aware
* High performance
* Comprehensive and well documented API, with full IntelliSense 
documentation.
* All optional XQuery features implemented
* Full implementation of the XDM Serialization specification.
* XPath 2.0 support sharing the same optimization and runtime engine as 
XQuery.
* Can be used in Low and Medium Trust environments.

It is completely free for non-commercial use; commercial licenses are 
also available.

For more information please visit http://www.xqsharp.com/ or contact us 
at http://x-query.com/mailman/listinfo/talk

Oliver Hallam
http://www.xqsharp.com

How XML Threatens Big Data

2009-09-03T12:35:18Z

Ivan Pedruzzi schrieb:
> On Wed, Sep 2, 2009 at 4:49 AM, Michael Ludwig 
> wrote:

>> Also, it never fails to amaze me how people can think that documents
>> exceeding a certain size - 1 MB, okay: 5 MB - are a good idea. Just
>> imagine the amount of processing that has to be done before
>> discovering on parse completion that something's wrong.
>
> If you consider 5 MB documents a bad idea may be you are not using the
> appropriate tools :)
>
> Ivan Pedruzzi
> Progress DataDirect Technologies

If I control the whole processing pipeline I can use the tools I want.
But if I publish documents over the wire I may not be in a position to
tell consumers what tools to use.

-- 
Michael Ludwig

size of XQuery developer community

2009-09-03T11:21:39Z

The difference between R and XQuery however is that R is a language
with one implementation.
Thus the site for downloading R can also provide all this information
to users, for XQuery it is too fragmented.

Best Regards,
Bryan Rasmussen

On Tue, Sep 1, 2009 at 5:40 PM, Gary Lewis wrote:
> On Wed Aug 26 17:39:05 PDT 2009, Daniela Florescu wrote:
>
>> ... XQuery has tremendous potential for adding
>> value to customers, but the proof isn't there yet, and the path isn't
>> clear either.
>>
>> There has to be a larger XQuery pool of expertise ...
>
> Hi Daniela - I finally got around to reading this entire thread thru
> to 9/1. It's truly humbling to see how much about XQuery, XML, etc I
> do not know.
>
> But perhaps my newcomer status will provide a useful perspective.
>
> First some quick background. I definitely don't yet qualify as an
> XQuery developer. But I've been working pretty conscientiously over
> the past 4 or 5 months. I come to XQuery with decades of SQL, data
> warehouse design, and policy analysis background (ie, I see databases
> as a means to analysis, not an end in themselves). I got interested in
> XQuery because I needed a tool that would let me query the Web in a
> similar fashion to querying relational databases. The Web and web
> query tools are still primitive by comparison, but I've been quite
> pleased with what's possible. For example, my latest XQuery
> demonstration project mashed up US Dept of Education data with Federal
> Reserve data to examine the question of whether higher education in
> the US is countercyclical. See:
> http://garymlewis.com/instchg/2009/08/10/another-xquery-use-case-is-higher-education-countercyclical/
>
> Given a choice, I'll almost always choose tools with power and
> capability versus those with ease-of-use but limited application. So,
> for example, I'm willing to curse and beat my head against a wall
> every time I use the R stat programs because I know that somehow
> there's a solution in R and that someone in the R community will know
> what the solution is.
>
> I've probably written several thousand XQuery programs now. But I only
> feel comfortable in a very narrow niche. There is just oodles about
> XML and all the other X standards and tools that I do not know. Some
> of this is surely just beginner's lament (ie, the notion that anything
> you can't learn instantly is way too complex). But some too is a steep
> learning curve in the absence of adequate support for learning.
>
> You ask why there is not more XQuery expertise. The XQuery community
> might benefit from looking at the R community [see:
> http://www.r-project.org/ ] and the considerable help resources
> available there. It's easier for a developer to get untracked with any
> new tool if there is visible help available. With regard to R, I'm
> talking here about specialized search sites, repositories of learning
> materials, open source and free tools, dozens upon dozens of forums,
> examples as code fragments, online books and manuals, and a community
> of very active R developers/enhancers.
>
> I like XQuery a lot. I will continue to use it. And no doubt in the
> course of using it, I'll fill in the many holes of what I don't know.
> But I could sure benefit from something like an XQuery learning
> resource site.
>
> Gary
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

size of XQuery developer community

2009-09-03T09:21:52Z

> The difference between R and XQuery however is that R is a language
> with one implementation.
> Thus the site for downloading R can also provide all this information
> to users, for XQuery it is too fragmented.
>
> Best Regards,
> Bryan Rasmussen

Hi Bryan - Absolutely true. And that's part of the point I was trying
to make. If the community of xquery developers is small, part of the
reason is the multitude of xquery implementations. That's not going to
change, but the various implementors might want to consider what they
have in common and then do something (eg, create an xquery learning
resource site) that would benefit all xquery developers regardless of
the flavor of xquery they use.

Yes, very definitely I am saying that part of the cause for Daniela's
lament originates in the balkanization of the xquery community.

Gary

request: get the attribute value

2009-09-02T17:15:33Z

dengping wei wrote:

> staticContext.declareNamespace("sawsdl",
> "    http://www.w3.org/ns/sawsdl");

> xmlns:sawsdl="http://www.w3.org/ns/sawsdl"

  Both strings do not match (see the extra whitespaces in Java.)

  Regards,

-- 
Florent Georges
http://www.fgeorges.org/

request: get the attribute value

2009-09-02T14:08:46Z

> But when I want to get value of attribute "sawsdl:modelReference", I use the
> query
>
> for $x in
> doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType
> return {$x/@sawsdl:modelReference}
>
> It returns nothing. How could I get the value of the attribute "
> sawsdl:modelReference"?
Only to be sure about that: did you declare the namespace? You would
do that by adding the line

declare namespace sawsdl="http://www.w3.org/ns/sawsdl";

As an alternative you could use the query

for $x in doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType
return {$x/@*:modelReference}

which looks for attributes "modelReference" in all namespaces.

Markus

On Wed, Sep 2, 2009 at 12:36 PM, dengping wei wrote:
> Dear all,
>
> I have a question when I using xquery the get the attribute value of an
> element.
>
> when my query is
> for $x in doc(\"aa.wsdl\")/wsdl:
> definitions/wsdl:types/xsd:schema/xsd:complexType
> return {$x}
> It returns the results as follows:
> 
>    >                     xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
>
> xmlns:impl="http://dmas.dfki.de/axis/services/CarPrice-impl"
>                     xmlns:sawsdl="http://www.w3.org/ns/sawsdl"
>
> xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
>                     xmlns:intf="http://dmas.dfki.de/axis/services/CarPrice"
>                     xmlns:apachesoap="http://xml.apache.org/xml-soap"
>                     xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/"
>                     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
>                     xmlns="http://dmas.dfki.de/axis/services/CarPrice"
>                     name="PriceType"
>
> sawsdl:modelReference="http://127.0.0.1/ontology/concept.owl#Price">
>         
>           
>           
>         
>       
> 
>
> Then I want to get the value of attribute "name", I use the query
> for $x in
> doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType
> return {$x/@name}
>
> It works and return the results: 
>
> But when I want to get value of attribute "sawsdl:modelReference", I use the
> query
>
> for $x in
> doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType
> return {$x/@sawsdl:modelReference}
>
> It returns nothing. How could I get the value of the attribute "
> sawsdl:modelReference"?
>
> Thanks very much.
>
>
>
> --
> Dengping WEI
> School of Computer Science,
> National University of Defense Technology,
> Changsha, Hunan  410073
> P.R. China
>
> _______________________________________________
> http://x-query.com/mailman/listinfo/talk
> http://x-query.com/mailman/listinfo/talk
>

request: get the attribute value

2009-09-02T13:36:50Z

Dear all, I have a question when I using xquery the get the attribute value of an element. when my query is for $x in doc(\"aa.wsdl\")/wsdl: definitions/wsdl:types/xsd:schema/xsd:complexType return {$x} It returns the results as follows: SOAP-ENC=" http://schemas.xmlsoap.org/soap/encoding/" xmlns:intf="http://dmas.dfki.de/axis/services/CarPrice" xmlns:apachesoap="http://xml.apache.org/xml-soap" xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://dmas.dfki.de/axis/services/CarPrice" name="PriceType" sawsdl:modelReference=" http://127.0.0.1/ontology/concept.owl#Price"> Then I want to get the value of attribute "name", I use the query for $x in doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType return {$x/@name} It works and return the results: But when I want to get value of attribute "sawsdl:modelReference", I use the query for $x in doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType return {$x/@sawsdl:modelReference} It returns nothing. How could I get the value of the attribute " sawsdl:modelReference"? Thanks very much. -- Dengping WEI School of Computer Science, National University of Defense Technology, Changsha, Hunan 410073 P.R. China -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090902/5f143868/attachment-0001.htm

request: get the attribute value

2009-09-02T11:57:41Z

dengping wei wrote: Hi, > Then I want to get the value of attribute "name", > I use the query [...] return {$x/@name} BTW, that does not give you the value of the attribute @name but that gives you the attribute node itself (that's why you end up with an attribute on your element 'node'. > [...] > {$x/@sawsdl:modelReference} > It returns nothing. How could I get the value of the > attribute " sawsdl:modelReference"? Because you wrote doc(\"aa.wsdl\") (with the backslashes) I guess you evaluate your queries directly from Java or another similar language. How do you bind the prefix 'sawsdl' to the correct URI on the static context? Regards, -- Florent Georges http://www.fgeorges.org/

How XML Threatens Big Data

2009-09-02T11:49:57Z

A. Steven Anderson schrieb: > How timely....;-) > http://dataspora.com/blog/xml-and-big-data/ Well, if data is tabular, what benefits can be obtained by storing it as XML? None that I could see. If you later need XML, it is really easy to generate. So why would you store it as XML in the first place? Because you've fallen victim to some hype? That does not mean that "XML fails". (See Nestor's comment on the blog.) If your data is full of optional stuff and pretty irregular in general, XML can mean a significant improvement. A twenty-table-join is not easy to understand. I've never set up Hibernate, but I think that isn't easy either - a handy 700 page manual is required to bridge the OO/RDBMS/OO mismatch. For those situations, objects or document databases may be a much better match. There's more than one tool in the box. Also, it never fails to amaze me how people can think that documents exceeding a certain size - 1 MB, okay: 5 MB - are a good idea. Just imagine the amount of processing that has to be done before discovering on parse completion that something's wrong. -- Michael Ludwig

Tuple types

2009-09-02T11:44:02Z

John Snelson wrote: Hi, > > This makes sense, though it would be a nuisance to have to > > define ref() and deref() every time. Is there any > > consideration to make them standard library functions? > My recollection is that the WG considered adding them and > rejected the idea on the grounds that functions that can be > defined in XQuery itself better belong in something like EXPath > function modules. It's highly likely that I'll submit such a > module to EXPath in the future. My first reaction was: "Why? Just use the HOF feature directly." But defining ref() and deref() would actually have the double advantage to 1/ make the intent clearer and 2/ enable one to provide them on a processor that does not support HOFs (yet.) Good idea. Regards, -- Florent Georges http://www.fgeorges.org/

Tuple types

2009-09-02T10:36:59Z

On Wed, Sep 2, 2009 at 3:44 AM, Florent Georges wrote: >> > This makes sense, though it would be a nuisance to have to >> > define ref() and deref() every time. Is there any >> > consideration to make them standard library functions? > >> My recollection is that the WG considered adding them and >> rejected the idea on the grounds that functions that can be >> defined in XQuery itself better belong in something like EXPath >> function modules. It's highly likely that I'll submit such a >> module to EXPath in the future. > > My first reaction was: "Why? Just use the HOF feature directly." It's an obvious solution in case of deref(), but having to type out identity lambdas in full every time will get repetitive fast, which is why I think ref() is still handy regardless.

Help with multiple deletes

2009-09-02T10:33:10Z

I guess your approach is a bit different than what the spec planned - their idea was that you'd just want to perform the updates, but not "witness" the effects. A different approach would be this: copy $x := doc("tree.xml") modify ( let $node := $x//tree[@ID = 1] /branch[@ID = 2] /branch[@ID = 13] /branch[@ID = 65] /branch[@ID = 15] /leaf[@ID = 25] let $empty-branches := $x//branch[count(.//leaf except $node) = 0] let $empty-trees := $x//tree[count(.//branch except $empty-branches) = 0] return delete nodes ($empty-trees, $empty-branches, $node) ) return $x There might be more efficient ways to find the nodes that need pruning, in particular if your implementation would support the ancestor:: axis, you could walk upwards from $node instead of scanning the whole tree for the $empty-* variables. Martin

How XML Threatens Big Data

2009-09-02T00:31:57Z

How timely....;-) http://dataspora.com/blog/xml-and-big-data/ -- A. Steven Anderson Independent Consultant -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/ddaefb71/attachment.htm

size of XQuery developer community

2009-09-01T21:37:30Z

Ron Bourret wrote: >> What's the single killer app for SQL? Ad hoc queries. For the most part, the DBMS platforms that preceded the arrival of SQL, the hierarchical and CODASYL models, implemented queries with traversal logic in application programs. There were exceptions, including IBM GIS and TRW GIM, that actually used English-like commands for searching, deleting, inserting and updating. But IMS and CODASYL were the defacto and de jure standard and writing a program using a known schema and sub-schema was how you implemented a query.

size of XQuery developer community

2009-09-01T20:41:00Z

Hmmmmm. What's the single killer app for SQL? In both cases, I don't think there is one, in the sense that the Web is the killer app for the Internet or spreadsheets were the killer app for PCs. I think that the combination of XQuery and XML databases simply make it possible to build applications that weren't feasible before because of limitations of the relational model to represent certain kinds of data, such as documents and semi-structured data. Whether any of these is killer really depends on what business you're in. If you're an accountant, I doubt you'll be interested. If you're a publisher, you probably are. -- Ron Andrew Welch wrote: > 2009/8/31 Daniela Florescu : >>> why do developers and architects so stubbornly refuse just to _listen_ ? >> Because we don't talk to them (a) enough and (b) with a message that they >> understand. > > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it?

size of XQuery developer community

2009-09-01T20:36:42Z

I agree with Daniela. For some real-world examples of this kind of thing, see: http://www.rpbourret.com/xml/UseCases.htm It's worth noting that people tried to build many of these applications on relational databases and failed. The article explains some of the reasons why. This article is based on a non-scientific survey of XML database companies. It was done in 2005 and included about half of the XML database companies at the time, including most of the major ones. -- Ron Daniela Florescu wrote: >> >> >> I honestly think that all the talk about XDM as a data model and XQ as >> a turing complete declarative language, etc., is sort of moot unless >> they do somehow enable something that RDBMS/BI/Data warehouses/search >> vendors can't readily do with existing approaches. > > Well, that's relatively simple to answer, I would say. > > Here is are some examples of what you could do with XML/XQuery that you > cannot do with the traditional three > tier stack (communication + application logic written in an imperative > language + declarative query layer) > > 1. Have schema flexibility. Start writing code without a fixed idea of > your schema, and/or change > your mind later. (one schema is evil enough, three layers of > superimposed schemas, in three different models, > are exponentially evil for flexibility and customizability. Almost no > evolution of schema is possible.) > > 2. Integrate text and documents into your information processing flow > without adding software license, yet another layer, > more mappings, and different programming languages. > > 3. Optimize the performance of your information processing program > globally, not locally, layer by layer, > by adding more and more intermediate caches and shortcuts, which in turn > will turn your code maintenance a nightmare. > > 4. Be done in 2 weeks instead of 10 months. Time to market, by writing > significantly less amount > of code, layer to layer connectors, plus data mappers from one model to > another, etc, etc. Less > useless and redundant code to write and later deal with. > > 5. Spend less time maintaining the code. If the XQuery optimizer is > doing a good job, you don't need > to constantly rewrite your code to fix performance problems, as you do > in java. Plus less number of lines of > code usually means less bugs. So smaller total cost of ownership of the > code. > > That would be my quick personal answer. > > But that of course works if you know what you are doing with XQuery .... > > Best > Dana

Tuple types

2009-09-01T17:38:41Z

Pavel Minaev wrote: > On Tue, Sep 1, 2009 at 6:44 AM, John Snelson wrote: >> Tuple types won't make it into XQuery 1.1, but the closures support in >> higher order functions allows you to effectively return sequences of >> sequences: >> >> (: Wrap a sequence as a function item :) >> declare function local:ref($arg as item()*) as function() as item()* >> { >> function() { $arg } >> }; >> >> (: Unwrap a sequence from a function item :) >> declare function local:deref($arg as function() as item()*) as item()* >> { >> $arg() >> }; > > This makes sense, though it would be a nuisance to have to define > ref() and deref() every time. Is there any consideration to make them > standard library functions? My recollection is that the WG considered adding them and rejected the idea on the grounds that functions that can be defined in XQuery itself better belong in something like EXPath function modules. It's highly likely that I'll submit such a module to EXPath in the future. John -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net

Help with multiple deletes

2009-09-01T16:35:13Z

Thanks Martin, that works. Except the 3rd delete doesn't run on the updated tree from the 2nd delete, so 2 copy statements are necessary. Is there a different way to do this, so the delete statements operate on the updated tree w/o doing a copy each time? i.e. some sort of piping construct ? copy $x := doc("tree.xml") modify ( delete node $x//tree[@ID = 1] /branch[@ID = 2] /branch[@ID = 13] /branch[@ID = 65] /branch[@ID = 15] /leaf[@ID = 25] ) return copy $y := $x modify ( delete node $y//tree//branch[count(.//leaf) = 0] ) return copy $z := $y modify ( delete node $z//tree[count(.//branch) = 0] ) return $z Thanks -----Original Message----- From: Martin Probst [mailto:http://x-query.com/mailman/listinfo/talk] Sent: Saturday, August 29, 2009 10:47 AM To: Newman, John W Cc: Michael Kay; http://x-query.com/mailman/listinfo/talk Subject: Re: Help with multiple deletes > After I spent a nice chunk of time testing this in stylus studio, I took the > concept and ported it into our application. Only to discover that the > implementation we are using, MXQuery 0.6.0, does not support ancestor! > Argh.. =) So if anyone thinks it can possibly be rewritten without using > ancestor (or any other axes I'm guessing), please share your thoughts. But > I think we are going to end up using an XSLT for this operation. You could simply transform twice, which might be less efficient, but its certainly simpler to write: copy $x := doc("tree.xml") modify ( delete node $x//tree[@ID = 1] /branch[@ID = 2] /branch[@ID = 13] /branch[@ID = 65] /branch[@ID = 15] /leaf[@ID = 25] ) return copy $y := $x modify ( delete node $x//tree//branch[count(.//leaf) = 0] , delete node $x//tree[count(.//branch) = 0] ) return $y Martin

Tuple types

2009-09-01T15:44:36Z

Hi Pavel, Tuple types won't make it into XQuery 1.1, but the closures support in higher order functions allows you to effectively return sequences of sequences: (: Wrap a sequence as a function item :) declare function local:ref($arg as item()*) as function() as item()* { function() { $arg } }; (: Unwrap a sequence from a function item :) declare function local:deref($arg as function() as item()*) as item()* { $arg() }; John Pavel Minaev wrote: > I can't find the link now, but I recall seeing some mentions of a > possibility of introducing some form of tuple type (in a sense of an > aggregate of arbitrary-typed values - including sequences) in XQuery > 1.1, specifically to tackle the multiple-return-values and > sequences-of-sequences problems. Are there any recent developments on > the subject? I don't see anything like this in the most recent draft, > but then higher-order functions aren't there either, and yet, judging > by their presence in Saxon 9.2, they are well-shaped by now. > > If this isn't going to make it into 1.1, are there any other new > features that I might have missed that would make it possible, for > example, for a function to return two sequences? > _______________________________________________ > http://x-query.com/mailman/listinfo/talk > http://x-query.com/mailman/listinfo/talk -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net

size of XQuery developer community

2009-09-01T13:34:02Z

> why do developers and architects so stubbornly refuse just to _listen_ ? Maybe we should try listening to them? To find out why they are not including xquery products in their architectures. I can make a few guesses - This are somethings that customers might tell you: 1) until recently, no update was available - making it simply not a choice. 2) there are some queries in my xquery project that don't perform well. Is this problem intrinsic to Xquery? The implementation? My xquery code? How do I fix it?(This is usually related to a join or predicate not pushed to a database because of the way the xquery was written. Often in a function that calls several other functions, that call other functions... eventually involving a thousand lines of xquery or so - making the exact problem difficult to locate) 3) I found the Books/Authors xquery examples helpful. Now, I have 5,000 database tables, and I need to join up to 50 of them at a time. Do you have any samples like that? 4) I want to include the output of a stored procedure that retuns an Object type in my xquery. How do I do that? 5) How can I use Xquery for ETL? (it needs to be better than my existing ETL tool). - Mike _____ From: Daniela Florescu [mailto:http://x-query.com/mailman/listinfo/talk] Sent: Monday, August 31, 2009 3:39 PM To: Hans-Juergen Rennau Cc: http://x-query.com/mailman/listinfo/talk Subject: Re: size of XQuery developer community Only a few days ago it occurred to me that "information processing language" may be better than "data processing language", Thanks for saying this. As soon as I hit the send button I remembered that I promised myself never use "data" in connection to XML, but "information". (I am a database person, certain habits are hard to break....) But yes, I would think too that XQuery is a declarative information processing language. why do developers and architects so stubbornly refuse just to _listen_ ? Because we don't talk to them (a) enough and (b) with a message that they understand. I remember years ago, when I was really frustrated that architects would not _listen_ that Xquery was this wonderful and useful thing, I asked advice from Jim Gray. He was always ready to listen and help, and, even though he was too much of a believer in SQL and schemas to start embracing XQuery, he gave me the following advice. If you want people to listen then (a) write articles and give talks, (b) write software that works, (c) make interesting demos. Then repeat, and don't loose patience. I think that at this point we have around XQuery lots of software that does work: databases, in memory processors, etc. What we can do is write more articles, give more talks (Web 2.0, eGov 2.0 anyone ?), and make more interesting demos. About demos: I am still surprised that nobody is trying XQuery's mashup strength on the examples on http://www.programmableweb.com/. Wonderful site, BTW. Or that noone is using XQuery on the http://www.data.gov/ site and win the price of the best eGovernment application.... Best regards Dana With kind regards, Hans-Juergen ----- Ursprüngliche Mail ---- Von: Daniela Florescu An: Hans-Juergen Rennau CC: HYPERLINK "mailto:http://x-query.com/mailman/listinfo/talk"http://x-query.com/mailman/listinfo/talk Gesendet: Montag, den 31. August 2009, 08:58:34 Uhr Betreff: Re: size of XQuery developer community a widespread lack of a _deep_ understanding of XML. Hans-Juergen, Yes, you are right. There is a deep misunderstanding in IT of our core message about the role of XML in the new architectures. XML is not syntax, it can/should be the primary way of modeling data. XQuery is not a query language (oh well), but a full blown (declarative) data processing language. However, if the world misunderstands it, it is hardly the world's fault. It is that we do not express this message in a clear, concise and convincing enough manner -- and probably not in the right places, either. I hope we don't have to wait 20 years for the message to be heard.... Best regards Dana _______________________________________________ HYPERLINK "mailto:http://x-query.com/mailman/listinfo/talk"http://x-query.com/mailman/listinfo/talk http://x-query.com/mailman/listinfo/talk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/3ff1d2a6/attachment.htm

size of XQuery developer community

2009-09-01T13:13:18Z

Hans-Juergen Rennau schrieb: > > of course you are right in that XSLT and XQuery are built on the XDM, > not on XML as a serialization format, and you are right in that the > XDM can be completely separated from the serialization format. But I > prefer to view XML and the XDM as one organic whole, and exactly this > is what I meant by the term "deep understanding": an understanding > covering three _levels_ of XML: level #1: the lexical; level #2: the > inforset (= structural model totally unconcerned with aspects of > processing); level #3: the XDM extending the infoset in order to > embrace the reality of processing = a global information model which > preserves the structural achievements of the infoset and embeds it > into a stunningly flexible definition of information value. Each level > evolved from the previous one, I think, was it not so, historically? Hallo Hans-Jürgen, historically, I'm a newbie; but the other day on XML-Dev, one SGML/XML veteran said that it was rather the work of "ladling committees" (had to look up the word - seems similar to "anflanschen" in what it conveys) ladling on patch after patch to XML. RE: [xml-dev] Xml Revisited - Len Bullard http://markmail.org/message/n5zscmypan4rzmaj I really like this realistic view of things, but I also have a lot of sympathy for idealistic interpretations. As for the three levels you're referring to, I don't know enough about level 2 - the XML InfoSet - in order to locate it in the scheme of things. What I've learnt (from Ken Holman) is that with XML, syntax came first, and data models (plural) are an afterthought. Maybe a simple and obvious thing to many, but - at least to me - crucial to understanding this whole XML business. Like a lot of things, it only makes sense seen with history in mind. > This evolution is a wonderful achievement Why not call it an Evolution? The process that has brought forth XML is too visible in order for us to call it a Creation. And it works too well to call it a Fabrication. (Although unfortunately, some fabrications work pretty well, too.) > and I am emphatically against "marketing" XQuery and XSLT as tree > processing languages, as if you could come along with some tree and > get it processed. Well, you can. You just have to make sure your tree is represented in a format amenable to an XML processing chain. Quite often, I start out with documents like , parse those into a DOM or XOM, and then append data retrieved from various sources (HTTP, SQL, XQuery) to then finally hand off the XML tree to XSLT and generate an HTML tree. > What should be done is rather: encourage people to get familiar with > the new reality which XML (via XDM) constitutes: a global information > model of stunning power and flexibility, enabling XPath, XSLT, XQuery > and what is to come. > > A poet (Joseph Brodsky) once wrote: it is not so that the poets should > adopt the language of the masses; common people should adopt the > language of literature. But common people have, to my knowledge, never done so; which is fine. And if you take a look around today, I fear you'll have to concede that there is movement, but in the opposite sense of what Brodsky deemed desirable. > In the realm of IT, XDM/XPath/XSLT/XQuery are like literature, and I > would not sell them for something else. I agree there is beauty in it: the power and flexibility of the tree navigation model. Just consider that the computer has to perform some work to set up this nice machinery; and while I find it often worth while doing so and pretty convenient for me, I have to concede that it is not always necessary. -- Michael Ludwig

size of XQuery developer community

2009-09-01T12:40:42Z

On Wed Aug 26 17:39:05 PDT 2009, Daniela Florescu wrote: > ... XQuery has tremendous potential for adding > value to customers, but the proof isn't there yet, and the path isn't > clear either. > > There has to be a larger XQuery pool of expertise ... Hi Daniela - I finally got around to reading this entire thread thru to 9/1. It's truly humbling to see how much about XQuery, XML, etc I do not know. But perhaps my newcomer status will provide a useful perspective. First some quick background. I definitely don't yet qualify as an XQuery developer. But I've been working pretty conscientiously over the past 4 or 5 months. I come to XQuery with decades of SQL, data warehouse design, and policy analysis background (ie, I see databases as a means to analysis, not an end in themselves). I got interested in XQuery because I needed a tool that would let me query the Web in a similar fashion to querying relational databases. The Web and web query tools are still primitive by comparison, but I've been quite pleased with what's possible. For example, my latest XQuery demonstration project mashed up US Dept of Education data with Federal Reserve data to examine the question of whether higher education in the US is countercyclical. See: http://garymlewis.com/instchg/2009/08/10/another-xquery-use-case-is-higher-education-countercyclical/ Given a choice, I'll almost always choose tools with power and capability versus those with ease-of-use but limited application. So, for example, I'm willing to curse and beat my head against a wall every time I use the R stat programs because I know that somehow there's a solution in R and that someone in the R community will know what the solution is. I've probably written several thousand XQuery programs now. But I only feel comfortable in a very narrow niche. There is just oodles about XML and all the other X standards and tools that I do not know. Some of this is surely just beginner's lament (ie, the notion that anything you can't learn instantly is way too complex). But some too is a steep learning curve in the absence of adequate support for learning. You ask why there is not more XQuery expertise. The XQuery community might benefit from looking at the R community [see: http://www.r-project.org/ ] and the considerable help resources available there. It's easier for a developer to get untracked with any new tool if there is visible help available. With regard to R, I'm talking here about specialized search sites, repositories of learning materials, open source and free tools, dozens upon dozens of forums, examples as code fragments, online books and manuals, and a community of very active R developers/enhancers. I like XQuery a lot. I will continue to use it. And no doubt in the course of using it, I'll fill in the many holes of what I don't know. But I could sure benefit from something like an XQuery learning resource site. Gary

size of XQuery developer community

2009-09-01T12:33:51Z

This really is the interesting question. My bet is something in search. Given that all Excel/Word/Outlook/etc. documents are or can be represented in XML, and that standard text search doesn't do a great job generally in the space (top vendors in the enterprise use facets, entity extraction, and other tricks to help the problem), XQuery has a chance to be the syntax that enables custom search solutions. Someone on this list brought up MarkMail; that's really all that is. Banging out a search solution in a standard language like XQ that's really good at dealing with semi-structured data is simply easier than using other tools. I honestly think that all the talk about XDM as a data model and XQ as a turing complete declarative language, etc., is sort of moot unless they do somehow enable something that RDBMS/BI/Data warehouses/search vendors can't readily do with existing approaches. If XQ is just an incremental improvement, it will always be a marginal language (for example, that's how I see the XQ scripting in the browser situation; it's simply not THAT much better than JavaScript, and has the disadvantage of not having a million libraries and code snippets and programmers). I actually think that it is much more than incremental in search, and it may be elsewhere as well. -Rob On Tue, Sep 1, 2009 at 4:50 AM, Andrew Welch wrote: > 2009/8/31 Daniela Florescu : > >> why do developers and architects so stubbornly refuse just to _listen_ ? > > > > Because we don't talk to them (a) enough and (b) with a message that they > > understand. > > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it? > > > > > -- > Andrew Welch > http://andrewjwelch.com > Kernow: http://kernowforsaxon.sf.net/ > _______________________________________________ > http://x-query.com/mailman/listinfo/talk > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/9f688093/attachment.htm

size of XQuery developer community

2009-09-01T12:30:54Z

> With all due respect, one major reason why XML/XQuery/XSLT isn't as popular > as many other programming languages has less to do with ignorance of it's > capabilities and more to do with it's 1) lack of performance, 2) overall > information bloat, and 3) unintuitive language constructs. Regarding performance, we're certainly not at the performance levels of relational databases, but I think for many cases we're "good enough" (even though faster is always better/cheaper). About language constructs: yes, there are some things which could have been better in my opinion, but probably everyone has its own opinion about that. My pet peeves are xmlns="" declarations in literals that change namespace scope, "return" that does not return, and the general wordiness of the language. For my taste in languages, XQuery has much too much syntax. This is one of the reasons why I wouldn't recommend it as a general purpose programming language. > Sure disk space is cheap and bandwidth is getting more *broad* every day, > and "many elegant designs have been sacrificed for the sake of > performance...yada...yada...yada...", but at the end of the day, the same > data that can be represented in XML can be represented a lot more concise in > other platform-independent formats. If you look at XML databases, you might find that they do not go around and store angle brackets. In xDB's case, what ends up on the hard drive is usually quite a bit smaller than the original XML (you know, the one with angle brackets). So if you need to store and process large amounts of XML, just pick the right tools - this is not an inherent problem with XML. What I find much more problematic is the amount of cruft XML by now has, and which greatly complicates anything built on it. We have processing instructions, DTDs, notations, entity references, internal DTD subsets, document fragments, and so on. All of these need to be supported by XML processors at least to some degree, and they greatly increase the conceptual surface of the language. I'd be willing to bet that you can find some XML construct that behaves at least unexpectedly in any larger XML processing tool, just because it's getting increasingly difficult to cover all cases. This is where simplified schemes such as JSON shine - they have a much smaller "surface". Martin

size of XQuery developer community

2009-09-01T12:21:06Z

Andrew Welch schrieb: > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it? Maybe Markmail. But being a commercial endeavour, it does not attract a so-called "community". I haven't used eXist yet, but it looks very appealing. Build a new open source bugtracker using XQuery on an app server like eXist. If it works and looks nice and shiny, it'll catch the attention of some developers. That way they will become interested in XQuery. -- Michael Ludwig

size of XQuery developer community

2009-09-01T10:52:33Z

2009/8/29 Martin Probst : > JSON [expletive deleted] for mixed content, while XML/XQuery is pretty bad at > supporting key/value maps. Both is possible in each of them, but > painful. Sequences of sequences would help here... -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/

size of XQuery developer community

2009-09-01T10:50:48Z

2009/8/31 Daniela Florescu : >> why do developers and architects so stubbornly refuse just to _listen_ ? > > Because we don't talk to them (a) enough and (b) with a message that they > understand. What is the number one "killer app" for XQuery, shining example that can be given to people when talking about XML databases - one that is high profile enough that they may have heard of it? -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/

size of XQuery developer community

2009-09-01T10:09:08Z

> it will always be a marginal language (for example, that's how I > see the XQ scripting in the browser situation; it's simply not THAT > much better than JavaScript, and has the disadvantage of not having > a million libraries and code snippets and programmers). XQuery in the broswer's advantage is not the fact that you write better code then in Javascript (I agree, good enough is good enough, and javascript is good enough). The major advantage of XQuery in the browser is the fact that you can do data and query shipping from the server to the browser and back. The fact that you don't need to write the processing twice : once in Java when you want it to run it on the App server, and once in Javascript when you want to run it in the client. Same XML data can be used on both sides, and same XQuery code can run on server and client, and you can decide what to run where LATER. That's a major problem with existing applications servers today: they cannot run in disconnected mode. You cannot take your relational data and SQL server with you, plus the app server and download them on your laptop before you take a plane. You CAN do that with XML and XQuery. Best regards Dana

size of XQuery developer community

2009-09-01T10:09:05Z

And let me add two more things that you could do with XQuery that you can hardy do otherwise -- because there are more cool :-) 6) virtualize it for the cloud execution. Scale it up and down for cloud execution. The more layers, and less declarative the layers, the harder is this to do. 7) automatically translate Xquery programs into parallel Hadoop programs. This doesn't work with imperative programs. Best regards Dana On Sep 1, 2009, at 8:55 AM, Daniela Florescu wrote: >> >> >> I honestly think that all the talk about XDM as a data model and XQ >> as a turing complete declarative language, etc., is sort of moot >> unless they do somehow enable something that RDBMS/BI/Data >> warehouses/search vendors can't readily do with existing approaches. > > Well, that's relatively simple to answer, I would say. > > Here is are some examples of what you could do with XML/XQuery that > you cannot do with the traditional three > tier stack (communication + application logic written in an > imperative language + declarative query layer) > > 1. Have schema flexibility. Start writing code without a fixed idea > of your schema, and/or change > your mind later. (one schema is evil enough, three layers of > superimposed schemas, in three different models, > are exponentially evil for flexibility and customizability. Almost > no evolution of schema is possible.) > > 2. Integrate text and documents into your information processing > flow without adding software license, yet another layer, > more mappings, and different programming languages. > > 3. Optimize the performance of your information processing program > globally, not locally, layer by layer, > by adding more and more intermediate caches and shortcuts, which in > turn will turn your code maintenance a nightmare. > > 4. Be done in 2 weeks instead of 10 months. Time to market, by > writing significantly less amount > of code, layer to layer connectors, plus data mappers from one model > to another, etc, etc. Less > useless and redundant code to write and later deal with. > > 5. Spend less time maintaining the code. If the XQuery optimizer is > doing a good job, you don't need > to constantly rewrite your code to fix performance problems, as you > do in java. Plus less number of lines of > code usually means less bugs. So smaller total cost of ownership of > the code. > > That would be my quick personal answer. > > But that of course works if you know what you are doing with > XQuery .... > > Best > Dana > > > > > > > > >> If XQ is just an incremental improvement, it will always be a >> marginal language (for example, that's how I see the XQ scripting >> in the browser situation; it's simply not THAT much better than >> JavaScript, and has the disadvantage of not having a million >> libraries and code snippets and programmers). I actually think >> that it is much more than incremental in search, and it may be >> elsewhere as well. >> >> -Rob >> >> >> On Tue, Sep 1, 2009 at 4:50 AM, Andrew Welch >> > wrote: >> 2009/8/31 Daniela Florescu : >> >> why do developers and architects so stubbornly refuse just to >> _listen_ ? >> > >> > Because we don't talk to them (a) enough and (b) with a message >> that they >> > understand. >> >> What is the number one "killer app" for XQuery, shining example that >> can be given to people when talking about XML databases - one that is >> high profile enough that they may have heard of it? >> >> >> >> >> -- >> Andrew Welch >> http://andrewjwelch.com >> Kernow: http://kernowforsaxon.sf.net/ >> _______________________________________________ >> http://x-query.com/mailman/listinfo/talk >> http://x-query.com/mailman/listinfo/talk >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/d6058752/attachment.htm

size of XQuery developer community

2009-09-01T09:55:54Z

> > > I honestly think that all the talk about XDM as a data model and XQ > as a turing complete declarative language, etc., is sort of moot > unless they do somehow enable something that RDBMS/BI/Data > warehouses/search vendors can't readily do with existing approaches. Well, that's relatively simple to answer, I would say. Here is are some examples of what you could do with XML/XQuery that you cannot do with the traditional three tier stack (communication + application logic written in an imperative language + declarative query layer) 1. Have schema flexibility. Start writing code without a fixed idea of your schema, and/or change your mind later. (one schema is evil enough, three layers of superimposed schemas, in three different models, are exponentially evil for flexibility and customizability. Almost no evolution of schema is possible.) 2. Integrate text and documents into your information processing flow without adding software license, yet another layer, more mappings, and different programming languages. 3. Optimize the performance of your information processing program globally, not locally, layer by layer, by adding more and more intermediate caches and shortcuts, which in turn will turn your code maintenance a nightmare. 4. Be done in 2 weeks instead of 10 months. Time to market, by writing significantly less amount of code, layer to layer connectors, plus data mappers from one model to another, etc, etc. Less useless and redundant code to write and later deal with. 5. Spend less time maintaining the code. If the XQuery optimizer is doing a good job, you don't need to constantly rewrite your code to fix performance problems, as you do in java. Plus less number of lines of code usually means less bugs. So smaller total cost of ownership of the code. That would be my quick personal answer. But that of course works if you know what you are doing with XQuery .... Best Dana > If XQ is just an incremental improvement, it will always be a > marginal language (for example, that's how I see the XQ scripting in > the browser situation; it's simply not THAT much better than > JavaScript, and has the disadvantage of not having a million > libraries and code snippets and programmers). I actually think that > it is much more than incremental in search, and it may be elsewhere > as well. > > -Rob > > > On Tue, Sep 1, 2009 at 4:50 AM, Andrew Welch > wrote: > 2009/8/31 Daniela Florescu : > >> why do developers and architects so stubbornly refuse just to > _listen_ ? > > > > Because we don't talk to them (a) enough and (b) with a message > that they > > understand. > > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it? > > > > > -- > Andrew Welch > http://andrewjwelch.com > Kernow: http://kernowforsaxon.sf.net/ > _______________________________________________ > http://x-query.com/mailman/listinfo/talk > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/8519c99c/attachment.htm

Tuple types

2009-09-01T09:21:28Z

On Tue, Sep 1, 2009 at 6:44 AM, John Snelson wrote: > Tuple types won't make it into XQuery 1.1, but the closures support in > higher order functions allows you to effectively return sequences of > sequences: > > (: Wrap a sequence as a function item :) > declare function local:ref($arg as item()*) as function() as item()* > { > function() { $arg } > }; > > (: Unwrap a sequence from a function item :) > declare function local:deref($arg as function() as item()*) as item()* > { > $arg() > }; This makes sense, though it would be a nuisance to have to define ref() and deref() every time. Is there any consideration to make them standard library functions?

size of XQuery developer community

2009-08-31T20:55:03Z

Daniela Florescu schrieb: >> a widespread lack of a _deep_ understanding of XML. > > Hans-Juergen, > > Yes, you are right. There is a deep misunderstanding in IT of our > core message about the role of XML in the new architectures. What is that core message? (If it is not what I'm going to write below.) > XML is not syntax, it can/should be the primary way of modeling data. To XSLT and XQuery, XML is the XPath Data Model, which - I think - could exist quite merrily without a serialization format. To the Programming World, XML is the angle-bracket serialization format. I've heard XML experts say that the Programming World does not like angle brackets. (I don't know if this is true, but I've seen a lot of editors that hide the angle brackets.) I think that some people do not like the verboseness of the XML serialization format. XML, which isn't SGML, cannot change its serialization format without major disruptions. The XPath Data Model seems to have an alternative serialization format in XQuery: element bla { attribute blub { ... } }. But instead of mucking about with the serialization format, wouldn't it be more clever to market XQuery and XSLT as Tree Processing Languages? (Being very careful, bien entendu, not to come dangerously close to deforestation operations in order not to scare away the environmentalists.) Trees enjoy a pretty good reputation. Much better than angle brackets! > XQuery is not a query language (oh well), but a full blown > (declarative) data processing language. Well, but it typically lives where there is XML data, and it is typically used to query that data. It looks very well integrated to me. Isn't this a big plus? -- Michael Ludwig

size of XQuery developer community

2009-08-31T19:21:35Z

With all due respect, one major reason why XML/XQuery/XSLT isn't as popular as many other programming languages has less to do with ignorance of it's capabilities and more to do with it's 1) lack of performance, 2) overall information bloat, and 3) unintuitive language constructs. Sure disk space is cheap and bandwidth is getting more *broad* every day, and "many elegant designs have been sacrificed for the sake of performance...yada...yada...yada...", but at the end of the day, the same data that can be represented in XML can be represented a lot more concise in other platform-independent formats. I won't even get into the whole information density argument which can be Googled. It's been my experience (19+ years of professional software development) that there is not a one-size-fits-all solution. i.e. my tool belt has more than just a hammer. ;-) On Mon, Aug 31, 2009 at 5:37 PM, Hans-Juergen Rennau wrote: > Hello Michael, > > of course you are right in that XSLT and XQuery are built on the XDM, not > on XML as a serialization format, and you are right in that the XDM can be > completely separated from the serialization format. But I prefer to view XML > and the XDM as one organic whole, and exactly this is what I meant by the > term "deep understanding": an understanding covering three _levels_ of XML: > level #1: the lexical; level #2: the inforset (= structural model totally > unconcerned with aspects of processing); level #3: the XDM extending the > infoset in order to embrace the reality of processing = a global information > model which preserves the structural achievements of the infoset and embeds > it into a stunningly flexible definition of information value. Each level > evolved from the previous one, I think, was it not so, historically? > > This evolution is a wonderful achievement, and I am emphatically against > "marketing" XQuery and XSLT as tree processing languages, as if you could > come along with some tree and get it processed. What should be done is > rather: encourage people to get familiar with the new reality which XML (via > XDM) constitutes: a global information model of stunning power and > flexibility, enabling XPath, XSLT, XQuery and what is to come. > > A poet (Joseph Brodsky) once wrote: it is not so that the poets should > adopt the language of the masses; common people should adopt the language of > literature. In the realm of IT, XDM/XPath/XSLT/XQuery are like literature, > and I would not sell them for something else. > -- A. Steven Anderson Independent Consultant -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090831/23f3f12b/attachment.htm

size of XQuery developer community

2009-08-31T17:32:46Z

> I remember years ago, when I was really frustrated that architects > would not _listen_ that Xquery > was this wonderful and useful thing, I asked advice from Jim Gray. > He was always ready to listen and help, > and, even though he was too much of a believer in SQL and schemas to > start embracing XQuery, he gave me > the following advice. > > If you want people to listen then (a) write articles and give talks, > (b) write software that works, > (c) make interesting demos. Then repeat, and don't loose patience. > > I think that at this point we have around XQuery lots of software > that does work: databases, in memory > processors, etc. > > What we can do is write more articles, give more talks (Web 2.0, > eGov 2.0 anyone ?), and make more > interesting demos. > > About demos: I am still surprised that nobody is trying XQuery's > mashup strength on the examples > on http://www.programmableweb.com/. Wonderful site, BTW. I just don't think a language that feels it is acceptable for more than a few typeswitch cases is worthwhile (OK, I have my helmet on). XQuery has the simplicity to be able to run on an XML DB. That is it's main selling point. XSL can do much better if it could run on an XML DB. As for XQuery as a web application or scripting language, I think there are better. > > Or that noone is using XQuery on the http://www.data.gov/ site and > win the price of the best eGovernment application.... Or, why isn't the data transparent so we can see what we pay for?

AW: size of XQuery developer community

2009-08-31T16:52:55Z

Yes, of course there are domains where XML cannot be afforded due to performance and data volume issues. I suggest that we take this for granted, I assure you there is no "one-size-fits-all" ambition - let us now consider the domains where those issues do not play a primary role. Your hints interest me very much: you speak of other, more concise and platform-independent formats. So they may be regarded as siblings of XML as a serialization format - siblings of level #1 XML, so to speak. Please tell me - are they also embedded into an information model which might be compared with level #3 XML - the XDM - supporting hierarchy, order and processing? An information model that does, or in principle could, support a pure expression language like XQuery? With kind regards, Hans-Juergen Rennau ________________________________ Von: A. Steven Anderson An: Hans-Juergen Rennau CC: http://x-query.com/mailman/listinfo/talk; http://x-query.com/mailman/listinfo/talk; http://x-query.com/mailman/listinfo/talk Gesendet: Montag, den 31. August 2009, 15:21:35 Uhr Betreff: Re: size of XQuery developer community With all due respect, one major reason why XML/XQuery/XSLT isn't as popular as many other programming languages has less to do with ignorance of it's capabilities and more to do with it's 1) lack of performance, 2) overall information bloat, and 3) unintuitive language constructs. Sure disk space is cheap and bandwidth is getting more *broad* every day, and "many elegant designs have been sacrificed for the sake of performance...yada...yada...yada...", but at the end of the day, the same data that can be represented in XML can be represented a lot more concise in other platform-independent formats. I won't even get into the whole information density argument which can be Googled. It's been my experience (19+ years of professional software development) that there is not a one-size-fits-all solution. i.e. my tool belt has more than just a hammer. ;-) -- A. Steven Anderson Independent Consultant -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090831/6ea3f8a3/attachment-0001.htm