Andrew Welch wrote:
> I mentioned the streaming aspect for 2 reasons:
>
> 1) If validation performance is an issue, can RNG + Schematron still
> be considered when XSD is validation so fast?
>   
I think Andrew is identifying one of the basic flaws of XSD evolution:  
an optimization controls the design.  

But I don't believe the claim that XSD validation is necessarily faster 
than RNG + Schematron, even at the poor state of optimization we have at 
the moment.

1) There are many kinds of constraints that Schematron can do that XSD 
1.1 assertions etc cannot do. XSD cannot be considered faster at things 
that are completely out of its reach!  On these, I can state that RELAX 
NG and Schematron are absolutely faster than XSD  :-) 

2) An XSD assertion requires that a local XDM trimmed branch be 
constructed. So the worst case for XSD is where all elements have an 
assertion: the same amount of tree building would have to go on as for 
building a full XDM tree for the typical non-streaming implementation of 
Schematron. There would be a space saving, but not necessarily a speed 
saving.  (In fact, I think only the non leaf elements would need 
assertions to get to this same point.)

3) There are Schematron implementations with a terminate on fail 
construction. (Ken Holman contributed this IIRC.) So where pass/fail 
testing is required,  these can be very fast in the amount of work they 
need to do. Combine them with a streaming implementation or even a 
lazily constructed DOM, and they certainly could be faster than an XSD 
implementation that attempts to run over a whole large file.

3a)  A RELAX NG implementation that just provides a validation result 
does much less work than an XSD implementation that produces a fill 
PSVI, too.

4) XSD implementations are not necessarily streaming, but may be random 
access. For example, my implementation of XSD by converting it to 
Schematron would use whatever the Schematron implementation used. Or a 
validator that ran over data in a database directly without pickling it 
first.

5) Where the application that uses the XML requires a tree, the tree 
needs to be built even if you have streaming validation: so you aren't 
actually saving any tree construction time or space. In fact, since the 
PSVI has no standard XML form or standard streaming API form, I actually 
imagine that most uses of XSD actually result in a tree being built (or 
the data being entered into a DBMS): the point of the PSVI is to make 
extra information available for systems which are typically random 
access, keyed access or object trees (anything except streaming!)   

6) Where a document is not large, it is not certain that a streaming 
implementation of a validator using a modern language with automatic 
garbage-collected will actually allocate or use fewer objects compared 
to a tree-building implementation.  And object-allocation-avoidance 
strategies such as a cross-thread pool of DOM objects can also benefit 
in-memory implementations just as much as streaming implementations. The 
size of documents limits the number of simultaneous process more in the 
case of the tree-building implementation, but not necessarily the number 
of objects allocated.  (In fact, if the system is a validator, it may be 
that the event stream may need to be queued untill validation has 
finished before passing it on to the application: this will limit 
opportunities from speed-ups due to less object allocation from pooling 
or singleton strategies, for example.)

The exception might be XSD validation used for firewalls. But when you 
look at, for example, the Lloyds London Market system, they validate 
incoming data using XSD for coarse-grain validation, then Schematron for 
fine-grain validation: non-streaming is not a bar for their documents.

7)  Where there is a resource constraint like a real-time constraint,  
benchmarking is ultimately the most objective way of determining 
performance. Whitebox knowledge of algorithms and implementation details 
may certainly give hints about behaviour, but they are just armchair 
hints that may vary with different implementations, schemas and input 
documents: an algorithm that is efficient but explosive may give better 
performance than an algorithm that is slow but inefficient. 

(For example, a system that takes  10+n^2  has better performance than a 
system that is 110 + 10n for n <11.  We know about XSLT engines that the 
slowest is at least 24 times slower than the fastest even for basic 
transformations, for example, so the constants could plausibly swamp the 
exponents. ) 

8) XSD schemas can be very large and verbose, with multiple files, and 
many internal checks of the schema components such as derivation by 
restriction and UPA. I see no reason to expect that  loading a large XSD 
schema with all that extra work would necessarily be more efficient than 
the effort in loading a RNC or Schematron schema.  Indeed, in XSD is is 
quite common that the schema is larger than the instance: even when 
there is a streaming validation, most of the process is taken up with 
creating persistent objects for the schema.

Putting all these together, I certainly concede that if you have a large 
document, a small memory, a compiled and pre-loaded schema, a small 
schema with only a few assertions, constraints that are only local, 
where the document is thrown away after validation and the PSVI or tree 
or stream not passed on, the document is parsed from XML rather than 
coming in as a DOM,  and you want the validation to get as much outcome 
as possible,  then you might reasonably suspect that a streaming 
implementation (whether XSD or RNG + Schematron) would be faster to go 
through an entire document to confirm that no errors exist than an 
in-memory implementation made with the same attention to memory issues, 
in the absence of benchmarking. 

Added to that, I think there is tremendous scope for optimization of 
Xpaths and XSLT, and consequently Schematron. Michael Kay's optimization 
work in XSLT and XPath is interesting. There are a lot of fun 
possibilities for Schematron-specific optimization based on getting fast 
results (e.g. http://www.topologi.com/public/SchematronHeuristic.pdf) or 
optimization on tries and feature sets 
(http://broadcast.oreilly.com/2009/06/validation-using-tries-and-fea.html)
 

> 2) Isn't it the case that some of the complexities of XSD are that way
> to allow for that validation speed?
>   
Do you have an example?  (I imagine it causes some simplifications as 
well as some complexities.)

Cheers
Rick Jelliffe

Glidden, Douglass A wrote:
> Phillipe,
> 
> That sounds pretty slick--definitely sounds like it goes even above and beyond what I've been looking for in many ways.  I noticed though that you didn't comment on my complaints about typing--does ASL support the concept of complex data typing with type derivation, etc.?  If so, it definitely sounds promising.

Well, complex data typing with type derivation is rather a concept 
tightly close to W3C XML Schema; the design approch is somewhat 
different in ASL, it is rather based on schema composition optionally 
with the help of active catalogs as the glue.

IMHO, I'm not sure that this OO paradigm can be applied in XML: a class 
is defined by its unordered members that a subclass can simply override, 
whereas the sequential nature of XML doesn't allow to override only a 
given part of a content model. Contorsionists at W3C XML Schema's group 
make things hard to achieve at the cost of an extreme verbosity and 
extreme complexity that removes the benefits of this OO feature.

With ASL, I didn't try to adopt at any cost a feature that doesn't fit 
well in XML technologies, I'd rather tried to think different. Moving 
from a static abstract tree to a dynamic one that can adapt itself 
according to the incoming data to validate is one of the best feature of 
ASL. Bringing semantic capabilities that are not supported by other 
schema technologies is another one.

Kind regards.

> 
> By the way, I love your signature--gave me a great laugh this morning! :-)

:)

> 
> Doug Glidden
> Software Engineer
> The Boeing Company
> Douglass.A.Glidden@boeing.com
> 
> -----Original Message-----
> From: Philippe Poulard [mailto:philippe.poulard@sophia.inria.fr]
> Sent: Thursday, November 19, 2009 05:42
> To: Glidden, Douglass A
> Cc: xml-dev@lists.xml.org
> Subject: Re:  XML spec and XSD
> 
> Hi,
> 
> I have presented the "Active Schema Language" at Balisage last year, that covers rather well most of the features you expect:
> http://www.balisage.net/Proceedings/vol1/html/Poulard01/BalisageVol1-Poulard01.html
> Slides:
> http://hal.inria.fr/docs/00/32/26/61/ANNEX/Bal2008poul061003.pdf
> Implementation:
> http://reflex.gforge.inria.fr/
> 
> Glidden, Douglass A wrote:
>> From the perspective of an application developer who uses XML for storing fairly complex hierarchical data (as opposed to documents), I have a very basic question:  Is there any real, practical reason that Relax NG is _universally_ better than XSD?
>>
>> Here's the background for my question:  I've been using XSDs for generating object-oriented data models for some time, and have been frustrated by various inflexibilities in XSDs.  For instance:
>> - Derivation by restriction, while conceptually very useful, seems slightly cumbersome in operation; in particular, I wish it could be accomplished with less repetition (I'm a fanatical believer in DRY).  I can live with the repetition, though, because I am assured that, if I make a typographical or some other error, the XSD will (almost) always fail to validate ("Bar is not a valid restriction of Foo"), so the the possiblity of unintended changes being introduced due to the repetition is kept to a minimum.
>> - Certain fairly simple data structures can only be achieved in XSD (if at all) with ridiculously verbose patterns; for instance, how do you represent a structure with five possible elements, at least one of which must be present and none of which may be repeated?  The best method I have found takes about 25 lines and far too much code repetition.
> 
> With ASL, you just express things in the way you think of them:
>      <asl:element name="property">
>          <asl:select max-occurs="unbounded">
>              <asl:element ref-elem="width" min-occurs="0"/>
>              <asl:element ref-elem="color" min-occurs="0"/>
>              <asl:element ref-elem="border" min-occurs="0"/>
>          </asl:select>
>      </asl:element>
> by default, cardinalities are set to 1, which means that <property> must have at least 1 occurrence, and within, the other elements can't appear more than once
> 
> <property><!--valid-->
>         <width/>
> </property>
> <property><!--valid-->
>         <width/>
>         <border/>
> </property>
> <property>
>         <width/>
>         <width/><!--unexpected element-->
> </property>
> <property>
>         <width/>
>         <border/>
>         <width/><!--unexpected element-->
> </property>
> <property/><!--element missing-->
> 
>> - Most business logic cannot be represented, like "the value of the foo element must be less than or equal to the value of the bar element".
> 
> You will find here some usefull use cases of business logic representation:
> http://reflex.gforge.inria.fr/tutorial-schemas.html
> 
>> - I occasionally was foiled by XSDs strict "no ambiguity" rules, which make it difficult to enforce rules such as "if the value of the foo element is xyz, then the bar and baz elements are required, but otherwise they are optional".
> 
> Straightforward in ASL:
> 
> <asl:element name="xxxxxx">
>    <xcl:if test="{/path/to/foo = 'xyz'}">
>      <xcl:then>
>        <asl:sequence>
>          <asl:element ref-elem="bar"/>
>          <asl:element ref-elem="baz"/>
>        </asl:sequence>
>      </xcl:then>
>      <xcl:else>
>        <asl:sequence>
>          <asl:element ref-elem="bar" min-occurs="0"/>
>          <asl:element ref-elem="baz" min-occurs="0"/>
>        </asl:sequence>
>      </xcl:else>
>    </xcl:if>
> </asl:element>
> 
> you will notice that ASL can be mixed with other XML libraries (above, XCL that stands for the XML Control Language); this is a very useful feature that makes the abstract tree of the schema dynamic (other schema technologies have a static abstract tree) which boost the expressiveness of the language (isn't that a features that you are looking for ?)
> 
>> Nonetheless, I have been more or less pleased with the final results in most cases.  Having heard much praise of Relax NG, I recently decided to try it out, hoping it would be able to resolve these issues.  Here's what I found:
>> - It does relieve the "no ambiguity" problem and makes enforcing that sort of rules much easier.
>> - It does not resolve most of the business logic issues, but there are other technologies that can be used in combination with either Relax NG or XSD to resolve these issues, so that can be worked around in either case.
>> - It does not appear to relieve the complexity of representing some types of data structures (see the one described above-as far as I can tell, it would not be significantly easier to define in Relax NG than it is in XSD).
>> - There is only a rudimentary concept of type definition and virually
>> none of inheritance.  Designing good, object-oriented data models is
>> practically impossible in Relax NG; for me, this is a HUGE deficiency.
>> For instance, say Foo is an abstract data type with several fields,
>> including an id field (any mixed data allowed) and a type field (with
>> enumerated values); Bar is one of several concrete subtypes of Foo,
>> which restricts the content of the id field (must contain two
>> elements, a groupId and a subId) and the type field (to a subset of
>> the values enumerated by Foo) and appends several new fields; other
>> subtypes have completely different restrictions on the id and type
>> fields and append various different new fields.  In XSD, implementing
>> this is fairly straightforward (although not necessarily simple and
>> certainly more verbose than I would prefer):  Foo is an abstract
>> complex type, and Bar and the other subtypes must derive from it first
>> via restriction (creating the re
> strictions on id and type) and then via extension (appending the new fields).  In Relax NG, by contrast, I could find only two ways of doing this, neither of which was at all acceptable:  One is to get rid of Foo completely and reference its fields separately in each subtype-this is unacceptable to me as it completely violates DRY and object-oriented design; the other is to define each subtype of Foo in a separate file, importing Foo and redefining as necessary-this is marginally better from an OOP point-of-view, but still makes it impossible for a schema to use "any valid subtype of Foo" as a content definition without violating the Open-Closed principle.
>> - Relax NG provides no way of specifying cardinalities other than 0, 1, and infinity!  This is also a big one for me--if I need at least 2 and no more than 20 occurrences of the foo element (fairly common in my data definitions), there's NO WAY I'm going to define two required foo elements and eighteen optional ones!
> 
> In ASL, cardinalities support "2" or "20", but can also be dynamic "{1+1}" or "{count(../column[1]/cell)}" or whatever you find useful with XPath
> 
>> Now, I'm hoping that some Relax NG expert will come along and prove me wrong about some or all of these things, since I only experimented with it for a couple of days.  If not, for me these deficiencies in Relax NG *far* outweigh its benefits over XSD-there's really no comparison.  As much as I would like to see some aspects of XSD improved, unless someone can propose real alternatives for issues like the ones above, Relax NG is simply not an option for the type of data I work with.
>>
>> As a postscript, let me say I'm very thankful that the tone of this
>> thread has been gradually transitioning to actual constructive
>> argument (at which this list is normally rather good) rather than the
>> ridiculously juvenile mud-slinging that was predominant early last
>> week-in particular, I've appreciated the level-headed input from Jim
>> Tivy, Liam Quin, Ken Holman, and Michael Kay.  Unfortunately there are
>> still some emotionally charged but pretty much meaningless (or worse,
>> completely false) words being thrown around, but at least those aren't
>> in the majority anymore.  In all good humor, I have to say that
>> XML-dev is by far the most argumentative mailing list to which I've
>> ever been a subscriber. :-)
>>
>> Doug Glidden
>> Software Engineer
>> The Boeing Company
>> Douglass.A.Glidden@boeing.com
>> ______________________________________________________________________
>> _
>>
>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS to
>> support XML implementation and development. To minimize spam in the
>> archives, you must subscribe before posting.
>>
>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>> subscribe: xml-dev-subscribe@lists.xml.org List archive:
>> http://lists.xml.org/archives/xml-dev/
>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>
> 
> 
> --
> Cordialement,
> 
>                ///
>               (. .)
>   --------ooO--(_)--Ooo--------
> |      Philippe Poulard       |
>   -----------------------------
>   http://reflex.gforge.inria.fr/
>         Have the RefleX !
> 
> _______________________________________________________________________
> 
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
> 
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> 


-- 
Cordialement,

               ///
              (. .)
  --------ooO--(_)--Ooo--------
|      Philippe Poulard       |
  -----------------------------
  http://reflex.gforge.inria.fr/
        Have the RefleX !

An RNC version would be especially nice thing to see.
Leigh. 

-----Original Message-----
From: Peter A. Bigot [mailto:bigotp@acm.org] 
... PyXB is only appropriate where an XML schema is already available or useful for other reasons...

Hi,

I have presented the "Active Schema Language" at Balisage last year, 
that covers rather well most of the features you expect:
http://www.balisage.net/Proceedings/vol1/html/Poulard01/BalisageVol1-Poulard01.html
Slides:
http://hal.inria.fr/docs/00/32/26/61/ANNEX/Bal2008poul061003.pdf
Implementation:
http://reflex.gforge.inria.fr/

Glidden, Douglass A wrote:
> From the perspective of an application developer who uses XML for storing fairly complex hierarchical data (as opposed to documents), I have a very basic question:  Is there any real, practical reason that Relax NG is _universally_ better than XSD?
> 
> Here's the background for my question:  I've been using XSDs for generating object-oriented data models for some time, and have been frustrated by various inflexibilities in XSDs.  For instance:
> - Derivation by restriction, while conceptually very useful, seems slightly cumbersome in operation; in particular, I wish it could be accomplished with less repetition (I'm a fanatical believer in DRY).  I can live with the repetition, though, because I am assured that, if I make a typographical or some other error, the XSD will (almost) always fail to validate ("Bar is not a valid restriction of Foo"), so the the possiblity of unintended changes being introduced due to the repetition is kept to a minimum.
> - Certain fairly simple data structures can only be achieved in XSD (if at all) with ridiculously verbose patterns; for instance, how do you represent a structure with five possible elements, at least one of which must be present and none of which may be repeated?  The best method I have found takes about 25 lines and far too much code repetition.

With ASL, you just express things in the way you think of them:
     <asl:element name="property">
         <asl:select max-occurs="unbounded">
             <asl:element ref-elem="width" min-occurs="0"/>
             <asl:element ref-elem="color" min-occurs="0"/>
             <asl:element ref-elem="border" min-occurs="0"/>
         </asl:select>
     </asl:element>
by default, cardinalities are set to 1, which means that <property> must 
have at least 1 occurrence, and within, the other elements can't appear 
more than once

<property><!--valid-->
	<width/>
</property>
<property><!--valid-->
	<width/>
	<border/>
</property>
<property>
	<width/>
	<width/><!--unexpected element-->
</property>
<property>
	<width/>
	<border/>
	<width/><!--unexpected element-->
</property>
<property/><!--element missing-->

> - Most business logic cannot be represented, like "the value of the foo element must be less than or equal to the value of the bar element".

You will find here some usefull use cases of business logic representation:
http://reflex.gforge.inria.fr/tutorial-schemas.html

> - I occasionally was foiled by XSDs strict "no ambiguity" rules, which make it difficult to enforce rules such as "if the value of the foo element is xyz, then the bar and baz elements are required, but otherwise they are optional".

Straightforward in ASL:

<asl:element name="xxxxxx">
   <xcl:if test="{/path/to/foo = 'xyz'}">
     <xcl:then>
       <asl:sequence>
         <asl:element ref-elem="bar"/>
         <asl:element ref-elem="baz"/>
       </asl:sequence>
     </xcl:then>
     <xcl:else>
       <asl:sequence>
         <asl:element ref-elem="bar" min-occurs="0"/>
         <asl:element ref-elem="baz" min-occurs="0"/>
       </asl:sequence>
     </xcl:else>
   </xcl:if>
</asl:element>

you will notice that ASL can be mixed with other XML libraries (above, 
XCL that stands for the XML Control Language); this is a very useful 
feature that makes the abstract tree of the schema dynamic (other schema 
technologies have a static abstract tree) which boost the expressiveness 
of the language (isn't that a features that you are looking for ?)

> 
> Nonetheless, I have been more or less pleased with the final results in most cases.  Having heard much praise of Relax NG, I recently decided to try it out, hoping it would be able to resolve these issues.  Here's what I found:
> - It does relieve the "no ambiguity" problem and makes enforcing that sort of rules much easier.
> - It does not resolve most of the business logic issues, but there are other technologies that can be used in combination with either Relax NG or XSD to resolve these issues, so that can be worked around in either case.
> - It does not appear to relieve the complexity of representing some types of data structures (see the one described above-as far as I can tell, it would not be significantly easier to define in Relax NG than it is in XSD).
> - There is only a rudimentary concept of type definition and virually none of inheritance.  Designing good, object-oriented data models is practically impossible in Relax NG; for me, this is a HUGE deficiency.  For instance, say Foo is an abstract data type with several fields, including an id field (any mixed data allowed) and a type field (with enumerated values); Bar is one of several concrete subtypes of Foo, which restricts the content of the id field (must contain two elements, a groupId and a subId) and the type field (to a subset of the values enumerated by Foo) and appends several new fields; other subtypes have completely different restrictions on the id and type fields and append various different new fields.  In XSD, implementing this is fairly straightforward (although not necessarily simple and certainly more verbose than I would prefer):  Foo is an abstract complex type, and Bar and the other subtypes must derive from it first via restriction (creating the re
strictions on id and type) and then via extension (appending the new fields).  In Relax NG, by contrast, I could find only two ways of doing this, neither of which was at all acceptable:  One is to get rid of Foo completely and reference its fields separately in each subtype-this is unacceptable to me as it completely violates DRY and object-oriented design; the other is to define each subtype of Foo in a separate file, importing Foo and redefining as necessary-this is marginally better from an OOP point-of-view, but still makes it impossible for a schema to use "any valid subtype of Foo" as a content definition without violating the Open-Closed principle.
> - Relax NG provides no way of specifying cardinalities other than 0, 1, and infinity!  This is also a big one for me--if I need at least 2 and no more than 20 occurrences of the foo element (fairly common in my data definitions), there's NO WAY I'm going to define two required foo elements and eighteen optional ones!

In ASL, cardinalities support "2" or "20", but can also be dynamic 
"{1+1}" or "{count(../column[1]/cell)}" or whatever you find useful with 
XPath

> 
> Now, I'm hoping that some Relax NG expert will come along and prove me wrong about some or all of these things, since I only experimented with it for a couple of days.  If not, for me these deficiencies in Relax NG *far* outweigh its benefits over XSD-there's really no comparison.  As much as I would like to see some aspects of XSD improved, unless someone can propose real alternatives for issues like the ones above, Relax NG is simply not an option for the type of data I work with.
> 
> As a postscript, let me say I'm very thankful that the tone of this thread has been gradually transitioning to actual constructive argument (at which this list is normally rather good) rather than the ridiculously juvenile mud-slinging that was predominant early last week-in particular, I've appreciated the level-headed input from Jim Tivy, Liam Quin, Ken Holman, and Michael Kay.  Unfortunately there are still some emotionally charged but pretty much meaningless (or worse, completely false) words being thrown around, but at least those aren't in the majority anymore.  In all good humor, I have to say that XML-dev is by far the most argumentative mailing list to which I've ever been a subscriber. :-)
> 
> Doug Glidden
> Software Engineer
> The Boeing Company
> Douglass.A.Glidden@boeing.com
> _______________________________________________________________________
> 
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
> 
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> 


-- 
Cordialement,

               ///
              (. .)
  --------ooO--(_)--Ooo--------
|      Philippe Poulard       |
  -----------------------------
  http://reflex.gforge.inria.fr/
        Have the RefleX !

Please ignore the following if you have no interest in XML-related training.

In Q1'2010 I will be in Europe a couple of times for some standards 
meetings (OASIS and ISO) and conferences ( http://www.XMLPrague.cz 
and PEPPOL) and I like to take advantage of my presence there by 
offering Crane's hands-on training classes in the week before or 
after each of the events, open to public subscription.

I'll be in Copenhagen in February and in Prague and Stockholm in March.

But it is a gamble to decide on which training class to offer and 
find no-one is interested in that particular topic, because of hotel 
and event obligations that need to be paid in advance or reserved 
with the risk of forfeit.  We realize not many people in these 
economic times have money for training, so we are now trying to 
target our offerings specifically where we learn there is demand and 
availability of students.

Last year's http://www.XMLPrague.cz conference was great and we did 
our training adjacent to the conference and our students were 
conference attendees.  The venue we have chosen again in Prague next 
year has room for only 6 students (a large 14th century hotel room), 
so if XML training at this time of year and in conjunction with the 
conference is of interest to you, it would be important to sign up 
your interest soon.

To address this urgency to start making plans, I am polling the 
various XML lists for any interest in training in these cities early 
next year.  If you are interested in hands-on training on any of 
XSLT, XQuery, XSL-FO, the Universal Business Language or code lists 
in XML, and you would be interested in attending, please let us know 
and we will figure out what we can do.  To see the proposed dates, 
please visit:

   http://www.CraneSoftwrights.com/forms/interest.php
   (the page has links to syllabi)

On the form, please indicate all of the offered dates that you would 
be interested in, and we'll make a decision based on the response 
level we get for each class and date.

Thanks!  Please direct any questions to me off-list.

. . . . . . . . . Ken

p.s. it hasn't been decided but I might be in Perth Australia in 
January or February, so you won't see it on the form, but if you are 
in that area, please let me know in a mail message if you are 
interested in training on any of our subjects and we'll see about 
setting up a publicly-subscribed class.

cc: XML-Dev, XSL List, XQuery Talk, XML-Doc, UBL Dev, CLR Dev, 
Antenna House list, RenderX list, MarkLogic list

--
Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

> > 2) Isn't it the case that some of the complexities of XSD 
> are that way 
> > to allow for that validation speed?
> >   
> Do you have an example?  (I imagine it causes some 
> simplifications as well as some complexities.)
> 

Some of the restrictions are certainly there for that reason, like the
subset of XPath permitted in identity constraints, the ban on backtracking
(UPA etc), and the rules in conditional type assignment that restrict you to
looking at attributes of the current element.

I'm always suspicious of such rules: on the whole I think it's better to let
users decide where to make the trade-off between performance and
functionality; and I think writers of specifications are sometimes
unimaginative when they consider implementation options. There's a classic
in XSD, the rules for restriction of xs:all content models, where the spec
explicitly says that a rule is there only for implementation convenience,
and it actually causes my implementation a great deal of inconvenience to
enforce it.

As for streaming, I think we'll find in the fullness of time that a
reasonably large subset of the assertions that people write (either in XSD
1.1 or in Schematron) can actually be evaluated in streaming mode.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

Phillipe,

That sounds pretty slick--definitely sounds like it goes even above and beyond what I've been looking for in many ways.  I noticed though that you didn't comment on my complaints about typing--does ASL support the concept of complex data typing with type derivation, etc.?  If so, it definitely sounds promising.

By the way, I love your signature--gave me a great laugh this morning! :-)

Doug Glidden
Software Engineer
The Boeing Company
Douglass.A.Glidden@boeing.com

-----Original Message-----
From: Philippe Poulard [mailto:philippe.poulard@sophia.inria.fr]
Sent: Thursday, November 19, 2009 05:42
To: Glidden, Douglass A
Cc: xml-dev@lists.xml.org
Subject: Re:  XML spec and XSD

Hi,

I have presented the "Active Schema Language" at Balisage last year, that covers rather well most of the features you expect:
http://www.balisage.net/Proceedings/vol1/html/Poulard01/BalisageVol1-Poulard01.html
Slides:
http://hal.inria.fr/docs/00/32/26/61/ANNEX/Bal2008poul061003.pdf
Implementation:
http://reflex.gforge.inria.fr/

Glidden, Douglass A wrote:
> From the perspective of an application developer who uses XML for storing fairly complex hierarchical data (as opposed to documents), I have a very basic question:  Is there any real, practical reason that Relax NG is _universally_ better than XSD?
>
> Here's the background for my question:  I've been using XSDs for generating object-oriented data models for some time, and have been frustrated by various inflexibilities in XSDs.  For instance:
> - Derivation by restriction, while conceptually very useful, seems slightly cumbersome in operation; in particular, I wish it could be accomplished with less repetition (I'm a fanatical believer in DRY).  I can live with the repetition, though, because I am assured that, if I make a typographical or some other error, the XSD will (almost) always fail to validate ("Bar is not a valid restriction of Foo"), so the the possiblity of unintended changes being introduced due to the repetition is kept to a minimum.
> - Certain fairly simple data structures can only be achieved in XSD (if at all) with ridiculously verbose patterns; for instance, how do you represent a structure with five possible elements, at least one of which must be present and none of which may be repeated?  The best method I have found takes about 25 lines and far too much code repetition.

With ASL, you just express things in the way you think of them:
     <asl:element name="property">
         <asl:select max-occurs="unbounded">
             <asl:element ref-elem="width" min-occurs="0"/>
             <asl:element ref-elem="color" min-occurs="0"/>
             <asl:element ref-elem="border" min-occurs="0"/>
         </asl:select>
     </asl:element>
by default, cardinalities are set to 1, which means that <property> must have at least 1 occurrence, and within, the other elements can't appear more than once

<property><!--valid-->
        <width/>
</property>
<property><!--valid-->
        <width/>
        <border/>
</property>
<property>
        <width/>
        <width/><!--unexpected element-->
</property>
<property>
        <width/>
        <border/>
        <width/><!--unexpected element-->
</property>
<property/><!--element missing-->

> - Most business logic cannot be represented, like "the value of the foo element must be less than or equal to the value of the bar element".

You will find here some usefull use cases of business logic representation:
http://reflex.gforge.inria.fr/tutorial-schemas.html

> - I occasionally was foiled by XSDs strict "no ambiguity" rules, which make it difficult to enforce rules such as "if the value of the foo element is xyz, then the bar and baz elements are required, but otherwise they are optional".

Straightforward in ASL:

<asl:element name="xxxxxx">
   <xcl:if test="{/path/to/foo = 'xyz'}">
     <xcl:then>
       <asl:sequence>
         <asl:element ref-elem="bar"/>
         <asl:element ref-elem="baz"/>
       </asl:sequence>
     </xcl:then>
     <xcl:else>
       <asl:sequence>
         <asl:element ref-elem="bar" min-occurs="0"/>
         <asl:element ref-elem="baz" min-occurs="0"/>
       </asl:sequence>
     </xcl:else>
   </xcl:if>
</asl:element>

you will notice that ASL can be mixed with other XML libraries (above, XCL that stands for the XML Control Language); this is a very useful feature that makes the abstract tree of the schema dynamic (other schema technologies have a static abstract tree) which boost the expressiveness of the language (isn't that a features that you are looking for ?)

>
> Nonetheless, I have been more or less pleased with the final results in most cases.  Having heard much praise of Relax NG, I recently decided to try it out, hoping it would be able to resolve these issues.  Here's what I found:
> - It does relieve the "no ambiguity" problem and makes enforcing that sort of rules much easier.
> - It does not resolve most of the business logic issues, but there are other technologies that can be used in combination with either Relax NG or XSD to resolve these issues, so that can be worked around in either case.
> - It does not appear to relieve the complexity of representing some types of data structures (see the one described above-as far as I can tell, it would not be significantly easier to define in Relax NG than it is in XSD).
> - There is only a rudimentary concept of type definition and virually
> none of inheritance.  Designing good, object-oriented data models is
> practically impossible in Relax NG; for me, this is a HUGE deficiency.
> For instance, say Foo is an abstract data type with several fields,
> including an id field (any mixed data allowed) and a type field (with
> enumerated values); Bar is one of several concrete subtypes of Foo,
> which restricts the content of the id field (must contain two
> elements, a groupId and a subId) and the type field (to a subset of
> the values enumerated by Foo) and appends several new fields; other
> subtypes have completely different restrictions on the id and type
> fields and append various different new fields.  In XSD, implementing
> this is fairly straightforward (although not necessarily simple and
> certainly more verbose than I would prefer):  Foo is an abstract
> complex type, and Bar and the other subtypes must derive from it first
> via restriction (creating the re
strictions on id and type) and then via extension (appending the new fields).  In Relax NG, by contrast, I could find only two ways of doing this, neither of which was at all acceptable:  One is to get rid of Foo completely and reference its fields separately in each subtype-this is unacceptable to me as it completely violates DRY and object-oriented design; the other is to define each subtype of Foo in a separate file, importing Foo and redefining as necessary-this is marginally better from an OOP point-of-view, but still makes it impossible for a schema to use "any valid subtype of Foo" as a content definition without violating the Open-Closed principle.
> - Relax NG provides no way of specifying cardinalities other than 0, 1, and infinity!  This is also a big one for me--if I need at least 2 and no more than 20 occurrences of the foo element (fairly common in my data definitions), there's NO WAY I'm going to define two required foo elements and eighteen optional ones!

In ASL, cardinalities support "2" or "20", but can also be dynamic "{1+1}" or "{count(../column[1]/cell)}" or whatever you find useful with XPath

>
> Now, I'm hoping that some Relax NG expert will come along and prove me wrong about some or all of these things, since I only experimented with it for a couple of days.  If not, for me these deficiencies in Relax NG *far* outweigh its benefits over XSD-there's really no comparison.  As much as I would like to see some aspects of XSD improved, unless someone can propose real alternatives for issues like the ones above, Relax NG is simply not an option for the type of data I work with.
>
> As a postscript, let me say I'm very thankful that the tone of this
> thread has been gradually transitioning to actual constructive
> argument (at which this list is normally rather good) rather than the
> ridiculously juvenile mud-slinging that was predominant early last
> week-in particular, I've appreciated the level-headed input from Jim
> Tivy, Liam Quin, Ken Holman, and Michael Kay.  Unfortunately there are
> still some emotionally charged but pretty much meaningless (or worse,
> completely false) words being thrown around, but at least those aren't
> in the majority anymore.  In all good humor, I have to say that
> XML-dev is by far the most argumentative mailing list to which I've
> ever been a subscriber. :-)
>
> Doug Glidden
> Software Engineer
> The Boeing Company
> Douglass.A.Glidden@boeing.com
> ______________________________________________________________________
> _
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS to
> support XML implementation and development. To minimize spam in the
> archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org List archive:
> http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>


--
Cordialement,

               ///
              (. .)
  --------ooO--(_)--Ooo--------
|      Philippe Poulard       |
  -----------------------------
  http://reflex.gforge.inria.fr/
        Have the RefleX !

Andrew Welch wrote:

> Is this still the Schematron-that-is-a-generated-XSLT-transform, or is
> there a Java based streaming validator available now?

What difference does it make? A Java implementation will not necessarily
be streaming:  Probotron4J for example is Java but not streaming.

Streaming has never been a design requirement for Schematron. There are a
couple of libraries that allow XPath rewriting of reverse axes, and these
could indeed allow streaming implementation.

However, someone could have made a streaming implementation long ago with
the STAX library, if they had wanted to. But no-one has in the last 10
years, to my knowledge, so I have no evidence that a streaming
implementation is something that anyone actually wants.

Cheers
Rick Jelliffe

Glidden, Douglass A wrote:
> From the perspective of an application developer who uses XML for storing fairly complex hierarchical data (as opposed to documents), I have a very basic question:  Is there any real, practical reason that Relax NG is _universally_ better than XSD?
>
> Here's the background for my question:  I've been using XSDs for generating object-oriented data models for some time, and have been frustrated by various inflexibilities in XSDs.  For instance:
> - Derivation by restriction, while conceptually very useful, seems slightly cumbersome in operation; in particular, I wish it could be accomplished with less repetition (I'm a fanatical believer in DRY).  I can live with the repetition, though, because I am assured that, if I make a typographical or some other error, the XSD will (almost) always fail to validate ("Bar is not a valid restriction of Foo"), so the the possiblity of unintended changes being introduced due to the repetition is kept to a minimum.
> - Certain fairly simple data structures can only be achieved in XSD (if at all) with ridiculously verbose patterns; for instance, how do you represent a structure with five possible elements, at least one of which must be present and none of which may be repeated?  The best method I have found takes about 25 lines and far too much code repetition.
> - Most business logic cannot be represented, like "the value of the foo element must be less than or equal to the value of the bar element".
> - I occasionally was foiled by XSDs strict "no ambiguity" rules, which make it difficult to enforce rules such as "if the value of the foo element is xyz, then the bar and baz elements are required, but otherwise they are optional".
>
> Nonetheless, I have been more or less pleased with the final results in most cases.  Having heard much praise of Relax NG, I recently decided to try it out, hoping it would be able to resolve these issues.  Here's what I found:
> - It does relieve the "no ambiguity" problem and makes enforcing that sort of rules much easier.
> - It does not resolve most of the business logic issues, but there are other technologies that can be used in combination with either Relax NG or XSD to resolve these issues, so that can be worked around in either case.
> - It does not appear to relieve the complexity of representing some types of data structures (see the one described above-as far as I can tell, it would not be significantly easier to define in Relax NG than it is in XSD).
> - There is only a rudimentary concept of type definition and virually none of inheritance.  Designing good, object-oriented data models is practically impossible in Relax NG; for me, this is a HUGE deficiency.  For instance, say Foo is an abstract data type with several fields, including an id field (any mixed data allowed) and a type field (with enumerated values); Bar is one of several concrete subtypes of Foo, which restricts the content of the id field (must contain two elements, a groupId and a subId) and the type field (to a subset of the values enumerated by Foo) and appends several new fields; other subtypes have completely different restrictions on the id and type fields and append various different new fields.  In XSD, implementing this is fairly straightforward (although not necessarily simple and certainly more verbose than I would prefer):  Foo is an abstract complex type, and Bar and the other subtypes must derive from it first via restriction (creating the restrictions on id and type) and then via extension (appending the new fields).  In Relax NG, by contrast, I could find only two ways of doing this, neither of which was at all acceptable:  One is to get rid of Foo completely and reference its fields separately in each subtype-this is unacceptable to me as it completely violates DRY and object-oriented design; the other is to define each subtype of Foo in a separate file, importing Foo and redefining as necessary-this is marginally better from an OOP point-of-view, but still makes it impossible for a schema to use "any valid subtype of Foo" as a content definition without violating the Open-Closed principle.
> - Relax NG provides no way of specifying cardinalities other than 0, 1, and infinity!  This is also a big one for me--if I need at least 2 and no more than 20 occurrences of the foo element (fairly common in my data definitions), there's NO WAY I'm going to define two required foo elements and eighteen optional ones!
>
> Now, I'm hoping that some Relax NG expert will come along and prove me wrong about some or all of these things, since I only experimented with it for a couple of days.  If not, for me these deficiencies in Relax NG *far* outweigh its benefits over XSD-there's really no comparison.  As much as I would like to see some aspects of XSD improved, unless someone can propose real alternatives for issues like the ones above, Relax NG is simply not an option for the type of data I work with
>   
Three responses.
 

First is that XSD was not designed as an abstract data modeling language 
but rather a markup description language: even though the grammars have 
been somewhat extended with xsd:any and wildcards (and now assertions 
and conditional types), XSD is not a substitute for the kinds of things 
you would use, say, UML for. (And then convert the UML to RELAX NG.) 
XBRL is an example of a system that attempts to piggyback data modeling 
on top of XSD, and makes an almost fatal complexity. 

The second is that IMHO it is better to see RELAX NG as part of the ISO 
DSDL standard, so when there is some feature missing from RELAX NG, it 
may be provided elsewhere.  This is a layered approach, where each 
language tries to do one thing really well, rather than a monolithic 
approach where . For example, you organize your system to that there is 
a master RELAX NG schema, then you add your restrictions as a layer in 
Schematron. You do the detailed cardnality constraints in Schematron. 
The advantage of this approach is that you are not at the mercy of the 
abstractions provided by the schema language: you don't need to think 
"what combination of restiction/extension/redeclaration do I need, and 
what are the order, ambiguity and UPA issues?" because you can just say 
"Inside a Bar, element X is not allowed" directly.

Why model abstractly things that can be more simply said directly?   If 
the intent of all the data modeling is to reveal the connections and 
design of the system, the grammar-based systems have the fundamentally 
poor abstraction weakness that they don't allow negatives or exclusions, 
one of the great tools of logic simplification: you don't need to have 
types in order to do restrictions, and you don't have to have types in 
order to document and explain restrictions.

The third thing is that the design of RELAX NG is, to my mind, that 
abstract assertions about the relationships between schema items does 
not belong in the schemas themselves. They would better left to tools. 
For example, there is a third party tool that checks if a RELAX NG 
schema is ambiguous or not. But certainly the lack of tools for RELAX NG 
shows that it has not been adopted by people who are so keen on 
automated conversion from abstract types to the schema language. Because 
of RELAX NGs strong theoretical base, it should be easier to do these 
kind of set operations on. 

I would note that one large schema we maintain, in which a master 
vocabulary is used by a dozen committees to make multiple local schemas, 
we actually remove type derivation information from the working draft 
schemas, and only add it at the end (it is a pain) because it is too 
much trouble maintaining the base schemas to track the work in progress. 
So we use XSD, but in the same kind of approach (and custom tooling) 
that would have been necessary if we were using RELAX NG. (In fact, they 
are discussing abandoning much use of type derivation and just moving to 
a master XSD and much more Schematron, just for reasons of directness 
and simplicity.)

> As a postscript, let me say I'm very thankful that the tone of this thread has been gradually transitioning to actual constructive argument (at which this list is normally rather good) rather than the ridiculously juvenile mud-slinging that was predominant early last week-in particular, I've appreciated the level-headed input from Jim Tivy, Liam Quin, Ken Holman, and Michael Kay.
Argumentative on this topic: that would be me! I plead guilty.  For 
constructive responses, please see the ISO standards for RELAX NG and 
Schematron etc which a number of us on this list have worked on, and the 
open source implementations we have made. Please also note that there 
was no mud-slinging at individuals. If some of us have lost patience at 
XSD because we have to use the wretched monstrosity, please don't blame 
the victims!

Cheers
Rick Jelliffe

P.S. "how do you represent a structure with five possible elements, at 
least one of which must be present and none of which may be repeated? 
The best method I have found takes about 25 lines and far too much code 
repetition."

Here is one straight-forward Schematron schema, in 10 lines, including 
documentation, if I understand the requirement.

<rule context="x">
   <assert test="count(a) &lt;= 1">An a may only appear once in an 
x</assert>
   <assert test="count(b) &lt;= 1">An b may only appear once in an 
x</assert>
   <assert test="count(c) &lt;= 1">An d may only appear once in an 
x</assert>
   <assert test="count(d) &lt;= 1">An e may only appear once in an 
x</assert>
   <assert test="count(e) &lt;= 1">An f may only appear once in an 
x</assert>
   <assert test="count(*) > 0">There must be at least one element in 
x</assert>
   <assert test="count(*) = count(a) + count(b) + count(c) + count(d) + 
count(e)">
     The only elements allowed in x are a, b, c, d, and e</assert>
</rule>

And here is another version, which demonstrates that you can do kinds of 
abstract type modeling
in Schematron that are unavailable in XSD. In this case, it is mixin 
style modeling (like interfaces
in Java). Note that there is no combinatorial explosion: in fact, to add 
an extra child you only
need to alter the last XPath, not even add a new element.

<rule abstract="true" id="non-empty-element">
   <assert test="count(*) > 0">There must be at least one element in 
<name/></assert>
</rule>

<rule abstract="true"  id="bag">
   <assert test="*[not(following-sibling::*[name()=current()/name()])]">
   No child of x may appear more than once.</assert>
</rule>

<rule context="x">
   <extends rule="bag" />
   <extends rule="non-empty-element" /> 
   <assert test="count(*) = count(a) + count(b) + count(c) + count(d) + 
count(e)">
     The only elements allowed in x are a, b, c, d, and e</assert>
</rule>

I think that XSD 1.1 will have an OK story: you could use xsd:any with 
maxOccurs=1,
and then an assertion using     count(*) > 0      I guess you could use 
RELAX NG & in the
same way.

PyXB version 1.0.0 is now available from SourceForge at: 
http://sourceforge.net/projects/pyxb/

PyXB (Python XML Schema Bindings; "pixbee") is a pure Python package 
that generates Python source code for classes that correspond to data 
structures defined by XMLSchema.  In concept it is similar to JAXB for 
Java and CodeSynthesis XSD for C++.  The current release supports XML 
Schema 1.0.

Version 1.0.0 is feature-complete and suitable for production 
development where validation of incoming and generated documents is 
desired.  The following schema constructs are supported:

    * Class constants corresponding to string enumeration constraints
    * Simple and complex type definitions
    * List and union datatypes
    * Constraints on (simple) datatypes (e.g., minInclusive, length)
    * Model groups and attribute groups
    * Complex content models (all, sequence, choice); minOccurs and 
maxOccurs
    * Abstract types, xsi:type, substitution groups
    * Nillable elements with xsi:nil
    * Namespace qualified attributes and elements
    * Documentation annotations present in the schema are converted to 
Python docstrings in the generated bindings.
    * Pattern constraints (as long as the simple type derives from 
something that is represented as a Python string)

The full distribution includes generated bindings for twenty-three 
namespaces related to web services (including several versions of SOAP 
and WSDL), and twenty-eight namespaces from the Open Geospatial 
Consortium's Geographic Information System schema.

For further information and links to the support forum and mailing list, 
please consult the documentation available at: http://pyxb.sourceforge.net/

Peter

2009/11/18  <rjelliffe@allette.com.au>:
> Andrew Welch wrote:
>
>> Is this still the Schematron-that-is-a-generated-XSLT-transform, or is
>> there a Java based streaming validator available now?
>
> What difference does it make? A Java implementation will not necessarily
> be streaming:  Probotron4J for example is Java but not streaming.

Looking at the source for Probotron4J, it simply creates and then
applies the XSLT.... ?   That isn't really an implemenation, more like
a helper utility.

I mentioned the streaming aspect for 2 reasons:

1) If validation performance is an issue, can RNG + Schematron still
be considered when XSD is validation so fast?

2) Isn't it the case that some of the complexities of XSD are that way
to allow for that validation speed?



-- 
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

> Undoubtedly XSD could have been better, but if your requirements are met by
> what it offers then there's no real motivation to use some other approach.

For the work I've done, XSD with xs:assert and xs:alternative covers
most things (roll on XSD 1.1!)... If someone is considering which
schema language to use on a project today, surely XSD is the still the
right choice with 1.1 around the corner (or just use it now with
Saxon)?

XSD 1.1 vs RNG / Schematron / Whatever is a more worthwhile
discussion, the debate involving 1.0 has been done to death, and will
be out of date soon.





-- 
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

2009/11/18 Rick Jelliffe <rjelliffe@allette.com.au>:
> The second is that IMHO it is better to see RELAX NG as part of the ISO DSDL
> standard, so when there is some feature missing from RELAX NG, it may be
> provided elsewhere.  This is a layered approach, where each language tries
> to do one thing really well, rather than a monolithic approach where . For
> example, you organize your system to that there is a master RELAX NG schema,
> then you add your restrictions as a layer in Schematron. You do the detailed
> cardnality constraints in Schematron.

Is this still the Schematron-that-is-a-generated-XSLT-transform, or is
there a Java based streaming validator available now?



-- 
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

I think a lot of this is horses for courses.

I think for a lot of programmers with a Java/C#/C++ background the structure 
of XSD mirrors what they do in classes.  So something like:

<xs:element name='foo' type='xs:int' minOccurs='0' maxOccurs='unbounded'/>

is more readily mappable to their everyday mindset than:

<zeroOrMore>
    <element name='foo'>
        <data type='int/>
    </element>
</zeroOrMore>

Also, as classes tend to be a bag of bits without much co-occurence 
constraints it probably doesn't even occur to them that XSD is weak in this 
area.

I think for a lot of people XML is just a means to an end, and the less they 
have to deal with it the better.  So they pick up the first thing they see, 
decide that it's adequate for their needs and go with it.  Maybe also a case 
of nobody got fired for choosing XSD!

For example, it was stated previously on this list that people tend to do 
things in Java that would be better done in XSLT.  Sometimes the devil you 
know is the better route to take.  Especially as pointy-haired boss likes to 
see regular progress updates, and doesn't consider surfing the web as 
productive.

Undoubtedly XSD could have been better, but if your requirements are met by 
what it offers then there's no real motivation to use some other approach.

My 2 cents (for today at least!)

Pete Cordell
Codalogic Ltd
Interface XML to C++ the easy way using XML C++
data binding to convert XSD schemas to C++ classes.
Visit http://codalogic.com/lmx/ or http://www.xml2cpp.com
for more info
----- Original Message ----- 
From: "Rick Jelliffe" <rjelliffe@allette.com.au>
To: <xml-dev@lists.xml.org>
Sent: Wednesday, November 18, 2009 7:13 AM
Subject: Re:  XML spec and XSD



Glidden, Douglass A wrote:
> From the perspective of an application developer who uses XML for storing 
> fairly complex hierarchical data (as opposed to documents), I have a very 
> basic question:  Is there any real, practical reason that Relax NG is 
> _universally_ better than XSD?
>
> Here's the background for my question:  I've been using XSDs for 
> generating object-oriented data models for some time, and have been 
> frustrated by various inflexibilities in XSDs.  For instance:
> - Derivation by restriction, while conceptually very useful, seems 
> slightly cumbersome in operation; in particular, I wish it could be 
> accomplished with less repetition (I'm a fanatical believer in DRY).  I 
> can live with the repetition, though, because I am assured that, if I make 
> a typographical or some other error, the XSD will (almost) always fail to 
> validate ("Bar is not a valid restriction of Foo"), so the the possiblity 
> of unintended changes being introduced due to the repetition is kept to a 
> minimum.
> - Certain fairly simple data structures can only be achieved in XSD (if at 
> all) with ridiculously verbose patterns; for instance, how do you 
> represent a structure with five possible elements, at least one of which 
> must be present and none of which may be repeated?  The best method I have 
> found takes about 25 lines and far too much code repetition.
> - Most business logic cannot be represented, like "the value of the foo 
> element must be less than or equal to the value of the bar element".
> - I occasionally was foiled by XSDs strict "no ambiguity" rules, which 
> make it difficult to enforce rules such as "if the value of the foo 
> element is xyz, then the bar and baz elements are required, but otherwise 
> they are optional".
>
> Nonetheless, I have been more or less pleased with the final results in 
> most cases.  Having heard much praise of Relax NG, I recently decided to 
> try it out, hoping it would be able to resolve these issues.  Here's what 
> I found:
> - It does relieve the "no ambiguity" problem and makes enforcing that sort 
> of rules much easier.
> - It does not resolve most of the business logic issues, but there are 
> other technologies that can be used in combination with either Relax NG or 
> XSD to resolve these issues, so that can be worked around in either case.
> - It does not appear to relieve the complexity of representing some types 
> of data structures (see the one described above-as far as I can tell, it 
> would not be significantly easier to define in Relax NG than it is in 
> XSD).
> - There is only a rudimentary concept of type definition and virually none 
> of inheritance.  Designing good, object-oriented data models is 
> practically impossible in Relax NG; for me, this is a HUGE deficiency. 
> For instance, say Foo is an abstract data type with several fields, 
> including an id field (any mixed data allowed) and a type field (with 
> enumerated values); Bar is one of several concrete subtypes of Foo, which 
> restricts the content of the id field (must contain two elements, a 
> groupId and a subId) and the type field (to a subset of the values 
> enumerated by Foo) and appends several new fields; other subtypes have 
> completely different restrictions on the id and type fields and append 
> various different new fields.  In XSD, implementing this is fairly 
> straightforward (although not necessarily simple and certainly more 
> verbose than I would prefer):  Foo is an abstract complex type, and Bar 
> and the other subtypes must derive from it first via restriction (creating 
> the restrictions on id and type) and then via extension (appending the new 
> fields).  In Relax NG, by contrast, I could find only two ways of doing 
> this, neither of which was at all acceptable:  One is to get rid of Foo 
> completely and reference its fields separately in each subtype-this is 
> unacceptable to me as it completely violates DRY and object-oriented 
> design; the other is to define each subtype of Foo in a separate file, 
> importing Foo and redefining as necessary-this is marginally better from 
> an OOP point-of-view, but still makes it impossible for a schema to use 
> "any valid subtype of Foo" as a content definition without violating the 
> Open-Closed principle.
> - Relax NG provides no way of specifying cardinalities other than 0, 1, 
> and infinity!  This is also a big one for me--if I need at least 2 and no 
> more than 20 occurrences of the foo element (fairly common in my data 
> definitions), there's NO WAY I'm going to define two required foo elements 
> and eighteen optional ones!
>
> Now, I'm hoping that some Relax NG expert will come along and prove me 
> wrong about some or all of these things, since I only experimented with it 
> for a couple of days.  If not, for me these deficiencies in Relax NG *far* 
> outweigh its benefits over XSD-there's really no comparison.  As much as I 
> would like to see some aspects of XSD improved, unless someone can propose 
> real alternatives for issues like the ones above, Relax NG is simply not 
> an option for the type of data I work with
>
Three responses.


First is that XSD was not designed as an abstract data modeling language
but rather a markup description language: even though the grammars have
been somewhat extended with xsd:any and wildcards (and now assertions
and conditional types), XSD is not a substitute for the kinds of things
you would use, say, UML for. (And then convert the UML to RELAX NG.)
XBRL is an example of a system that attempts to piggyback data modeling
on top of XSD, and makes an almost fatal complexity.

The second is that IMHO it is better to see RELAX NG as part of the ISO
DSDL standard, so when there is some feature missing from RELAX NG, it
may be provided elsewhere.  This is a layered approach, where each
language tries to do one thing really well, rather than a monolithic
approach where . For example, you organize your system to that there is
a master RELAX NG schema, then you add your restrictions as a layer in
Schematron. You do the detailed cardnality constraints in Schematron.
The advantage of this approach is that you are not at the mercy of the
abstractions provided by the schema language: you don't need to think
"what combination of restiction/extension/redeclaration do I need, and
what are the order, ambiguity and UPA issues?" because you can just say
"Inside a Bar, element X is not allowed" directly.

Why model abstractly things that can be more simply said directly?   If
the intent of all the data modeling is to reveal the connections and
design of the system, the grammar-based systems have the fundamentally
poor abstraction weakness that they don't allow negatives or exclusions,
one of the great tools of logic simplification: you don't need to have
types in order to do restrictions, and you don't have to have types in
order to document and explain restrictions.

The third thing is that the design of RELAX NG is, to my mind, that
abstract assertions about the relationships between schema items does
not belong in the schemas themselves. They would better left to tools.
For example, there is a third party tool that checks if a RELAX NG
schema is ambiguous or not. But certainly the lack of tools for RELAX NG
shows that it has not been adopted by people who are so keen on
automated conversion from abstract types to the schema language. Because
of RELAX NGs strong theoretical base, it should be easier to do these
kind of set operations on.

I would note that one large schema we maintain, in which a master
vocabulary is used by a dozen committees to make multiple local schemas,
we actually remove type derivation information from the working draft
schemas, and only add it at the end (it is a pain) because it is too
much trouble maintaining the base schemas to track the work in progress.
So we use XSD, but in the same kind of approach (and custom tooling)
that would have been necessary if we were using RELAX NG. (In fact, they
are discussing abandoning much use of type derivation and just moving to
a master XSD and much more Schematron, just for reasons of directness
and simplicity.)

> As a postscript, let me say I'm very thankful that the tone of this thread 
> has been gradually transitioning to actual constructive argument (at which 
> this list is normally rather good) rather than the ridiculously juvenile 
> mud-slinging that was predominant early last week-in particular, I've 
> appreciated the level-headed input from Jim Tivy, Liam Quin, Ken Holman, 
> and Michael Kay.
Argumentative on this topic: that would be me! I plead guilty.  For
constructive responses, please see the ISO standards for RELAX NG and
Schematron etc which a number of us on this list have worked on, and the
open source implementations we have made. Please also note that there
was no mud-slinging at individuals. If some of us have lost patience at
XSD because we have to use the wretched monstrosity, please don't blame
the victims!

Cheers
Rick Jelliffe

P.S. "how do you represent a structure with five possible elements, at
least one of which must be present and none of which may be repeated?
The best method I have found takes about 25 lines and far too much code
repetition."

Here is one straight-forward Schematron schema, in 10 lines, including
documentation, if I understand the requirement.

<rule context="x">
   <assert test="count(a) &lt;= 1">An a may only appear once in an
x</assert>
   <assert test="count(b) &lt;= 1">An b may only appear once in an
x</assert>
   <assert test="count(c) &lt;= 1">An d may only appear once in an
x</assert>
   <assert test="count(d) &lt;= 1">An e may only appear once in an
x</assert>
   <assert test="count(e) &lt;= 1">An f may only appear once in an
x</assert>
   <assert test="count(*) > 0">There must be at least one element in
x</assert>
   <assert test="count(*) = count(a) + count(b) + count(c) + count(d) +
count(e)">
     The only elements allowed in x are a, b, c, d, and e</assert>
</rule>

And here is another version, which demonstrates that you can do kinds of
abstract type modeling
in Schematron that are unavailable in XSD. In this case, it is mixin
style modeling (like interfaces
in Java). Note that there is no combinatorial explosion: in fact, to add
an extra child you only
need to alter the last XPath, not even add a new element.

<rule abstract="true" id="non-empty-element">
   <assert test="count(*) > 0">There must be at least one element in
<name/></assert>
</rule>

<rule abstract="true"  id="bag">
   <assert test="*[not(following-sibling::*[name()=current()/name()])]">
   No child of x may appear more than once.</assert>
</rule>

<rule context="x">
   <extends rule="bag" />
   <extends rule="non-empty-element" />
   <assert test="count(*) = count(a) + count(b) + count(c) + count(d) +
count(e)">
     The only elements allowed in x are a, b, c, d, and e</assert>
</rule>

I think that XSD 1.1 will have an OK story: you could use xsd:any with
maxOccurs=1,
and then an assertion using     count(*) > 0      I guess you could use
RELAX NG & in the
same way.


_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

From: Rick Jelliffe [mailto:rjelliffe@allette.com.au]
Three responses.


First is that XSD was not designed as an abstract data modeling language but rather a markup description language: even though the grammars have been somewhat extended with xsd:any and wildcards (and now assertions and conditional types), XSD is not a substitute for the kinds of things you would use, say, UML for. (And then convert the UML to RELAX NG.) XBRL is an example of a system that attempts to piggyback data modeling on top of XSD, and makes an almost fatal complexity.



Okay, I'm not sure I entirely followed that; using UML to model data structures is fine and great, but UML doesn't enforce anything.  If the contract specified by your abstract (UML) data model is not enforced by your concrete data model (be that XSD or RNG or a combination of layers), then your abstract model is a lie and any code that depends on it will be unreliable.  However...



The second is that IMHO it is better to see RELAX NG as part of the ISO DSDL standard, so when there is some feature missing from RELAX NG, it may be provided elsewhere.  This is a layered approach, where each language tries to do one thing really well, rather than a monolithic approach where . For example, you organize your system to that there is a master RELAX NG schema, then you add your restrictions as a layer in Schematron. You do the detailed cardnality constraints in Schematron.
The advantage of this approach is that you are not at the mercy of the abstractions provided by the schema language: you don't need to think "what combination of restiction/extension/redeclaration do I need, and what are the order, ambiguity and UPA issues?" because you can just say "Inside a Bar, element X is not allowed" directly.



Fair enough, but the projects I work on tend already to have a rather painful number of layers, so I try to avoid adding more layers than is absolutely necessary.  On the other hand, I have recently been looking into adding a Schematron layer anyway, in order to enforce some of the rules that XSD cannot (or cannot easily) enforce, so I may re-evaluate RNG as part of that process.



[...]

The third thing is that the design of RELAX NG is, to my mind, that abstract assertions about the relationships between schema items does not belong in the schemas themselves. They would better left to tools. [...]



I'm not sure that I agree with that, but that is perhaps a philosophical discussion in which I'm not prepared to engage just now. :-)



I would note that one large schema we maintain, in which a master vocabulary is used by a dozen committees to make multiple local schemas, we actually remove type derivation information from the working draft schemas, and only add it at the end (it is a pain) because it is too much trouble maintaining the base schemas to track the work in progress. [...]



I would only comment that it _sounds_ (if I understand your description correctly) like you're not following the open-closed principle; perhaps the difficulty is due to the design and not the grammar.

Doug Glidden
Software Engineer
The Boeing Company
Douglass.A.Glidden@boeing.com

From: Michael Sokolov [mailto:sokolov@ifactory.com]
"I guess that's just to say that they seem to occupy different niches."

I agree with that 100%, and I would agree with you that that's how it should be.  I can certainly see how RNG is easier to learn quickly than XSD, and I think that's great and would not want it to change due to scope creep.  I have no problem with the idea that for _many_ applications, XSD is overkill and RNG would be vastly better.  What I'm struggling to understand is that some people seem to have a mentality that "XSD is fundamentally and universally bad and should never be used by anyone".  I don't think those people would agree with your statement above, but I'm trying to honestly figure out why those people think the way they do before writing it off as simple prejudice.

The sense I'm starting to get is that, for the most part, these people have become frustrated with being forced to use XSD in situations where it _would_ be far better to use RNG, and in their frustration, they are throwing the baby out with the bathwater.  If that's the case, I can totally understand that (I just wish those people would be a little less dogmatic about it! :-) ).

Doug Glidden
Software Engineer
The Boeing Company
Douglass.A.Glidden@boeing.com

On Wed, Nov 18, 2009 at 2:21 AM, Glidden, Douglass A
<Douglass.A.Glidden@boeing.com> wrote:
> - Certain fairly simple data structures can only be achieved in XSD (if at all) with ridiculously verbose patterns;

I agree, that XSD Schemas can be quite verbose sometimes. But I think,
that's some of the cost we probably incuur to achieve needed
functionality.

For example,

You may consider this element definition to be OK, and not verbose:

[1]
<xs:element name="x" type="xs:string" />

But you may consider this one as verbose:

[2]
<xs:element name="x">
  <xs:simpleType>
    <xs:restriction base="xs:string">
      <xs:enumeration value="a"/>
      <xs:enumeration value="b"/>
      <xs:enumeration value="c"/>
    </xs:restriction>
  </xs:simpleType>
</xs:element>

But a definition like this one [2], helps us realize a necessary (for
somebody) use case. Unfortunately, a certain amount of
syntax/verboseness is needed to express something.

If needed, somebody may use an IDE to write XSD Schemas, which can
help with (most of) the verboseness issues.

> - Most business logic cannot be represented, like "the value of the foo element must be less than or equal to the value of the bar element".

I think, this will be expressible in XSD 1.1, using assertions for e.g:

<xs:assert test="foo le bar" />

> - I occasionally was foiled by XSDs strict "no ambiguity" rules, which make it difficult to enforce rules such as "if the value of the foo element is xyz, then the bar and baz elements are required, but otherwise they are optional".

Again this should be possible to express with XSD 1.1 assertions, for
e.g something like:

<xs:assert test="if (foo eq 'xyz') then (bar and baz)" />


-- 
Regards,
Mukul Gandhi

To my way of thinking, the compact notation and relative simplicity is what
makes RNC/G extremely attractive.  I'm actually really glad it doesn't solve
your problem, Douglass!  No offense, but if it did, it would be as complex
as XSD.  I guess that's just to say that they seem to occupy different
niches.

In my first project using Relax, I was able to write schemas for my fairly
complex documents after reading documentation (James Clark's excellent
tutorial http://www.relaxng.org/compact-tutorial-20030326.html) for only
about an hour.  Admittedly my problems were much simpler than the ones you
describe - only simple inheritance schemes using extension (adding
additional elements and attributes in the subtype), but what was great was I
didn't have to become an expert to do it, and all my colleagues: programmers
and customers (content editors), are able to read the schemas with almost no
training at all.

-Mike


> -----Original Message-----
> From: Glidden, Douglass A [mailto:Douglass.A.Glidden@boeing.com] 
> Sent: Tuesday, November 17, 2009 5:09 PM
....
> Hmmm...I must confess I didn't really explore the shorthand 
> version of RNG--perhaps I should take a look at it.  That is 
> effectively the same as the longhand way I was doing it, and 
> while it doesn't strike me as particularly elegant, it at 
> least takes up less space.
> 
> 
> 

- Certain fairly simple data structures can only be achieved in XSD (if
  at all) with ridiculously verbose patterns; for instance, how do you
  represent a structure with five possible elements, at least one of
  which must be present and none of which may be repeated?  The best
  method I have found takes about 25 lines and far too much code
  repetition. 
...

- It [rng]  does not appear to relieve the complexity of representing some
   types of data structures (see the one described above-as far as I can
   tell, it would not be significantly easier to define in Relax NG than
   it is in XSD).


If I understand correctly, in relaxNG, something like:



start = x

x = element x {
   ( a  & b? & c? & d? & e?) |
   ( a? & b  & c? & d? & e?) |
   ( a? & b? & c  & d? & e?) |
   ( a? & b? & c? & d  & e?) |
   ( a? & b? & c? & d? & e ) 
 }

a = element a {empty}
b = element b {empty}
c = element c {empty}
d = element d {empty}
e = element e {empty}


does what you want, it validates

<x>
 <a/>
 <b/>
</x>

and invalidates

<x>
 <b/>
 <a/>
 <b/>
</x>


- Relax NG provides no way of specifying cardinalities other than 0, 1,
  and infinity!  This is also a big one for me--if I need at least 2 and
  no more than 20 occurrences of the foo element (fairly common in my
  data definitions), there's NO WAY I'm going to define two required foo
  elements and eighteen optional ones!


usually you can persuade yourself that 1,?,+,* is enough and leave
cardinality checking to higher up the chain, but if you really need
this, then powers of two come in handy:-)


y = element y { z,z,z16,z2}
z16 = z8,z8
z8  = z4,z4
z4  = z2,z2
z2  = z?,z?
z = element z {empty}

validates
<y><z/><z/></y>

but invalidates instances with less than 1 or more than 20 zs.

david

________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

Mukul Gandhi wrote:
> It seems, that most people argue that a conformant XML document has
> ability to refer to user defined entities, and this can be specified
> only in a DTD. I think, the current XML spec, makes DTDs an
> inseparable part of XML spec is, because of the DTDs ability to
> specify user defined entities, which (the entities) are part of say
> the, XML 1.0 (and 1.1) spec.
>   
And because of attribute defaults and non-significant whitespace 
removal.  And perhaps for ID checking and indexing.
> To change the XML spec, to remove the concept of valid/validity and
> being able to remove DTD s from the XML spec, I think will also
> require us to remove the concepts of entities from the XML spec (I
> think, something like a future XML 2.0 spec should only specify the
> built in entities like & amp; etc and not allow users to specify user
> defined entities, unless they use an external thing like DTD or a
> future XSD (a 1.2 version, which may allow users to specify entities,
> via some global declaration perhaps))
Well, entities for special characters and entities for document or 
schema construction  are  perhaps different
animals.   I have long argued that XML should have all the ISO/Math/W3C 
special characters just built in. There was some argument that this 
would make an XML processor much larger which would be a difficulty for 
mobile devices: it seemed silly to me then.  (The fetishism for an 
absolutely minimal XML is, to me, a classic case of a local optimization 
that would cause a global sub-optimality: when you remove a feature from 
a layer that people need, it just goes to a different layer and you 
don't actually get a total simplification--indeed you may get more 
complexity to support the extra layers especially if they are 
uncoordinated. The W3C answer that the way to handle all the 
uncoordinated layers is to make the part of XProc (i.e. a different file 
to be downloaded in order to know the simple functionality of standard 
markup) is insultingly dumb: whatever XProc is good for, it is not going 
to be an adequate packaging mechanism.)

If you built the special characters in, and you built in XInclude, 
XBase,  xml:id, and some method of saying "this is element content so 
ignore the whitespace" then you could indeed move DTDs to a different 
layer.  Note that at ISO there already is a spec (as part of DSDL) for 
namespace-aware DTDs; if that was useful for anything. 

Also of possible interest is that you could have a DOCTYPE declaration 
use some non-DTD system. So, for example, you could restrict the 
internal subset of the prolog to entity declarations (if you wanted it 
all) and have the external subset being XSD or RELAX NG or Schematron etc.

Cheers
Rick Jelliffe

B.t.w. There is absolutely no problem with this and SGML compliance: it 
would just be a minor change to IS8879 to look after the formalities for 
"tag-valid" documents (in fact, IIRC it was Charles Goldfarb who asked a 
decade ago whether it would be useful for giving the W3C/XML/HTML people 
more freedom in this area, but I think the feeling was that we may as 
well wait for a real use case rather than second guess anyone.)  It 
isn't SGML that holds back innovation, though the belief that SGML can 
only be adhered to or dropped completely can hold back innovation.

[...]

If I understand correctly, in relaxNG, something like:



start = x

x = element x {
   ( a  & b? & c? & d? & e?) |
   ( a? & b  & c? & d? & e?) |
   ( a? & b? & c  & d? & e?) |
   ( a? & b? & c? & d  & e?) |
   ( a? & b? & c? & d? & e )
 }

a = element a {empty}
b = element b {empty}
c = element c {empty}
d = element d {empty}
e = element e {empty}


does what you want, it validates

<x>
 <a/>
 <b/>
</x>

and invalidates

<x>
 <b/>
 <a/>
 <b/>
</x>



Hmmm...I must confess I didn't really explore the shorthand version of RNG--perhaps I should take a look at it.  That is effectively the same as the longhand way I was doing it, and while it doesn't strike me as particularly elegant, it at least takes up less space.



[...]

usually you can persuade yourself that 1,?,+,* is enough and leave cardinality checking to higher up the chain, but if you really need this, then powers of two come in handy:-)


y = element y { z,z,z16,z2}
z16 = z8,z8
z8  = z4,z4
z4  = z2,z2
z2  = z?,z?
z = element z {empty}

validates
<y><z/><z/></y>

but invalidates instances with less than 1 or more than 20 zs.



LOL, well, it's a bit of a hack, but I suppose that works.  Still, it's nowhere near as clean and readable (and after all, readability is one of the big claims of RNG, from what I've seen) as XSD's minOccurs and maxOccurs.

Thanks for the response, David.  Hoping for more of this kind from others.

Doug Glidden
Software Engineer
The Boeing Company
Douglass.A.Glidden@boeing.com

From the perspective of an application developer who uses XML for storing fairly complex hierarchical data (as opposed to documents), I have a very basic question:  Is there any real, practical reason that Relax NG is _universally_ better than XSD?

Here's the background for my question:  I've been using XSDs for generating object-oriented data models for some time, and have been frustrated by various inflexibilities in XSDs.  For instance:
- Derivation by restriction, while conceptually very useful, seems slightly cumbersome in operation; in particular, I wish it could be accomplished with less repetition (I'm a fanatical believer in DRY).  I can live with the repetition, though, because I am assured that, if I make a typographical or some other error, the XSD will (almost) always fail to validate ("Bar is not a valid restriction of Foo"), so the the possiblity of unintended changes being introduced due to the repetition is kept to a minimum.
- Certain fairly simple data structures can only be achieved in XSD (if at all) with ridiculously verbose patterns; for instance, how do you represent a structure with five possible elements, at least one of which must be present and none of which may be repeated?  The best method I have found takes about 25 lines and far too much code repetition.
- Most business logic cannot be represented, like "the value of the foo element must be less than or equal to the value of the bar element".
- I occasionally was foiled by XSDs strict "no ambiguity" rules, which make it difficult to enforce rules such as "if the value of the foo element is xyz, then the bar and baz elements are required, but otherwise they are optional".

Nonetheless, I have been more or less pleased with the final results in most cases.  Having heard much praise of Relax NG, I recently decided to try it out, hoping it would be able to resolve these issues.  Here's what I found:
- It does relieve the "no ambiguity" problem and makes enforcing that sort of rules much easier.
- It does not resolve most of the business logic issues, but there are other technologies that can be used in combination with either Relax NG or XSD to resolve these issues, so that can be worked around in either case.
- It does not appear to relieve the complexity of representing some types of data structures (see the one described above-as far as I can tell, it would not be significantly easier to define in Relax NG than it is in XSD).
- There is only a rudimentary concept of type definition and virually none of inheritance.  Designing good, object-oriented data models is practically impossible in Relax NG; for me, this is a HUGE deficiency.  For instance, say Foo is an abstract data type with several fields, including an id field (any mixed data allowed) and a type field (with enumerated values); Bar is one of several concrete subtypes of Foo, which restricts the content of the id field (must contain two elements, a groupId and a subId) and the type field (to a subset of the values enumerated by Foo) and appends several new fields; other subtypes have completely different restrictions on the id and type fields and append various different new fields.  In XSD, implementing this is fairly straightforward (although not necessarily simple and certainly more verbose than I would prefer):  Foo is an abstract complex type, and Bar and the other subtypes must derive from it first via restriction (creating the restrictions on id and type) and then via extension (appending the new fields).  In Relax NG, by contrast, I could find only two ways of doing this, neither of which was at all acceptable:  One is to get rid of Foo completely and reference its fields separately in each subtype-this is unacceptable to me as it completely violates DRY and object-oriented design; the other is to define each subtype of Foo in a separate file, importing Foo and redefining as necessary-this is marginally better from an OOP point-of-view, but still makes it impossible for a schema to use "any valid subtype of Foo" as a content definition without violating the Open-Closed principle.
- Relax NG provides no way of specifying cardinalities other than 0, 1, and infinity!  This is also a big one for me--if I need at least 2 and no more than 20 occurrences of the foo element (fairly common in my data definitions), there's NO WAY I'm going to define two required foo elements and eighteen optional ones!

Now, I'm hoping that some Relax NG expert will come along and prove me wrong about some or all of these things, since I only experimented with it for a couple of days.  If not, for me these deficiencies in Relax NG *far* outweigh its benefits over XSD-there's really no comparison.  As much as I would like to see some aspects of XSD improved, unless someone can propose real alternatives for issues like the ones above, Relax NG is simply not an option for the type of data I work with.

As a postscript, let me say I'm very thankful that the tone of this thread has been gradually transitioning to actual constructive argument (at which this list is normally rather good) rather than the ridiculously juvenile mud-slinging that was predominant early last week-in particular, I've appreciated the level-headed input from Jim Tivy, Liam Quin, Ken Holman, and Michael Kay.  Unfortunately there are still some emotionally charged but pretty much meaningless (or worse, completely false) words being thrown around, but at least those aren't in the majority anymore.  In all good humor, I have to say that XML-dev is by far the most argumentative mailing list to which I've ever been a subscriber. :-)

Doug Glidden
Software Engineer
The Boeing Company
Douglass.A.Glidden@boeing.com

On Mon, Nov 16, 2009 at 4:26 AM, Jim Tivy <jimt@bluestream.com> wrote:
> Hearing this is truly music to my ears. The implication being that the XML
> spec would not even have to talk about "valid" or "validity" - thus we would
> not have to deal with this matter if inclusivity with respect to schema
> languages in the XML spec.

I think, removing references like "valid" or "validity" from XML spec
is a good idea. I think, this makes good distinct architectural
layerings like pure XML markup and document validity.

It seems, that most people argue that a conformant XML document has
ability to refer to user defined entities, and this can be specified
only in a DTD. I think, the current XML spec, makes DTDs an
inseparable part of XML spec is, because of the DTDs ability to
specify user defined entities, which (the entities) are part of say
the, XML 1.0 (and 1.1) spec.

To change the XML spec, to remove the concept of valid/validity and
being able to remove DTD s from the XML spec, I think will also
require us to remove the concepts of entities from the XML spec (I
think, something like a future XML 2.0 spec should only specify the
built in entities like & amp; etc and not allow users to specify user
defined entities, unless they use an external thing like DTD or a
future XSD (a 1.2 version, which may allow users to specify entities,
via some global declaration perhaps)).



-- 
Regards,
Mukul Gandhi

Hi Tim,
   I agree, that avoiding a Schema (both not designing at all, and not
using at run time with each XML instance) improves performance of XML
processing. XML Schema processing is certainly a processing overhead
for processing XML documents/messages.

But to my opinion, Schemas add tremendous value to XML processing,
when validation constraints need to be imposed. I think, this is a
basic notion of data integrity and a very  fundamental need for any
kind of data processing.

On Mon, Nov 16, 2009 at 4:02 AM, Tim Bray <Tim.Bray@sun.com> wrote:
> The textual flaw isn't that it doesn't mention XSD or RNG, the textual flaw is that it mentions *any* schema language.   A very high proportion of real-world XML processing is entirely free of anything schema-related.  The vast majority of the XML value proposition is delivered by schema-free well-formed XML.  Even in those apps that use a schema in their specification, the vast majority of run-time processing is schema-free.  One of the costliest common mistakes of XML app/language designers is putting too much importance on schemas.  The XML specification shouldn't be encouraging that mistake.
>
> My own vision of what XML.next ought to look like may be found at http://www.textuality.com/xml/xmlSW.html



-- 
Regards,
Mukul Gandhi

On Mon, Nov 16, 2009 at 1:00 AM, Michael Kay <mike@saxonica.com> wrote:
> Yes, it's ugly

I personally, won't say XSD is ugly. Just saw the meaning of ugly :)
"Inclined to anger or bad feelings with overtones of menace"
"Provoking horror"

I would not go to such an extent, while talking about XSD. But I
agree, that it is over engineered and over specified. But I guess,
over engineering/over specification is good for users.

> and very hard to use

Yes definitely, I agree that XSD has steep learning curve.

> and doesn't do a lot of things that people might like it to do.

Perhaps, this would get better with the 1.1 version.


-- 
Regards,
Mukul Gandhi

On Mon, Nov 16, 2009 at 10:33 PM, Mukul Gandhi <gandhi.mukul@gmail.com> wrote:
> On Mon, Nov 16, 2009 at 1:00 AM, Michael Kay <mike@saxonica.com> wrote:
>> Yes, it's ugly
>
> I personally, won't say XSD is ugly. Just saw the meaning of ugly :)
> "Inclined to anger or bad feelings with overtones of menace"
> "Provoking horror"


Sounds about right to me. :-)


-- 
Elliotte Rusty Harold
elharo@ibiblio.org

Len Bullard wrote:
> That's nuts and the opinion of a designer who writes code wonderfully but
> very few technical documents of any real complexity.
>   
As a subsystem gets more complex, we need addition levels of  'meta' to 
make the complexity tractable.

So we progress from plain documents to documents with namespaces to 
documents with content models and facets to documents with the content 
models and facets organized into types or patterns, and so on.  (The 
current stack of 'meta' currently ends just short of handling versions 
and variants well IMHO.)

So I think there is a requirement for agility: can a developer make an 
XML subsystem with just the right amount of 'meta' for their project?  
To oversimplify their positions:  Tim is keen that the developer who 
needs the least amount of 'meta' can have it (with DTD-less XML); 
Michael is keen on the person who needs quite a lot of meta (with typed 
information); Len is keen on the developer who needs a bit of 'meta' but 
not too much (the industrial document developer).  I guess I am 
concerned with developers who need kinds of 'meta' which cannot be 
productized or packaged well: bespoke 'meta'...

But the agility requirement is that it should be relatively easy to go 
from one level of 'meta' to the next, either up or down, with much less 
than commensurate effort.  I think moving from no-DTD XML to DTD-valid 
XML is that kind of low effort that allows agility. And moving from DTD 
to RELAX NG again is agile. But XSD works against agility: first because 
of its multiplication of concepts, but also because buying into the PSVI 
can represent a fundamental change in how you process and treat the 
data:  in practice, I think it is the choice between having a flow of 
documents (XML infoset), or having a DBMS (typed infoset or PSVI.)  The 
PSVI allows and promotes one system architecture (non-XML infoset) and 
works against others (XML infoset-based flows): so it is not just a 
neutral way of organizing XML data in any architecture, as you would 
want if you prized agility.

Cheers
Rick Jelliffe

Mukul Gandhi wrote:
> I don't think, there is anything wrong with the basic core/philosophy
> of XSD. I guess, some of users who don't like XSD generally, it's
> probably because of it's huge size, and steep learning curve.
>
>   
I do.

First, it does not encourage a schema-writer to make a user model, to 
know how to explain invalidity or versions/variants to humans. That 
information being missing from the chain, it entrenches the position of 
developers as gurus to whom the supplicant users must go to.   In other 
words, XSD actively *discourages* validation and human inspection. The 
PSVI certainly may have enough information for automated data exchange 
in fully debugged, in blind accept/reject gateways, or data-mapped 
systems; but I don't know that was actually a use case which was not 
already served adequately (by ODBC, CVS, CORBA, IDL, now JSON.)  In a 
decade where we supposedly appreciate Test-Driven Development more than 
ever before, we have a schema language that fights against this kind of 
use, in syntax, features, complexity and inexpressive autism. 

Second, it does not have enough hooks in it to allow traceability 
between the schema and business requirements.
Look at the question of why one element is supposed to be followed by 
another. There are several possible
reasons:

  1) it might be the instrinsic or typical order for renderings,
  2) by fixing an order, a schema-aware compressed form might save a bit 
(or is it half a bit?) - an entirely marginal reason to my way of 
thinking but I have heard it used,
  3) because derivation by extension is being used in XSD 1.0 and so 
suffixation was the only possibility,
  4) to mirror the order of some data which is transformed into this,
  5) err no reason, it just seems neater

XSD is not neutral about traceability: it is antagonistic to it.   Now I 
agree that XSD 1.1 has lots of additional things which help mechanical 
power (at the expense of a multiplication of concepts): the open schemas 
idea and so on are things I (and, in particular, Roger Costello) have 
long raised, so I am sure many people will benefit from the XSD 1.1 
changes (just as people benefit from lipstick on a pig?)  But still some 
of them are hacks, bits tacked on (like the cart in the Imaginarium of 
Dr Parnassus) without addressing what I think are the more fundamental 
issues such as disconnection from humans and disconnection from 
requirements. You would expect that kind of disconnect in a technology 
invented in the 1970s or 1980s, but not one for leading us into the 2010s.

So it is exactly in XSD's imagined core philosophy, detectable by how 
XSD has been used or implemented and where it thrived or failed, that I 
think the problem lies.  XSD is not a technology that encourages 
technocrats to empower the just-folks or non-technical management, in 
the way that HTML, XML and, I really hope, Schematron do. That puts it 
in the class of being things that are part of the problem rather than 
part of the solution.

Cheers
Rick Jelliffe


P.S. I don't know that this analogy will add to the clarity, so I have 
put it down here. Years ago there was a cartoon strip in the newspapers 
called Li'l Abner. It is rarely spoken of now, because of its depictions 
of US minorities, I gather. But there was a little animal called the 
Schmoo: in 1949 it was more popular than Mickey Mouse (see 
http://en.wikipedia.org/wiki/Shmoo For a pic see 
http://deniskitchen.com/Merchant2/merchant.mvc?Screen=PROD&Product_Code=BP_shmoopin 
).  It loves to be eaten (unlike the Magic Pudding, it gets used up.)  

The XSD standard is not a standard made by Schmoos. It centralizes the 
power of developers, rather than allowing developers to localize their 
role.

Elliotte Rusty Harold wrote:
> On Sat, Nov 14, 2009 at 12:49 PM, Simon St.Laurent
> <simonstl@simonstl.com> wrote
>   
>> A this point, personally, I'd drop coverage of XLink and XPointer.  A
>> paragraph seems reasonable.
>>
>> XInclude does get use in back-end processing, so I'd leave it in, most
>> likely - at least as a sidebar in the entity processing section.  It doesn't
>> require a huge amount of space to explain in any case.
>>     
>
> Sounds about right, though XInclude references XPointer so it's hard
> to really say.
 
I think this is a bit arse-ended: even apart from the dubious idea that 
we should consider a standard successful only if it sells books or 
achieves ubiquity in platforms (rather than, say, a standard being 
successful if it can be used to solve some problems well.)

Let me put it like this: one of the original goals of XML was "The 
number of optional features in XML is to be kept to the absolute 
minimum, ideally zero." This was because the highly parameterized 
approach that SGML took was not appropriate, for various reasons: one of 
which was that if there were optional features, you could not be 
guaranteed that the recipient could handle your document.

And so what happened after that?  A stream of piddly little standards 
for optional things: XBase, XInclude, xml:id.  And what is the result? 
It is impractical to rely on them: or, at least, they don't relieve the 
developer of any decision making or programming, since the developer has 
to enable them or program for them on a case-by-case basis.

Cheers
Rick Jelliffe

2009/11/16 Elliotte Rusty Harold <elharo@ibiblio.org>:
> On Mon, Nov 16, 2009 at 10:49 AM, Peter Hunsberger
> <peter.hunsberger@gmail.com> wrote:
>
>> So what's the alternative?  There are lots of things that people
>> sometimes want to do, but not always.  Having optional standards that
>> cover these areas is probably a good thing;
>
> No, the alternative is to build them into the specs and require
> parsers to support them, but not require documents to use them. For
> example, consider attributes. You don't have to use them in XML, but
> if you do use them you know they're supported. xml:id, xml:base, and a
> few other things (though probably not XInclude) should be baked in,
> not bolted on. Watch this space. :-)

I thought we were headed towards a slimmer, lightweight, dtd-less
XML... not adding more?





-- 
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

Hi,

Have a look at this book:
http://oreilly.com/catalog/9780596009748

I have read a long time ago the first edition that was covering that 
topic very well; I don't know about the second edition, but the chapter 
7 since to cover almost what you need; you might find snippet codes 
somewhere at oreilly.com

David A. Lee wrote:
> I have a project where I need to take XML and format it as ascii text.
> Sounds simple right ? But alas the requirements are subtle.  It seems 
> people dont do text formatting much anymore.
> I fondly remember the days of nroff where I could do good text 
> formatting and achieve things like word splitting (hyphenating),
> justification etc on text content.  In particular a challenge is to 
> support multi column tables.  Thats right, text spanning 5 columns,
> that needs to wrap in each column.   Oh and do that with 40 column line 
> widths. Piece of cake.
> Oh and its a Variable width text font ... ha !
> 
> What I tried so far which almost works sorta, just enough to not work 
> well enough :)
> Translate the XML to HTML with XSLT then I downloaded one of the free 
> "HTML to Text" tools which are prevalent on the net.
> It sorta did work, except for the tables.   But I need something that 
> can work in a production environment.   Open Source is prefered,
> but if it does an excellent job then a commercial product could be 
> justified (pun intended).
> 
> My next attempt will to try Apache FOP.    I looked at RenderX but they 
> dont claim text output.
> Apache FOP claims a text output but with caveots, the claim is its not 
> robust code with poor results.
> Maybe they are being kind and it really works great.   I will try.
> 
> Any other suggestions ?   I can always 'roll my own' but this project is 
> not on the budget yet and were trying to see if we can do something with 
> minimal work.
> Appreciated,
> 
> thanks,
> 
> 
> 
> 


-- 
Cordialement,

               ///
              (. .)
  --------ooO--(_)--Ooo--------
|      Philippe Poulard       |
  -----------------------------
  http://reflex.gforge.inria.fr/
        Have the RefleX !

On Mon, Nov 16, 2009 at 11:59 AM, Peter Hunsberger
<peter.hunsberger@gmail.com> wrote:

> However, optional but part of the base spec, vs. optional and part of
> a different spec really makes no difference as far as I can tell,
> other than to make the base spec larger and harder to get agreement
> on?

That's why you don't want anything in the base spec to be optional.
The problem is not so much base spec or supplementary spec, but
optional vs. required. No additions to XML post-namespaces (xml:base,
xml:id, schemas, infoset, XML 1.1, XInclude, etc.) have achieved
ubiquity and reliable support. People have built solid apps on top of
XML (XPath, XSLT, XQuery, XHTML, Atom, etc.) but the core of XML has
been extremely resistant to evolution.

-- 
Elliotte Rusty Harold
elharo@ibiblio.org

On Mon, Nov 16, 2009 at 11:38 AM, Andrew Welch <andrew.j.welch@gmail.com> wrote:

> I thought we were headed towards a slimmer, lightweight, dtd-less
> XML... not adding more?

yes to the first, no to the second. :-)

-- 
Elliotte Rusty Harold
elharo@ibiblio.org

On Mon, Nov 16, 2009 at 10:49 AM, Peter Hunsberger
<peter.hunsberger@gmail.com> wrote:

> So what's the alternative?  There are lots of things that people
> sometimes want to do, but not always.  Having optional standards that
> cover these areas is probably a good thing;

No, the alternative is to build them into the specs and require
parsers to support them, but not require documents to use them. For
example, consider attributes. You don't have to use them in XML, but
if you do use them you know they're supported. xml:id, xml:base, and a
few other things (though probably not XInclude) should be baked in,
not bolted on. Watch this space. :-)

-- 
Elliotte Rusty Harold
elharo@ibiblio.org

On Mon, Nov 16, 2009 at 10:17 AM, Elliotte Rusty Harold
<elharo@ibiblio.org> wrote:
> On Mon, Nov 16, 2009 at 10:49 AM, Peter Hunsberger
> <peter.hunsberger@gmail.com> wrote:
>
>> So what's the alternative?  There are lots of things that people
>> sometimes want to do, but not always.  Having optional standards that
>> cover these areas is probably a good thing;
>
> No, the alternative is to build them into the specs and require
> parsers to support them, but not require documents to use them. For
> example, consider attributes. You don't have to use them in XML, but
> if you do use them you know they're supported. xml:id, xml:base, and a
> few other things (though probably not XInclude) should be baked in,
> not bolted on. Watch this space. :-)
>

Well that sure doesn't jive with your original starting position:

> Let me put it like this: one of the original goals of XML was "The number of
> optional features in XML is to be kept to the absolute minimum, ideally
> zero."

However, optional but part of the base spec, vs. optional and part of
a different spec really makes no difference as far as I can tell,
other than to make the base spec larger and harder to get agreement
on?

-- 
Peter Hunsberger

On Sun, Nov 15, 2009 at 11:48 PM, Rick Jelliffe
<rjelliffe@allette.com.au> wrote:
> Elliotte Rusty Harold wrote:
>>
>
> Let me put it like this: one of the original goals of XML was "The number of
> optional features in XML is to be kept to the absolute minimum, ideally
> zero." This was because the highly parameterized approach that SGML took was
> not appropriate, for various reasons: one of which was that if there were
> optional features, you could not be guaranteed that the recipient could
> handle your document.
>
> And so what happened after that?  A stream of piddly little standards for
> optional things: XBase, XInclude, xml:id.  And what is the result? It is
> impractical to rely on them: or, at least, they don't relieve the developer
> of any decision making or programming, since the developer has to enable
> them or program for them on a case-by-case basis.
>
So what's the alternative?  There are lots of things that people
sometimes want to do, but not always.  Having optional standards that
cover these areas is probably a good thing; at least developers can
pick up the standard and run with it (if they so choose) instead of
reinventing things from scratch.  The only thing I would ask is that
they be done in such a way that anyone that doesn't know about them
have things break when they encounter them.  For the most part things
like XInclude can be ignored if you don't care about them?

-- 
Peter Hunsberger

On Mon, Nov 16, 2009 at 4:06 AM, Michael Ludwig <milu71@gmx.de> wrote:
> So yes, it is XML, but no, it is not easily readable, or immediately
> useful. And that's why I fail to see the reason why XML syntax was
> desired in the first place.

Personally speaking, I think having an XML syntax for XSD was a good
choice. Trying to add features like simple and complex types, type
derivation etc may get quite clumsy, if they were tried to be defined
in a flat DTD like syntax.

Having a XML syntax for XSD, makes easy comprehension for people to
understand a particular schema, as it's hierarchical. I personally
find hierarchical specification for most of the things, easy to
understand.

On the other hand, I also find flat (line oriented formats)
specifications like EBNF to be quite successful. Of course, DTDs were
also successful as a starter to XML Schema technology :)



-- 
Regards,
Mukul Gandhi

On Sun, Nov 15, 2009 at 5:52 PM, Liam Quin <liam@w3.org> wrote:

> But the fairy dust has scattered, the hype has moved on, and few
> organizations today would want to pay money to participate in
> XML 2.0 work, I fear, even if we could get consensus on what such
> a beast should look like.

[expletive deleted] consensus. [expletive deleted] organizations that pay to play.

The only way XML 2.0 could ever work is is if a few smart, like-minded
individuals get together and build something useful. If it's a
noticeable improvement on existing technology that offers significant
practical benefits to real customers (as opposed to  the completely
theoretical benefits of XML 1.1) the big companies can play catch up
later, just like they did with JSON; just like they did with XML 1.0.
Just like they're doing now with HTML 5.

"Never doubt that a small group of thoughtful, committed people can
change the world. Indeed, it is the only thing that ever has." --
Margaret Mead

That said, I'm not sure we could make XML 2.0 better enough to justify
the cost of switching. XML 1.0 is still pretty damn good. 5% better
wouldn't do it. It really needs to be infinitely better: i.e. move
some projects that are  practically impossible today into the realm of
the plausible. After 12 years, and listening to a lot of good ideas
from multiple folks, I think I know how to make XML 50% better, maybe
even 100% better; but I'm not sure I know how to make it infinitely
better. But one of these days someone will figure out how to. I
suspect it won't be the W3C that does though.

-- 
Elliotte Rusty Harold
elharo@ibiblio.org

Tim Bray schrieb am 14.11.2009 um 12:49:06 (-0800):

> The fact that  DTDs were baked into XML 1.0
> is a historical accident.

Given the role SGML seems to have played when XML was drafted, the
inclusion of DTDs rather appears to be a historical inevitability.
Wasn't DTD too useful to be left out?

To continue your oven analogy, wouldn't XML without DTD have been
like raisin cake without raisins?

Or maybe you mean it would have been wise to leave out only those parts
that constrain the structure of the document in terms of elements and
attributes, but not other parts, such as entity declarations?

Also, agreeing to enhance DTD instead of inventing XSD, allowing the
user to define and assign types, might have been an alternative.

I've often heard or read that people wanted an XML notation for the
schema language and were dissatisfied with DTD for this very reason.
However, if you try to process XSD at the XML level, let's say in order
to produce an HTML page to document the notoriously unreadable XSD
syntax, you eventually realize that the language is pretty complicated
and you need a normalized representation of its syntax first. (Which
Saxon provides in the SA and Enterprise versions.)

So yes, it is XML, but no, it is not easily readable, or immediately
useful. And that's why I fail to see the reason why XML syntax was
desired in the first place.

-- 
Michael Ludwig

On Sun, Nov 15, 2009 at 09:44:56PM +0530, Mukul Gandhi wrote:
> Thanks for sharing the historical perspective. Was this an important
> architectural decision, while writing the original XML spec (i.e,
> every DTD-valid document must also be a valid SGML document) - or to
> say, was this a requirement/prerequisite for the original XML spec?
Yes. XML was originally called Web-SGML.  The entire purpose of XML,
originally, was to facilitate the use of (a subset of) SGML on the Web,
and, specifically (at first) in Web-browser plugins.

> Perhaps, an XML 2.0 activity can rearchitect the XML specs, along the
> lines you outlined earlier.

Possibly, but I don't see strong enough market benefits to
justify the lostt in compatibility and interoperability.

The XML promise -- any XML processor can read any XML document --
is an oversimplification, but, even if you tone it down to
take "can read" to mean "is entitled to read", the extremely
high interoperability story of XML is a large part of its success.

Another important success story is of course XHTML -- which
relies on a DTD to define the HTML chracter entities.

Liam

-- 
Liam Quin, W3C XML Activity Lead, http://www.w3.org/People/Quin/
http://www.holoweb.net/~liam/ * http://www.fromoldbooks.org/

Hi Liam,

On Sun, Nov 15, 2009 at 5:47 PM, Liam Quin <liam@w3.org> wrote:
> Just to be clear, XML was designed so that every DTD-valid document
> was also a valid SGML document; we had some 10 years experience
> with SGML deployment, and that's where DTDs came from.

Thanks for sharing the historical perspective. Was this an important
architectural decision, while writing the original XML spec (i.e,
every DTD-valid document must also be a valid SGML document) - or to
say, was this a requirement/prerequisite for the original XML spec? If
yes, is it important to preserve it now as well (i.e, the SGML and XML
DTD connection)? Are there enough market drivers for this now?

Perhaps, an XML 2.0 activity can rearchitect the XML specs, along the
lines you outlined earlier.


-- 
Regards,
Mukul Gandhi

Hi Ken,
   I think, the subject of this thread was different, than what this
debate is turning out to be.

My point was not to say, which Schema technology is better. I was only
trying to point to this one point in XML spec:
"An XML document is valid if it's valid according to a DTD", and my
proposed changes to XML specs clearing up this confusion.

Tim Bray wrote earlier in this thread:
Mention of DTD into XML 1.0 spec, was historic accident. I think, Tim
was quite modest to support my view point. But I think, he recognizes
some truth with this textual flaw in the spec, in the current
scenario.

Liam Quin wrote:
An architectural breakup of XML specs can also be a viable option.

I think, Jim Tivy also supported my view points. Even I saw some
support from Mike Kay, but not entirely.

I think, rest of people in this thread spoke against this idea.

I've just tabled my ideas in this thread, and I am sure if it makes
sense something could be done about this issue, else this idea is
bound to be discarded :)

On Sun, Nov 15, 2009 at 7:01 PM, G. Ken Holman
<gkholman@cranesoftwrights.com> wrote:
> I cited it only as an example.  Forgive me if my comments came across as
> otherwise.
>
>> I think, even a correct implementation of
>> say, xs:redefine on one or two processors is good enough.
>
> But are there two implementations that are the same, and if they are the
> same, are they the "correct implementation" you cite?  I believe this is the
> root of the issue:  it isn't that vendors have implemented the specification
> incorrectly, it is that the specification is unclear enough that each vendor
> believes they have implemented it correctly yet end up with different
> results.  There are no "bugs" that can be identified and repaired in each
> vendor's incompatible work because there is no agreement on the
> interpretation of the specification as written.
>
> The validation semantics for W3C schema are written in prose.
>
> The validation semantics for RELAX-NG are written in formal unambiguous
> notation, guiding all implementers to a formally correct implementation if
> they properly implement the documented semantics.  Of course they can
> implement bugs, but because of the formalisms, the bugs can be identified as
> such without debate.
>
>> Sometimes,
>> vendors create differences in implementations to differentiate (I am
>> not really sure though, if that's true. At least the base standard
>> should be implementable).
>
> And that is my very point:  yes, it should be written so as to be
> implementable by all without ambiguity.  Practice has revealed this is not
> the case for W3C Schema.
>
>> I am not trying to be getting into a mud sludge game between computer
>> languages, or to express sarcasm to any XML validation language. I
>> appreciate, efforts of anybody taking pains to design anything like
>> these languages, and implement them.
>
> Indeed.  And please forgive me if my comments come across as sarcasm of W3C
> schema, as I have been trying very hard to be objective so as to illustrate
> the concerns with concrete examples.  It is not my intention to obfuscate
> the issues with mud, but to clarify the issues by citing identifiable
> sources of concerns with the technology.  Those in this debate who have not
> supported W3C schema have been speaking up in the interests of all XML users
> who may have, themselves, been misguided regarding the technology
> (intentionally or unintentionally).
>
> . . . . . . . . . . . . . Ken



-- 
Regards,
Mukul Gandhi

Do whatever floats your boat, fella, but the reason we had them was to make
complex production of complex documents easy to do and easy to configure by
people who generally did not program for a living.

Pretty much everything else talked about in this thread was tacked on after
that.  It isn't a matter of spec goodness but historical fact.

Be seeing you.

len

-----Original Message-----
From: Jim Tivy [mailto:jimt@bluestream.com] 
Sent: Sunday, November 15, 2009 6:31 PM
To: 'Len Bullard'; Tim.Bray@Sun.COM; 'Mukul Gandhi'
Cc: 'G. Ken Holman'; xml-dev@lists.xml.org
Subject: RE:  XML spec and XSD

Len

I think schemaless processing of XML is **ONE** reason to get rid of the DTD
(element,attr decls) from the XML spec.  There are likely a set of other
reasons - another reason being a concern of being inclusive of all schema
languages with respect to XML validation - eg: factor out the notion of
validity from core XML spec.

I am not sure how to deal with your modelling of designers/programmers or
"Facts are" statements - makes my brains fall out - however I am not sure
you want those addressed.

However, I think it **valid** that you pointed out the importance of
validation.

Jim

-----Original Message-----
From: Len Bullard [mailto:cbullard@hiwaay.net] 
Sent: Sunday, November 15, 2009 3:49 PM
To: Tim.Bray@Sun.COM; 'Mukul Gandhi'
Cc: 'G. Ken Holman'; xml-dev@lists.xml.org
Subject: RE:  XML spec and XSD

That's nuts and the opinion of a designer who writes code wonderfully but
very few technical documents of any real complexity.

A DTD in a production system with many writers attempting to remain within
the constraints of a common document design found DTDs to be quite useful.
Otherwise markup was crap layered into already complex content.   The
programmer viewpoint of markup is but one viewpoint and can't be used to
post facto justify well-formedness as the basis of XML goodness.  

Facts are that now the usefulness of XML itself is questioned in many
quarters but at the time when SGML was used as the basis of complex
documentation systems that emphasized the accuracy of technical writing over
database design or streaming the DTD was crucial.

len


From: Tim.Bray@Sun.COM [mailto:Tim.Bray@Sun.COM] 

The textual flaw isn't that it doesn't mention XSD or RNG, the textual flaw
is that it mentions *any* schema language.   A very high proportion of
real-world XML processing is entirely free of anything schema-related.  The
vast majority of the XML value proposition is delivered by schema-free
well-formed XML.  Even in those apps that use a schema in their
specification, the vast majority of run-time processing is schema-free.  One
of the costliest common mistakes of XML app/language designers is putting
too much importance on schemas.  The XML specification shouldn't be
encouraging that mistake.

My own vision of what XML.next ought to look like may be found at
http://www.textuality.com/xml/xmlSW.html



_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php




_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

>    Saying that something like, xs:redefine isn't implemented 
> consistently across different XSD processors, IMHO doesn't 
> justify condemning XSD completely. I think, even a correct 
> implementation of say, xs:redefine on one or two processors 
> is good enough. 

In the case of xs:redefine the problem is that the spec is underspecified,
so two implementations can do different things and both be consistent with
the spec: it's not the case that some of the implementations are
"incorrect". This is certainly one of the many technical weaknesses of the
XSD specification. It was of course an attempt to reproduce one of the ways
in which DTDs are used to define variant and customised content models.
Henry Thompson has produced some guidelines for use of xs:redefine which, if
you follow them, will give you interoperable behaviour and will probably
meet the requirement.

I would expect good experienced knowledgeable people like Tim Bray and Rick
Jelliffe to say "There are many weaknesses in XSD and we know today how many
of these things could have been done better" - which is perfectly true. I
would not expect them to say "XSD is a complete failure". I simply don't
think one can say that of a technology which is so widely and successfully
deployed, and which has some technical characteristics (notable the
attribution of types to elements and attributes) which are currently unique
to that technology.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

> XSD is a failure by any sane technical measure.

Yes, it's ugly, and very hard to use, and doesn't do a lot of things that
people might like it to do.

But you could say all that of Windows, and Windows is not exactly a failure,
though you could argue that in a saner world it would be.

There are other measures of success that are not entirely insane:

(a) it meets all the requirements that were specified at the start of the
project.

(b) it has half a dozen decent implementations, with a level of
interoperability that's not unreasonable by computer industry standards
(much better than SQL, for example).

(c) it's used by large numbers of people, many of whom have a choice.

(d) it does most of the things that most of those people need it to do.

(e) it underpins a large number of industry standards that enable effective
interchange of data around the world, creating enormous value by oiling the
wheels of commerce.

If it were really such a total failure, wouldn't the spec be gathering dust,
rather than being actively discussed on this list?

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 
 

On Sun, Nov 15, 2009 at 02:32:37PM -0800, Tim Bray wrote:
> The textual flaw isn't that [the XML Specification] doesn't mention XSD or
> RNG, the textual flaw is that it mentions *any* schema language.

Agreed.

But the fairy dust has scattered, the hype has moved on, and few
organizations today would want to pay money to participate in
XML 2.0 work, I fear, even if we could get consensus on what such
a beast should look like.

Liam


-- 
Liam Quin, W3C XML Activity Lead, http://www.w3.org/People/Quin/
http://www.holoweb.net/~liam/ * http://www.fromoldbooks.org/

That's nuts and the opinion of a designer who writes code wonderfully but
very few technical documents of any real complexity.

A DTD in a production system with many writers attempting to remain within
the constraints of a common document design found DTDs to be quite useful.
Otherwise markup was crap layered into already complex content.   The
programmer viewpoint of markup is but one viewpoint and can't be used to
post facto justify well-formedness as the basis of XML goodness.  

Facts are that now the usefulness of XML itself is questioned in many
quarters but at the time when SGML was used as the basis of complex
documentation systems that emphasized the accuracy of technical writing over
database design or streaming the DTD was crucial.

len


From: Tim.Bray@Sun.COM [mailto:Tim.Bray@Sun.COM] 

The textual flaw isn't that it doesn't mention XSD or RNG, the textual flaw
is that it mentions *any* schema language.   A very high proportion of
real-world XML processing is entirely free of anything schema-related.  The
vast majority of the XML value proposition is delivered by schema-free
well-formed XML.  Even in those apps that use a schema in their
specification, the vast majority of run-time processing is schema-free.  One
of the costliest common mistakes of XML app/language designers is putting
too much importance on schemas.  The XML specification shouldn't be
encouraging that mistake.

My own vision of what XML.next ought to look like may be found at
http://www.textuality.com/xml/xmlSW.html

Hi Ken,
   Saying that something like, xs:redefine isn't implemented
consistently across different XSD processors, IMHO doesn't justify
condemning XSD completely. I think, even a correct implementation of
say, xs:redefine on one or two processors is good enough. Sometimes,
vendors create differences in implementations to differentiate (I am
not really sure though, if that's true. At least the base standard
should be implementable).

For example, we use different web browsers (IE, Mozilla, Chrome or
whatever) without even caring how well all of these products implement
HTML or JavaScript. We just ignore, minor implementation faults. But
it's also true, that certain vendors take great care to meet the
specifications.

My experience has been, to see that most of the reputable products are
consitent in meeting standards. Some minor differences may happen, due
to the differences in how we interpret the spec, and also the desire
of vendors to differentiate.

As an analogy to this issue, for example I think we all love XSLT 2. I
think, we only have very few XSLT 2 implementations presently. Does
anybody bother to see, if say anything like the XSLT 2 instruction
xsl:for-each-group or any of other XSLT 2 instructions work
consistently across multiple XSLT 2 engines? Dispite for the ignorance
with these issues, people want to program with XSLT 2.

I am not trying to be getting into a mud sludge game between computer
languages, or to express sarcasm to any XML validation language. I
appreciate, efforts of anybody taking pains to design anything like
these languages, and implement them.

On Sun, Nov 15, 2009 at 8:25 AM, G. Ken Holman
<gkholman@cranesoftwrights.com> wrote:
> I wouldn't agree with that.  I also side with Tim and Rick and others who
> grudgingly find they have to work with XSD (usually to meet customer
> requests; personally I migrated from DTD to RELAX-NG and never embraced
> XSD), and for the same reason that Tim cites above: technical issues.  Size
> and complexity are not, necessarily, technical faults.
>
>> From functional point of view, I don't think XSD doesn't work.
>
> There are a number of areas that are problematic.  One example is no two
> vendors have implemented "redefine" the same way.  I had researched this to
> meet specific requirements identified for the UBL project and was burned by
> assuming the way one processor implemented redefine was the way others
> implemented it.
>
>> So many numerous XML applications currently used XSD.
>
> Popularity is no measure of functionality.  Vendors who are pushing XSD
> products would likely tell their clients there are no other choices for
> schema languages.  It is my assessment that vendors and users ignore or are
> unaware of other sources of XML-based technologies, such as ISO/IEC JTC 1/SC
> 34 which created the ISO/IEC 19757 Document Schema Definition Languages
> (DSDL) that includes RELAX-NG, Schematron and NVDL among other XML
> specifications.
>
>> I don't think, there is anything wrong with the basic core/philosophy
>> of XSD.
>
> An example of a core fault is that W3C Schema is not closed under union.  I
> cannot programmatically express the union of two XSD schemas to create a
> third XSD schema that validates instances of the other two.  I might be able
> to by hand, but not programmatically.
>
> This is not academic:  consider an XML document where in two different
> contexts there is an element <x> with different content models.  I then
> write a query that assembles all of the documents <x> elements under a
> single parent result element.  I have to hand-craft the schema that
> validates the result of the query (if it is possible and not an ambiguous
> content model), as I cannot express in XSD-speak two sibling elements with
> different content models.  If I could simply express the union of the two
> original <x> content models as the content model for the query result <x>,
> then I wouldn't have to resort to hand-crafting.
>
> There may be other "core" faults, but that is one that has been a problem
> for me.
>
> In RELAX-NG compact syntax, if I have a grammar "a" and a grammar "b" (be it
> document level, element level or any level of a grammar; at the element
> level both grammars might express different content models for the element
> <x>), the union of these two grammars is expressed as "a | b".  No other
> work is necessary.  An instance of <x> will validate true if it passes the
> constraints expressed by either grammar.
>
>> I guess, some of users who don't like XSD generally, it's
>> probably because of it's huge size, and steep learning curve.
>
> But cognoscenti such as Rick Jelliffe, Tim Bray and many others would not
> find a specification's huge size and steep learning curve a fault if,
> technically, the end result was something that worked well.
>
> It might indeed be a barrier for new people to W3C Schema, but my
> recollection is that negative opinions in this debate have not come from
> representatives of that user group.
>
>> I believe, XSD also get's functionally better with the upcoming 1.1
>> release.
>
> Probably the functionality will be better, yes, but will it address
> everything?  We won't know until it is released, and I suspect some legacy
> 1.0 issues will dog all dot-releases of XSD.
>
> I hope this is considered helpful to the discussion.
>
> . . . . . . . . . . . Ken



-- 
Regards,
Mukul Gandhi

Len

I think schemaless processing of XML is **ONE** reason to get rid of the DTD
(element,attr decls) from the XML spec.  There are likely a set of other
reasons - another reason being a concern of being inclusive of all schema
languages with respect to XML validation - eg: factor out the notion of
validity from core XML spec.

I am not sure how to deal with your modelling of designers/programmers or
"Facts are" statements - makes my brains fall out - however I am not sure
you want those addressed.

However, I think it **valid** that you pointed out the importance of
validation.

Jim

-----Original Message-----
From: Len Bullard [mailto:cbullard@hiwaay.net] 
Sent: Sunday, November 15, 2009 3:49 PM
To: Tim.Bray@Sun.COM; 'Mukul Gandhi'
Cc: 'G. Ken Holman'; xml-dev@lists.xml.org
Subject: RE:  XML spec and XSD

That's nuts and the opinion of a designer who writes code wonderfully but
very few technical documents of any real complexity.

A DTD in a production system with many writers attempting to remain within
the constraints of a common document design found DTDs to be quite useful.
Otherwise markup was crap layered into already complex content.   The
programmer viewpoint of markup is but one viewpoint and can't be used to
post facto justify well-formedness as the basis of XML goodness.  

Facts are that now the usefulness of XML itself is questioned in many
quarters but at the time when SGML was used as the basis of complex
documentation systems that emphasized the accuracy of technical writing over
database design or streaming the DTD was crucial.

len


From: Tim.Bray@Sun.COM [mailto:Tim.Bray@Sun.COM] 

The textual flaw isn't that it doesn't mention XSD or RNG, the textual flaw
is that it mentions *any* schema language.   A very high proportion of
real-world XML processing is entirely free of anything schema-related.  The
vast majority of the XML value proposition is delivered by schema-free
well-formed XML.  Even in those apps that use a schema in their
specification, the vast majority of run-time processing is schema-free.  One
of the costliest common mistakes of XML app/language designers is putting
too much importance on schemas.  The XML specification shouldn't be
encouraging that mistake.

My own vision of what XML.next ought to look like may be found at
http://www.textuality.com/xml/xmlSW.html



_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

Hearing this is truly music to my ears. The implication being that the XML
spec would not even have to talk about "valid" or "validity" - thus we would
not have to deal with this matter if inclusivity with respect to schema
languages in the XML spec.

-----Original Message-----
From: Tim.Bray@Sun.COM [mailto:Tim.Bray@Sun.COM] 
Sent: Sunday, November 15, 2009 2:33 PM
To: Mukul Gandhi
Cc: G. Ken Holman; xml-dev@lists.xml.org
Subject: Re:  XML spec and XSD

On 2009-11-15, at 8:01 AM, Mukul Gandhi wrote:

> Tim Bray wrote earlier in this thread:
> Mention of DTD into XML 1.0 spec, was historic accident. I think, Tim
> was quite modest to support my view point. But I think, he recognizes
> some truth with this textual flaw in the spec, in the current
> scenario.

The textual flaw isn't that it doesn't mention XSD or RNG, the textual flaw
is that it mentions *any* schema language.   A very high proportion of
real-world XML processing is entirely free of anything schema-related.  The
vast majority of the XML value proposition is delivered by schema-free
well-formed XML.  Even in those apps that use a schema in their
specification, the vast majority of run-time processing is schema-free.  One
of the costliest common mistakes of XML app/language designers is putting
too much importance on schemas.  The XML specification shouldn't be
encouraging that mistake.

My own vision of what XML.next ought to look like may be found at
http://www.textuality.com/xml/xmlSW.html

 -Tim

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

On 2009-11-15, at 8:01 AM, Mukul Gandhi wrote:

> Tim Bray wrote earlier in this thread:
> Mention of DTD into XML 1.0 spec, was historic accident. I think, Tim
> was quite modest to support my view point. But I think, he recognizes
> some truth with this textual flaw in the spec, in the current
> scenario.

The textual flaw isn't that it doesn't mention XSD or RNG, the textual flaw is that it mentions *any* schema language.   A very high proportion of real-world XML processing is entirely free of anything schema-related.  The vast majority of the XML value proposition is delivered by schema-free well-formed XML.  Even in those apps that use a schema in their specification, the vast majority of run-time processing is schema-free.  One of the costliest common mistakes of XML app/language designers is putting too much importance on schemas.  The XML specification shouldn't be encouraging that mistake.

My own vision of what XML.next ought to look like may be found at http://www.textuality.com/xml/xmlSW.html

 -Tim

Hi Liam,

On Sun, Nov 15, 2009 at 6:19 AM, Liam Quin <liam@w3.org> wrote:
> Speaking as W3C XML Activity Lead :-) I don't see a reason to
> give XSD special status in that way.

No problems :) That's fair enough for me.

> W3C XSD has a lot of uses, and works well if you are also
> using XSLT 2, XQuery, Web Services or other specifications
> where the XSD support is well-integrated.  This doesn't in
> any way preclude using RelaxNG, DTDs, Schematron, or any
> other sort of validation, and it's not really the place of
> the XML specification to give a list of other technologies
> that are used with it or build on it.

I agree with this. I was actually not forcing a point of view, to
really specify something like XSD, RelaxNG or other XML validation
technology into the XML spec. I just find specification of a statement
like, "an XML document is valid, if it's valid according to a DTD" in
the XML spec, confusing given the existence now, of other XML
validation technologies like XSD et al.

As you have suggested below, refactoring the XML spec and separating
DTD into a separate document looks a reasonable solution to this
problem, and would dispel the confusion which user's seemingly have
when they read the terms "validation" and DTD in the most recent XML
specs.

> As Tim and others said, DTDs are in the XML specification for
> historical reasons.

I appreciate this. It's good that, something like this existed in the
early days of XML (that was a nice innovation, in the absence of
anything else, as we speak of XSD and other validation technologies,
as of today). But now, the XML validation technologies, are in a
different paradigm and I guess, the XML and related specs should
explain the relationship between XML markup and it's validation,
reflecting the current world we are living in :) IMHO, it's fine if we
rearchitect the XML specs, as you suggested to achieve this clarity
for users.

> If we had the resources, I can certainly
> imagine refactoring XML 1.0 to put DTDs into a separate document,
> and to bring into that document namespaces and xml:base and
> maybe others. But the work of doing that is much greater than
> it sounds -- making sure the text retains the same meaning,
> and even that documents referring to specific productoins in the
> EBNF grammar are not broken as a result.

I hope this can happen. Perhaps, a work to create drafts for such a
architectural breakup of XML specs, can begin :)

> So I don't expect much enthusiasm for making a new edition of
> XML to make the Informative References include XSD, but
> please don't take that as a negative statement about XSD :-)

I appreciate your speaking about this topic, and supporting few of my points :)



-- 
Regards,
Mukul Gandhi

On Sun, Nov 15, 2009 at 2:19 AM, Tim Bray <Tim.Bray@sun.com> wrote:
> The fact that DTDs were
> baked into XML 1.0 is a historical accident.

I think, this statement affirms my point to some extent :)


-- 
Regards,
Mukul Gandhi

At 2009-11-15 17:15 +0530, Mukul Gandhi wrote:
>    Saying that something like, xs:redefine isn't implemented
>consistently across different XSD processors, IMHO doesn't justify
>condemning XSD completely.

I cited it only as an example.  Forgive me if my comments came across 
as otherwise.

>I think, even a correct implementation of
>say, xs:redefine on one or two processors is good enough.

But are there two implementations that are the same, and if they are 
the same, are they the "correct implementation" you cite?  I believe 
this is the root of the issue:  it isn't that vendors have 
implemented the specification incorrectly, it is that the 
specification is unclear enough that each vendor believes they have 
implemented it correctly yet end up with different results.  There 
are no "bugs" that can be identified and repaired in each vendor's 
incompatible work because there is no agreement on the interpretation 
of the specification as written.

The validation semantics for W3C schema are written in prose.

The validation semantics for RELAX-NG are written in formal 
unambiguous notation, guiding all implementers to a formally correct 
implementation if they properly implement the documented 
semantics.  Of course they can implement bugs, but because of the 
formalisms, the bugs can be identified as such without debate.

>Sometimes,
>vendors create differences in implementations to differentiate (I am
>not really sure though, if that's true. At least the base standard
>should be implementable).

And that is my very point:  yes, it should be written so as to be 
implementable by all without ambiguity.  Practice has revealed this 
is not the case for W3C Schema.

>I am not trying to be getting into a mud sludge game between computer
>languages, or to express sarcasm to any XML validation language. I
>appreciate, efforts of anybody taking pains to design anything like
>these languages, and implement them.

Indeed.  And please forgive me if my comments come across as sarcasm 
of W3C schema, as I have been trying very hard to be objective so as 
to illustrate the concerns with concrete examples.  It is not my 
intention to obfuscate the issues with mud, but to clarify the issues 
by citing identifiable sources of concerns with the 
technology.  Those in this debate who have not supported W3C schema 
have been speaking up in the interests of all XML users who may have, 
themselves, been misguided regarding the technology (intentionally or 
unintentionally).

. . . . . . . . . . . . . Ken

--
Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

Hi Tim,

On Sun, Nov 15, 2009 at 2:15 AM, Tim Bray <Tim.Bray@sun.com> wrote:
> I thought Rick was being fair.  XSD is a failure by any sane technical
> measure.

I agree, that XSD spec has been quite mammoth and functionally
complex. That's the only aspect, I see XSD being different from most
of other W3C specs.

From functional point of view, I don't think XSD doesn't work. So many
numerous XML applications currently used XSD.

I don't think, there is anything wrong with the basic core/philosophy
of XSD. I guess, some of users who don't like XSD generally, it's
probably because of it's huge size, and steep learning curve.
I believe, XSD also get's functionally better with the upcoming 1.1 release.

> There's no reason not to acknowledge this and hopefully learn from
> it in future standardization efforts.

Pls see my previous remarks :)

> If we don't learn from our failures
> we're condemned to repeat them.

I agree. That's true for any endeavors we get engaged in :)


-- 
Regards,
Mukul Gandhi

On Sun, Nov 15, 2009 at 01:19:54PM +0530, Mukul Gandhi wrote:
> > As Tim and others said, DTDs are in the XML specification for
> > historical reasons.
> 
> I appreciate this. It's good that, something like this existed in the
> early days of XML (that was a nice innovation, in the absence of
> anything else, as we speak of XSD and other validation technologies,
> as of today).

Just to be clear, XML was designed so that every DTD-valid document
was also a valid SGML document; we had some 10 years experience
with SGML deployment, and that's where DTDs came from.

Liam

-- 
Liam Quin, W3C XML Activity Lead, http://www.w3.org/People/Quin/
http://www.holoweb.net/~liam/ * http://www.fromoldbooks.org/

At 2009-11-14 21:55 -0500, I wrote:
>There may be other "core" faults, but that is one that has been a 
>problem for me.

Of course I thought of another one right after I posted the above, a 
fault that is a deficiency rather than something broken (though you 
could consider the specification broken without the functionality):

W3C Schema validation semantics cannot express the constraints of 
XHTML's "head" element which in DTD-speak is:

<!ENTITY % head.misc "SCRIPT|STYLE|META|LINK|OBJECT" -- repeatable 
head elements -->
<!ENTITY % head.content "TITLE & BASE?">
<!ELEMENT HEAD O O (%head.content;) +(%head.misc;) -- document head -->

The problem is with the limited cardinality of "title" and "base" and 
the arbitrary order of elements preventing the use of alternation 
because of the ambiguous content models.

In RELAX-NG the interleave concept addresses this.

And yet another fault of 1.0 that will be addressed with 1.1 is 
mutually-exclusive attributes, which is needed if one wanted to 
express W3C XSL-FO using a schema (for <basic-link> one has either 
internal-destination= or external-destination= but not both and not 
neither).  And I believe 1.1 doesn't address this with a semantic 
expressing the exclusivity as an operator with operands, but with an 
assertion after the fact that the presence of both is unacceptable.

In RELAX-NG attributes and elements can both be operands in the 
constraint semantics.

So these are but two examples of W3C specifications of XML 
vocabularies that existed before the W3C Schema specification was 
finalized, where each one has document constraints that cannot be 
expressed by W3C Schema semantics.

Other posters in this debate have cited new W3C specifications that 
are using RELAX-NG instead of W3C Schema, and I doubt the reason they 
used it was because of the size and complexity of W3C Schema.  If 
they are XML experts, they can make their way through specifications 
of any size to look for what they need.  I suspect the decisions were 
based on deficiencies in the validation semantics required for their needs.

Again, I hope this is considered helpful to the debate.

. . . . . . . . . Ken

--
Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

At 2009-11-15 07:34 +0530, Mukul Gandhi wrote:
>On Sun, Nov 15, 2009 at 2:15 AM, Tim Bray <Tim.Bray@sun.com> wrote:
> > I thought Rick was being fair. Â XSD is a failure by any sane technical
> > measure.
>
>I agree, that XSD spec has been quite mammoth and functionally
>complex. That's the only aspect, I see XSD being different from most
>of other W3C specs.

I wouldn't agree with that.  I also side with Tim 
and Rick and others who grudgingly find they have 
to work with XSD (usually to meet customer 
requests; personally I migrated from DTD to 
RELAX-NG and never embraced XSD), and for the 
same reason that Tim cites above: technical 
issues.  Size and complexity are not, necessarily, technical faults.

> From functional point of view, I don't think XSD doesn't work.

There are a number of areas that are 
problematic.  One example is no two vendors have 
implemented "redefine" the same way.  I had 
researched this to meet specific requirements 
identified for the UBL project and was burned by 
assuming the way one processor implemented 
redefine was the way others implemented it.

>So many numerous XML applications currently used XSD.

Popularity is no measure of 
functionality.  Vendors who are pushing XSD 
products would likely tell their clients there 
are no other choices for schema languages.  It is 
my assessment that vendors and users ignore or 
are unaware of other sources of XML-based 
technologies, such as ISO/IEC JTC 1/SC 34 which 
created the ISO/IEC 19757 Document Schema 
Definition Languages (DSDL) that includes 
RELAX-NG, Schematron and NVDL among other XML specifications.

>I don't think, there is anything wrong with the basic core/philosophy
>of XSD.

An example of a core fault is that W3C Schema is 
not closed under union.  I cannot 
programmatically express the union of two XSD 
schemas to create a third XSD schema that 
validates instances of the other two.  I might be 
able to by hand, but not programmatically.

This is not academic:  consider an XML document 
where in two different contexts there is an 
element <x> with different content models.  I 
then write a query that assembles all of the 
documents <x> elements under a single parent 
result element.  I have to hand-craft the schema 
that validates the result of the query (if it is 
possible and not an ambiguous content model), as 
I cannot express in XSD-speak two sibling 
elements with different content models.  If I 
could simply express the union of the two 
original <x> content models as the content model 
for the query result <x>, then I wouldn't have to resort to hand-crafting.

There may be other "core" faults, but that is one 
that has been a problem for me.

In RELAX-NG compact syntax, if I have a grammar 
"a" and a grammar "b" (be it document level, 
element level or any level of a grammar; at the 
element level both grammars might express 
different content models for the element <x>), 
the union of these two grammars is expressed as 
"a | b".  No other work is necessary.  An 
instance of <x> will validate true if it passes 
the constraints expressed by either grammar.

>I guess, some of users who don't like XSD generally, it's
>probably because of it's huge size, and steep learning curve.

But cognoscenti such as Rick Jelliffe, Tim Bray 
and many others would not find a specification's 
huge size and steep learning curve a fault if, 
technically, the end result was something that worked well.

It might indeed be a barrier for new people to 
W3C Schema, but my recollection is that negative 
opinions in this debate have not come from representatives of that user group.

>I believe, XSD also get's functionally better with the upcoming 1.1 release.

Probably the functionality will be better, yes, 
but will it address everything?  We won't know 
until it is released, and I suspect some legacy 
1.0 issues will dog all dot-releases of XSD.

I hope this is considered helpful to the discussion.

. . . . . . . . . . . Ken


--
Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

At 2009-11-14 20:40 -0500, Ian S. Worthington wrote:
>I decided to go with the pull model
>as, rightly or wrongly, I felt slightly more in control.

No criticism there ... a matter of style and future flexibility.  But 
I think you need the flexibility now that you are adding the 
hyperlink to your report.  You didn't have that in your earlier 
rendering of a fix.

>I ditched the inline
>data so I could generate a link to the full data contents for each fix. But I
>couldn't find a way of doing that *under* the for-each whilst still retaining
>access to the @href.

Using a variable would do that.

>So I ended up with the following which, whilst it works,
>is presumably reading the linked document twice just to extract the fix number
>and fix abstract:

In syntax, yes, in runtime, no ... a processor will keep the tree 
cached in memory so there is in fact no repeated load of the source tree.

>         <ul>
>           <xsl:for-each select="fix">
>             <!-- <xsl:for-each select="document( fix/@href )/fix"> -->
>             <li>
>               <a>
>                 <xsl:attribute name="href">
>                   <xsl:value-of select="@href"/>
>                 </xsl:attribute>
>                 <xsl:value-of select="document( @href )/fix/number"/>
>               </a>
>               <br/><xsl:value-of select="document( @href )/fix/abstract"/>
>             </li>
>           </xsl:for-each>
>         </ul>
>
>I'm sure there's a more efficient way, yes?

First let me show you how the above would take advantage of a 
variable, but the above and this won't handle the embedded fixes, 
only the referred fixes:

         <ul>
           <xsl:for-each select="fix">
             <xsl:variable name="link" select="@href"/>
             <xsl:for-each select="document( $link )/fix"> -->
               <li>
                 <a href="{$link}">
                   <xsl:value-of select="number"/>
                 </a>
                 <br/><xsl:value-of select="abstract"/>
               </li>
             </xsl:for-each>
           </xsl:for-each>
         </ul>

But from your earlier post you wanted to handle both embedded and 
referred fixes ... so you want two behaviours based on a fix and you 
want the handling in the order of the fixes, which is arbitrary.  The 
two behaviours are to hyperlink fixes that are referred, and to not 
hyperlink fixes that are embedded.

I'm going to suggest again the push method as follows, because it 
fits well with the processing of the mixture of types of fixes:

         <ul>
           <xsl:apply-templates select="fix"/>
         </ul>
       </body>
     </html>
   </xsl:template>

   <!--embedded-->
   <xsl:template match="fix">
      <li>
         <xsl:value-of select="number"/>
         <br/><xsl:value-of select="abstract"/>
      </li>
   </xsl:template>

   <!--referred-->
   <xsl:template match="fix[@href]">
     <xsl:variable name="link" select="@href"/>
     <xsl:for-each select="document( $link )/fix"> -->
       <li>
         <a href="{$link}">
           <xsl:value-of select="number"/>
         </a>
         <br/><xsl:value-of select="abstract"/>
       </li>
     </xsl:for-each>
   </xsl:template>

With the above you can now have any arbitrary order of embedded and 
linked fixes and they will be processed accordingly each to their 
needs.  This is a primary benefit of the push model:  handling 
arbitrary orders of nodes.  Mixed content is another example of where 
the push model shines, but for the same reason:  handling arbitrary 
orders of nodes.

I hope this helps.

. . . . . . . . . . . . Ken

--
Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

Hi Ken --

Many thanks for the link to your book.  I decided to go with the pull model
as, rightly or wrongly, I felt slightly more in control.  I ditched the inline
data so I could generate a link to the full data contents for each fix. But I
couldn't find a way of doing that *under* the for-each whilst still retaining
access to the @href.  So I ended up with the following which, whilst it works,
is presumably reading the linked document twice just to extract the fix number
and fix abstract:


        <ul>
          <xsl:for-each select="fix">
            <!-- <xsl:for-each select="document( fix/@href )/fix"> -->
            <li>
              <a>
                <xsl:attribute name="href">
                  <xsl:value-of select="@href"/>
                </xsl:attribute>          
                <xsl:value-of select="document( @href )/fix/number"/>
              </a>
              <br/><xsl:value-of select="document( @href )/fix/abstract"/>
            </li>  
          </xsl:for-each>
        </ul>

I'm sure there's a more efficient way, yes?

i


------ Original Message ------
Received: 05:50 PM COT, 11/14/2009
From: "G. Ken Holman" <gkholman@CraneSoftwrights.com>
To: <xml-dev@lists.xml.org>
Subject: Re:  xsl processing problem with multiple templates

> At 2009-11-14 16:55 -0500, Ian S. Worthington wrote:
> >The idea of pushing is new to me I'm afraid.
> 
> In fact you are already using it because you catch a node pushed at 
> your stylesheet when using a template match.
> 
> >Where should that "apply-templates" line go?
> 
> In place of the <xsl:for-each/>
> 
> >Obviously not here:
> >
> ><xsl:stylesheet version="1.0"
> >xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
> >   <xsl:template match="fixlist">
> >     <xsl:message>Matched data root</xsl:message>
> >     <html>
> >     <body>
> >       <xsl:for-each select="fix">
> >         <xsl:apply-templates select="fix[not(@href)] |
> >document( fix/@href )/fix"/>
> >         <p>Change number: <xsl:value-of select="number"/></p>
> >       </xsl:for-each>
> >     </body>
> >     </html>
> >   </xsl:template>
> ></xsl:stylesheet>
> 
> Your choice .... either using pull:
> 
>        <xsl:for-each select="fix[not(@href)] |
>                              document( fix/@href )/fix">
>          <p>Change number: <xsl:value-of select="number"/></p>
>        </xsl:for-each>
> 
> .. or using push:
> 
> <xsl:stylesheet version="1.0"
> xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
>    <xsl:template match="fixlist">
>      <xsl:message>Matched data root</xsl:message>
>      <html>
>      <body>
>       <xsl:apply-templates select="fix[not(@href)] |
>                                    document( fix/@href )/fix"/>
>      </body>
>      </html>
>    </xsl:template>
> 
>    <xsl:template match="fix">
>     <p>Change number: <xsl:value-of select="number"/></p>
>    </xsl:template>
> 
> </xsl:stylesheet>
> 
> 
> I hope this helps.  On my web site I have a free download of an 
> excerpt of my XSLT book where I've introduced the concepts of pull and
push.
> 
> I hope this helps.
> 
> . . . . . . . . Ken
> 
> --
> Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
> Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
> Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
> Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
> Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
> G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
> Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
> Legal business disclaimers:  http://www.CraneSoftwrights.com/legal
> 
> 
> _______________________________________________________________________
> 
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
> 
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> 
> 

On Sun, Nov 08, 2009 at 09:52:36AM +0530, Mukul Gandhi wrote:
[...]
> My averment was to give XSD the same status as DTD, as a XML
> validation technology in W3C XML spec.

Speaking as W3C XML Activity Lead :-) I don't see a reason to
give XSD special status in that way.

W3C XSD has a lot of uses, and works well if you are also
using XSLT 2, XQuery, Web Services or other specifications
where the XSD support is well-integrated.  This doesn't in
any way preclude using RelaxNG, DTDs, Schematron, or any
other sort of validation, and it's not really the place of
the XML specification to give a list of other technologies
that are used with it or build on it.

As Tim and others said, DTDs are in the XML specification for
historical reasons.  If we had the resources, I can certainly
imagine refactoring XML 1.0 to put DTDs into a separate document,
and to bring into that document namespaces and xml:base and
maybe others. But the work of doing that is much greater than
it sounds -- making sure the text retains the same meaning,
and even that documents referring to specific productoins in the
EBNF grammar are not broken as a result.

So I don't expect much enthusiasm for making a new edition of
XML to make the Informative References include XSD, but
please don't take that as a negative statement about XSD :-)

Liam

-- 
Liam Quin, W3C XML Activity Lead, http://www.w3.org/People/Quin/
http://www.holoweb.net/~liam/ * http://www.fromoldbooks.org/

> 
> Would you say that a new basic text book about XML should 
> hardly mention XInclude, XLink, and XPointer at all, at least 
> not in whole chapters, but maybe in some paragraph in a 
> chapter about failed standards?

In the case of XLink and XPointer, yes: I'd say these are historical
curiosities, and the only interest in them is in understanding why they
failed.

XInclude I think has some merit: even if it's not widely used, it's viable
to use it, and there are situations where I would recommend using it in
preference to external entity references. But plain URIs held in
user-defined attributes are probably just as effective for many
applications.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

As we all know on this list, the XML article in Wikipedia contains the
following:

"Some other specifications conceived as part of the "XML Core" have
failed to find wide adoption, including XInclude, XLink, and
XPointer."

But what does it mean?

Probably all text books about XML have a chapter dedicated to each of
those standards.

Would you say that a new basic text book about XML should hardly
mention XInclude, XLink, and XPointer at all, at least not in whole
chapters, but maybe in some paragraph in a chapter about failed
standards?

Cheers,
Jesper Tverskov

http://www.xmlkurser.dk
http://www.xmlplease.com

> 
> I have nothing to contest in this statement, but my point was 
> not about how far XML has gotten, but about the trend going 
> forward.  This feels like a wonderful toast to past success, 
> but says very little about new adoptions.
> 

Is there any evidence that *anyone* is building standards on top of of 
JSON (or any other non-XML data syntax), in the way that thousands of 
industry standards are built on top of XML?

Michael Kay
Saxonica 

At 2009-11-14 16:55 -0500, Ian S. Worthington wrote:
>The idea of pushing is new to me I'm afraid.

In fact you are already using it because you catch a node pushed at 
your stylesheet when using a template match.

>Where should that "apply-templates" line go?

In place of the <xsl:for-each/>

>Obviously not here:
>
><xsl:stylesheet version="1.0"
>xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
>   <xsl:template match="fixlist">
>     <xsl:message>Matched data root</xsl:message>
>     <html>
>     <body>
>       <xsl:for-each select="fix">
>         <xsl:apply-templates select="fix[not(@href)] |
>document( fix/@href )/fix"/>
>         <p>Change number: <xsl:value-of select="number"/></p>
>       </xsl:for-each>
>     </body>
>     </html>
>   </xsl:template>
></xsl:stylesheet>

Your choice .... either using pull:

       <xsl:for-each select="fix[not(@href)] |
                             document( fix/@href )/fix">
         <p>Change number: <xsl:value-of select="number"/></p>
       </xsl:for-each>

... or using push:

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
   <xsl:template match="fixlist">
     <xsl:message>Matched data root</xsl:message>
     <html>
     <body>
      <xsl:apply-templates select="fix[not(@href)] |
                                   document( fix/@href )/fix"/>
     </body>
     </html>
   </xsl:template>

   <xsl:template match="fix">
    <p>Change number: <xsl:value-of select="number"/></p>
   </xsl:template>

</xsl:stylesheet>


I hope this helps.  On my web site I have a free download of an 
excerpt of my XSLT book where I've introduced the concepts of pull and push.

I hope this helps.

. . . . . . . . Ken

--
Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

Hi Ken --

The idea of pushing is new to me I'm afraid.  Where should that
"apply-templates" line go?
Obviously not here:

<xsl:stylesheet version="1.0" 
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:template match="fixlist">
    <xsl:message>Matched data root</xsl:message>
    <html>
    <body>
      <xsl:for-each select="fix">
        <xsl:apply-templates select="fix[not(@href)] |
document( fix/@href )/fix"/>
        <p>Change number: <xsl:value-of select="number"/></p>
      </xsl:for-each>
    </body>
    </html>
  </xsl:template> 
</xsl:stylesheet>

where the inlines get processed fine but not the referrals.

i
------ Original Message ------
Received: 04:22 PM COT, 11/14/2009
From: "G. Ken Holman" <gkholman@CraneSoftwrights.com>
To: <xml-dev@lists.xml.org>
Subject: Re:  xsl processing problem with multiple templates

> XSLT and XPath questions would be better posted to the following list:
> 
>    http://www.mulberrytech.com/xsl/xsl-list
> 
> There are a number of subscribers who would enthusiastically respond 
> to such questions.
> 
> There is also an *excellent* XSLT FAQ at:
> 
>    http://www.dpawson.co.uk
> 
> At 2009-11-14 15:56 -0500, Ian S. Worthington wrote:
> >I have a bunch of fixn.xml files which contain something like:
> >
> ><?xml version="1.0" encoding="ISO-8859-1"?>
> ><?xml-stylesheet type="text/xsl" href="fix.xsl"?>
> ><fix>
> ><number>BED012</number>
> >...
> ></fix>
> >
> >Apart from being able to display them alone (via the embedded xsl) I'd like
to
> >turn them into a list of fixes.  It looks like xinclude is what I need for
> >this,
> 
> It doesn't look like that to me ... it looks like you need the 
> document() function.
> 
> >My master list of fixes looks like:
> >
> ><?xml version="1.0" encoding="ISO-8859-1"?>
> ><?xml-stylesheet type="text/xsl" href="xinclude.xsl"?>
> ><fixlist xmlns:xi="http://www.w3.org/2001/XInclude">
> >         <fix><number>BED000</number></fix>
> >         <fix><number>BED111</number></fix>
> >         <xi:include href="fix1.xml"/>
> >         <xi:include href="fix2.xml"/>
> ></fixlist>
> 
> How about something like:
> 
> <?xml version="1.0" encoding="ISO-8859-1"?>
> <?xml-stylesheet type="text/xsl" href="allfixes.xsl"?>
> <fixlist>
>          <fix><number>BED000</number></fix>
>          <fix><number>BED111</number></fix>
>          <fix href="fix1.xml"/>
>          <fix href="fix2.xml"/>
> </fixlist>
> 
> Then in allfixes.xsl have one instruction that pushes all fix elements:
> 
>    <xsl:apply-templates select="fix[not(@href)] |
>                                 document( fix/@href )/fix"/>
> 
> .. and then handle each fix element that arrives either from the 
> local file or the referred file.
> 
> >Is there anyway of getting them both to work in harmony?
> 
> The above is one push instruction for all fixes, embedded or referred.
> 
> I hope this helps.
> 
> . . . . . . . . . . Ken
> 
> --
> Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
> Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
> Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
> Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
> Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
> G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
> Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
> Legal business disclaimers:  http://www.CraneSoftwrights.com/legal
> 
> 
> _______________________________________________________________________
> 
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
> 
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> 
> 

XSLT and XPath questions would be better posted to the following list:

   http://www.mulberrytech.com/xsl/xsl-list

There are a number of subscribers who would enthusiastically respond 
to such questions.

There is also an *excellent* XSLT FAQ at:

   http://www.dpawson.co.uk

At 2009-11-14 15:56 -0500, Ian S. Worthington wrote:
>I have a bunch of fixn.xml files which contain something like:
>
><?xml version="1.0" encoding="ISO-8859-1"?>
><?xml-stylesheet type="text/xsl" href="fix.xsl"?>
><fix>
><number>BED012</number>
>...
></fix>
>
>Apart from being able to display them alone (via the embedded xsl) I'd like to
>turn them into a list of fixes.  It looks like xinclude is what I need for
>this,

It doesn't look like that to me ... it looks like you need the 
document() function.

>My master list of fixes looks like:
>
><?xml version="1.0" encoding="ISO-8859-1"?>
><?xml-stylesheet type="text/xsl" href="xinclude.xsl"?>
><fixlist xmlns:xi="http://www.w3.org/2001/XInclude">
>         <fix><number>BED000</number></fix>
>         <fix><number>BED111</number></fix>
>         <xi:include href="fix1.xml"/>
>         <xi:include href="fix2.xml"/>
></fixlist>

How about something like:

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="allfixes.xsl"?>
<fixlist>
         <fix><number>BED000</number></fix>
         <fix><number>BED111</number></fix>
         <fix href="fix1.xml"/>
         <fix href="fix2.xml"/>
</fixlist>

Then in allfixes.xsl have one instruction that pushes all fix elements:

   <xsl:apply-templates select="fix[not(@href)] |
                                document( fix/@href )/fix"/>

... and then handle each fix element that arrives either from the 
local file or the referred file.

>Is there anyway of getting them both to work in harmony?

The above is one push instruction for all fixes, embedded or referred.

I hope this helps.

. . . . . . . . . . Ken

--
Vote for your XML training:   http://www.CraneSoftwrights.com/x/i/
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/x/
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/x/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal

Hi.

I have a bunch of fixn.xml files which contain something like:

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="fix.xsl"?>
<fix>
<number>BED012</number>
...
</fix>

Apart from being able to display them alone (via the embedded xsl) I'd like to
turn them into a list of fixes.  It looks like xinclude is what I need for
this, but it isn't actually implemented in any standard browser yet.  However
there's a partial workaround at http://forums.devx.com/showthread.php?t=160331
which I'm trying to get going. 

But I appear to be able to drive the including *or* the transformation, but
not both.  Is this just a namespace issue or is there something insurmountable
here?

My master list of fixes looks like:

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="xinclude.xsl"?>
<fixlist xmlns:xi="http://www.w3.org/2001/XInclude">
	<fix><number>BED000</number></fix>
	<fix><number>BED111</number></fix>
	<xi:include href="fix1.xml"/>
	<xi:include href="fix2.xml"/>
</fixlist>

and the xinclude.xsl, based off the referenced article:

<xsl:stylesheet version="1.0" 
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:f="f" 
xmlns:xi="http://www.w3.org/2001/XInclude" exclude-result-prefixes="xi">

	<xsl:include href="no-ns-identity.xsl"/>
	
	<xsl:template match="xi:include">
	  <xsl:message>Matched include root</xsl:message>
		<xsl:apply-templates select="document(@href)"/>
	</xsl:template>		
 
  <xsl:template match="fixlist">
    <xsl:message>Matched data root</xsl:message>
    <html>
    <body>
      <xsl:for-each select="fix">
        <p>Change number: <xsl:value-of select="number"/></p>
      </xsl:for-each>
    </body>
    </html>
  </xsl:template> 
 </xsl:stylesheet>


Like this, the fixlist match fires and generates the desired html, but without
the includes.

Or if I prevent the fixlist from matching then the xi:include match will fire
and the correct xml is generated, but (of course) is not transformed into
html.

I've tried putting the fixlist into a different namespace, but with no
difference.

Is there anyway of getting them both to work in harmony?

ian 
.. 

Ian S. Worthington, MBCS.

me:     http://isw.me.uk/ 
photos: http://gallery.isw.me.uk/ 


Free 2GB online backups:
  https://mozy.com/?code=HJW4C8 (code gets you 256MB extra)

Web hosting and support from $2.99/month:
  www.cirtexhosting.com/affiliates/idevaffiliate.php?id=463


Dulce et decorum est pro patria mori, sed dulcius pro patria vivere, et
dulcissimus pro patria biber. Ergo, bibiamo pro salute patriae.

On Sat, Nov 14, 2009 at 12:49 PM, Simon St.Laurent
<simonstl@simonstl.com> wrote:

> At this point, personally, I'd drop coverage of XLink and XPointer.  A
> paragraph seems reasonable.
>
> XInclude does get use in back-end processing, so I'd leave it in, most
> likely - at least as a sidebar in the entity processing section.  It doesn't
> require a huge amount of space to explain in any case.

Sounds about right, though XInclude references XPointer so it's hard
to really say.

Maybe the next edition of XML in a Nutshell, if it's even worth
writing at this point, should drop the XLink and XPointer chapters and
use the space for XQuery.

The problem I keep running up against whenever I think about a
potential 4th edition (besides complete lack of time) is that XPath
2/XSLT 2/XQuery are really a book in their own right. In fact, they're
probably two books. Possibly XQuery in a Nutshell is the book I should
be writing, which would of course cover XPath 2 as well. This book
would ignore XPath 1/XSLT 1, which would stay in XML in a Nutshell
since they're very widely used and will continue to be so. I'm not
sure what to do about XSLT 2. Maybe I should just leave that to
Michael Kay's book. And then again the whole nutshell format is
increasingly weak given wide access to Web references, so maybe that's
not the XQuery book the world needs right now. In fact, I'm pretty
sure it isn't.

Curious what folks think about this. Which book would you buy? What
book would your companies and clients buy multiple copies of if it
existed? Or should we just give up on the whole notion of new XML
books?

-- 
Elliotte Rusty Harold
elharo@ibiblio.org

Michael Kay wrote:
>> I have nothing to contest in this statement, but my point was 
>> not about how far XML has gotten, but about the trend going 
>> forward.  This feels like a wonderful toast to past success, 
>> but says very little about new adoptions.
>>
> 
> Is there any evidence that *anyone* is building standards on top of of 
> JSON (or any other non-XML data syntax), in the way that thousands of 
> industry standards are built on top of XML?

Does it matter?

You can build standards on JSON, but this seems like an effort to tweak 
the definition of success such that it points toward XML.

If anything, JSON's cultural drift toward anarchy seems to have helped 
rather than hindered its adoption.

(And I think we're seeing similar trends in the weirdly-named NoSQL 
movement too.)

-- 
Simon St.Laurent
http://simonstl.com/

Jesper Tverskov wrote:
> As we all know on this list, the XML article in Wikipedia contains the
> following:
> 
> "Some other specifications conceived as part of the "XML Core" have
> failed to find wide adoption, including XInclude, XLink, and
> XPointer."
> 
> But what does it mean?
> 
> Probably all text books about XML have a chapter dedicated to each of
> those standards.
> 
> Would you say that a new basic text book about XML should hardly
> mention XInclude, XLink, and XPointer at all, at least not in whole
> chapters, but maybe in some paragraph in a chapter about failed
> standards?

At this point, personally, I'd drop coverage of XLink and XPointer.  A 
paragraph seems reasonable.

XInclude does get use in back-end processing, so I'd leave it in, most 
likely - at least as a sidebar in the entity processing section.  It 
doesn't require a huge amount of space to explain in any case.

-- 
Simon St.Laurent
http://simonstl.com/

On 2009-11-11, at 12:50 AM, Mukul Gandhi wrote:

> As suggested by Jim Tivy earlier, a generic phrase like "XML Schema"
> in the XML spec, while referring to XML validation would be helpful to
> readers, in the present context.

Absolutely not.  The various XML schema languages need well-formed  
XML. Well-formed XML exists independent of any schemas.  The fact that  
DTDs were baked into XML 1.0 is a historical accident.  -Tim

On 2009-11-08, at 5:27 AM, Mukul Gandhi wrote:

> On Sun, Nov 8, 2009 at 2:10 PM, Rick Jelliffe <rjelliffe@allette.com.au 
> > wrote:
>> Because XSD is complete crap that utterly betrayed the premises and  
>> goals of
>> XML perhaps?
>
> Rick, I do not know why you condemn XSD so much :) I feel, XSD doesn't
> deserve so much condemnation.

I thought Rick was being fair.  XSD is a failure by any sane technical  
measure.  There's no reason not to acknowledge this and hopefully  
learn from it in future standardization efforts.  If we don't learn  
from our failures we're condemned to repeat them.  -Tim

Simon St.Laurent wrote:
> At this point, personally, I'd drop coverage of XLink and XPointer.  A 
> paragraph seems reasonable.
>
> XInclude does get use in back-end processing, so I'd leave it in, most 
> likely - at least as a sidebar in the entity processing section.  It 
> doesn't require a huge amount of space to explain in any case.
>
I'd have to concur, unless the book is about XBRL, which is unique in 
making very heavy (if misguided) use of XLink.

XInclude is such a simple specification that it's handy to have the 
standard around when you need such a mechanism. Also, it's trivial to 
implement in XSLT 2.0.[1]

Evan Lenz
http://lenzconsulting.com

[1] http://lenzconsulting.com/namespaces-in-xslt/#inherit-namespaces

I have nothing to contest in this statement, but my point was not about 
how far XML has gotten, but about the trend going forward.  This feels 
like a wonderful toast to past success, but says very little about new 
adoptions.

COBOL was an amazing advance in its time as well...

Thanks,
Simon St.Laurent
http://simonstl.com/

Jim Tivy wrote:
> Well, I apologize for no hard data here but feel free to contest what I 
> say then I will try to find some data:
> 
> XML is a leading open content format and is a leading open data format.  
> The key term is "open" - which herein means there are lots of tools, it 
> has an exchange format (text) and it is backed by a number of open 
> standards that flesh it out in many dimensions. 
> 
> As an open format XML has gone further than any other such format in the 
> history of computing.
> 
> There are other important open data standards - CSV in the past and now 
> JSON as well as SGML.  None of these have seen the breadth of 
> application and number of adoptions as XML (measured in kilograms).


-- 
Simon St.Laurent
http://simonstl.com/

Hi,

A new version of Jing and Trang, 20091111 is available from
http://code.google.com/p/jing-trang/

There are tree issues fixes since the last version (20090818) affecting 
the NVDL validation, the conversion to DTD and packaging. For details 
please see:

http://code.google.com/p/jing-trang/issues/detail?id=85
http://code.google.com/p/jing-trang/issues/detail?id=86
http://code.google.com/p/jing-trang/issues/detail?id=89

Best Regards,
George
-- 
George Cristian Bina
<oXygen/> XML Editor, Schema Editor and XSLT Editor/Debugger
http://www.oxygenxml.com

XMLmind is happy to announce a new release of
XMLmind DITA Converter.


XMLmind DITA Converter 1.1.0 (November 9, 2009)
-----------------------------------------------

Allows to convert DITA documents to Eclipse Help. Several other
enhancements.
More info in http://www.xmlmind.com/ditac/changes.html

Download XMLmind DITA Converter 1.1.0 from
http://www.xmlmind.com/ditac/download.shtml

-----------------------------------------------
XMLmind DITA Converter v1.1.0 (ditac for short) allows to convert the
most complex DITA 1.1 documents to production-quality XHTML 1.0, XHTML
1.1, HTML 4.1, Java[tm] Help, HTML Help, Eclipse Help, PDF,
PostScript[reg], RTF (can be opened in Word 2000+), WordprocessingML
(can be opened in Word 2003+), Office Open XML (.docx, can be opened in
Word 2007+), OpenOffice (.odt, can be opened in OpenOffice.org 2+).

XMLmind DITA Converter has been developed in order to be integrated in
XMLmind commercial products. As such, XMLmind DITA Converter is
professionally developed software, well documented and well supported.

XMLmind DITA Converter is free, open source, software licensed under the
very liberal terms of the Mozilla Public License version 1.1.

All this makes XMLmind DITA Converter a very serious alternative to
using the DITA Open Toolkit.

_______________________________________________

> 
> Ah - "the chattering classes".  That's almost as useful a 
> phrase as "politically incorrect" for avoiding engagement.  
> And in data-poor fields, like this one, it pretty much lets 
> you believe whatever you'd prefer to believe.

In the absence of evidence, I think it's better to believe nothing than to
make wild guesses. And I do think there is a very big difference between
what it is fashionable to talk about, and what people are actually using. 
> 
> I could offer book sales data, in which XML book sales have 
> plunged well below the fall of the rest of the market, but I 
> suspect that's not very interesting as it effectively 
> measures the volume of chatter people are willing to pay for...
> 

I have a theory that the people who buy books about a technology tend to be
early adopters. (Late adopters go on training courses, or learn from their
colleagues, or learn by trial and error.) There aren't any early adopters of
XML any more. People are doing their third or fourth project, and you don't
need new books for that. 

Michael Kay

> My general sense - and this doubtless has a lot of 
> confirmation bias to it - is that XML is retreating to the 
> document-based home territory where it made the most sense 
> anyway.  JSON is easing it off the Web.

I think you're using the term "the Web" to mean "the edges of the web" -
i.e. pretty well the "last hop" to the browser. I think that's an
unfortunate usage, but so long as we know what you mean, I guess we have to
live with it. Anyway, from the rest of your message, you actually seem to be
saying "retreating to document processing and data interchange", which I
have always thought were the two core areas for application of XML, so I'm
not sure I see that as a retreat.  

But data interchange is pretty fundamental to an awful lot of applications,
and if XML is used for interchange, then it tends to pervade the whole
application.

And as always, I'm very suspicious of statements that claim to be
quantitative assessments of industry trends but are actually based on
nothing more than the volume of noise made by the chattering classes. That
simply isn't a good enough metric.

Michael Kay

http://www.google.co.uk/search?q=xslt+word+wrap

shows up several word wrapping possibilities in xslt.


________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

Peter Hunsberger wrote:
> The one place I still see a need for such types of XML is when you're
> doing cross domain integration and signatures, encryption,
> authentication, etc. become part of the exchange process.  JSON
> doesn't seem to have all the pieces lined up to push XML (SOAP or
> otherwise) out of this space (yet)?

That's an excellent point, especially on the data interchange front. 
Much of that infrastructure is tightly bound to XML as well, so isn't 
easily replaced.

> There's a lot of capability that has been built around XML that still
> needs to be replicated in the other spaces so I think it would be
> premature to declare any kind of overwhelming trend.  At this point I
> perceive what is going on as more as people reacting to the flaws in
> the current XML world and testing out alternate strategies to see if
> they do any better.  Whether these alternate strategies will, in the
> long run, grow to have as many capabilities (and flaws) as XML remains
> to be seen?

Fair enough.  I think another way of putting it is that XML is slowly 
losing its status as the default format in fields where it once seemed 
(rightly or wrongly) the obvious answer.

-- 
Simon St.Laurent
http://simonstl.com/

Michael Kay wrote:
> In the absence of evidence, I think it's better to believe nothing than to
> make wild guesses. And I do think there is a very big difference between
> what it is fashionable to talk about, and what people are actually using. 

That's a much more restrained claim than your previous swipe, but I see 
no evidence thus far that challenges the proposal I put forward.

> I have a theory that the people who buy books about a technology tend to be
> early adopters. (Late adopters go on training courses, or learn from their
> colleagues, or learn by trial and error.) There aren't any early adopters of
> XML any more. People are doing their third or fourth project, and you don't
> need new books for that. 

To some extent, that's true - the big waves of book sales are driven by 
early adopters.  Sales after those waves are typically to late arrivals 
who need to catch up, and they have many other ways to catch up.

However, in this case, that story doesn't help.  XML book sales were 
gigantic when XML first arrived.  The next technology adoption in the 
space was XSLT - almost as large.  Then XML Schema, far smaller.  Then 
XQuery, where the market barely registered a bump.

(Late arrival sales for XML itself and XSLT vastly outweigh sales of 
books on XML Schema and XQuery as well.  Semantic Web books are up and 
outperforming XML books, though still not enough for jubilation.)

I think it's safe to say that all of those subjects seemed of roughly 
equal difficulty when they arrived.  Though tech book sales have 
declined over time, the level of enthusiasm for these subjects relative 
to the book market in general has declined more severely.

The anecdotal evidence I've heard - chatter, perhaps - tells the same story.

-- 
Simon St.Laurent
http://simonstl.com/

Michael Kay wrote:
>> My general sense - and this doubtless has a lot of 
>> confirmation bias to it - is that XML is retreating to the 
>> document-based home territory where it made the most sense 
>> anyway.  JSON is easing it off the Web.
> 
> I think you're using the term "the Web" to mean "the edges of the web" -
> i.e. pretty well the "last hop" to the browser. I think that's an
> unfortunate usage, but so long as we know what you mean, I guess we have to
> live with it. 

It may not be _your_ usage, but that 'last hop' is at least historically 
where XML aimed to go, so I have no qualms in applying it to the XML 
conversation.

> Anyway, from the rest of your message, you actually seem to be
> saying "retreating to document processing and data interchange", which I
> have always thought were the two core areas for application of XML, so I'm
> not sure I see that as a retreat.  

Fair enough.  I do see it as a retreat from a lot of the XML database 
claims pushed in roughly 2000-2005, because...

> But data interchange is pretty fundamental to an awful lot of applications,
> and if XML is used for interchange, then it tends to pervade the whole
> application.

My point here is that I'm seeing less and less interest in that 
pervasiveness.  There was a time when more people said "I'm exchanging 
lots of this XML stuff, and maybe I should just store the XML stuff and 
work with it that way."

Lately, the conversation seems to be more "I need to maintain this 
information in a form that lets me scale processing and storage 
efficiently", and XML fares less well in that conversation.

For information that needs to be stored and processed as 
hierarchically-structured and sort of linear documents, it does well, 
but for other purposes it makes sense only as an interchange format, 
created and consumed at gateways to systems which think in terms other 
than XML.

> And as always, I'm very suspicious of statements that claim to be
> quantitative assessments of industry trends but are actually based on
> nothing more than the volume of noise made by the chattering classes. That
> simply isn't a good enough metric.

Ah - "the chattering classes".  That's almost as useful a phrase as 
"politically incorrect" for avoiding engagement.  And in data-poor 
fields, like this one, it pretty much lets you believe whatever you'd 
prefer to believe.

I could offer book sales data, in which XML book sales have plunged well 
below the fall of the rest of the market, but I suspect that's not very 
interesting as it effectively measures the volume of chatter people are 
willing to pay for...

-- 
Simon St.Laurent
http://simonstl.com/

On Thu, Nov 12, 2009 at 8:36 AM, Simon St.Laurent <simonstl@simonstl.com> wrote:
> [I wrote this bit as part of a NoSQL conversation at O'Reilly, but think it
> might be interesting, or at least provocative, in a broader XML context. Any
> thoughts?]
>
> My general sense - and this doubtless has a lot of confirmation bias
> to it - is that XML is retreating to the document-based home territory
> where it made the most sense anyway.  JSON is easing it off the Web.
> RDF and just plain APIs make a lot more sense for graph-centric
> data.
>
> XML will continue for interchange and in places where tree
> structures work and the overhead isn't too bad, but the "XML
> Everywhere" vision is retreating.
>

The one place I still see a need for such types of XML is when you're
doing cross domain integration and signatures, encryption,
authentication, etc. become part of the exchange process.  JSON
doesn't seem to have all the pieces lined up to push XML (SOAP or
otherwise) out of this space (yet)?

> In the long run, I think that's better for both the world and XML,
> though maybe not for XQuery.
>

There's a lot of capability that has been built around XML that still
needs to be replicated in the other spaces so I think it would be
premature to declare any kind of overwhelming trend.  At this point I
perceive what is going on as more as people reacting to the flaws in
the current XML world and testing out alternate strategies to see if
they do any better.  Whether these alternate strategies will, in the
long run, grow to have as many capabilities (and flaws) as XML remains
to be seen?

--
Peter Hunsberger



-- 
Peter Hunsberger

On Nov 12, 2009, at 10:03 AM, Kurt Cagle wrote:
> 
> Something that I think it worth noting is that for all the volubility of the AJAX crowd, JSON messaging is no longer really gaining traction against XML messaging, and may actually be shrinking. Admittedly, I'm biased in this regard, but I think as AJAX itself matures and toolkits become more "enterprise hardened" (and as RESTful services become more dominant), the mismatch between XML and JSON at the business level is a fairly significant impedance barrier to pervasive adoption.


The client-side crowd is vastly more interested in JSON than XML. XML comes up because the client-side dev was forced to use. Rarely would it be the first choice of data exchange between the server(s) and client.

I see it trending to offering multiple options. I like the Jersey and JAXB for this reason. I used to think jaxb was too brittle, but lately have been appreciating all the time it saves with generating the source code to enabling easy rendering of different content types. 

http://server.com/path/to/resource.xml
http://server.com/path/to/resource.json

can use the same class and method to handle the request with me providing the code like:

return Response.ok().entity(someBean).build();

That being said, I would really love a standards based way of knowing what conversion between XML and JSON structures would provide across all the tools that do that type of thing.

best,
-Rob

[I wrote this bit as part of a NoSQL conversation at O'Reilly, but think 
it might be interesting, or at least provocative, in a broader XML 
context. Any thoughts?]

My general sense - and this doubtless has a lot of confirmation bias
to it - is that XML is retreating to the document-based home territory
where it made the most sense anyway.  JSON is easing it off the Web.
RDF and just plain APIs make a lot more sense for graph-centric
data.

XML will continue for interchange and in places where tree
structures work and the overhead isn't too bad, but the "XML
Everywhere" vision is retreating.

In the long run, I think that's better for both the world and XML,
though maybe not for XQuery.

-- 
Simon St.Laurent
http://simonstl.com/

Even the way most browsers treat XML showing an indented XML tree, or
a webpage for RSS and ATOM xml news feeds, are not examples of styling
XML with CSS.

The browsers transform the news feeds with JavaScript and actually
transform XML to HTML. What looks like XML+CSS is in reality HTML+CSS.

We end up with the following conclusion: it is hard to find any
examples of importance of XML being styled with CSS to be displayed
directly in browsers.

Except for XML applications made for browsers like XHTML.

And when it comes to PRINCExml it seems that it mostly use CSS for
XHTML and HTML, or even for DocBook. PRINCExml can only style any XML
with CSS, if the XML is structured in a way that makes sense for
converting it to PDF directly. If the XML does not have such "layout"
structure, there must be some programming that can put it into place.

Cheers,
Jesper Tverskov

http://www.xmlkurser.dk
http://www.xmlplease.com

thank you, I was unaware that XSLT handled word wrapping in any form.


David A. Lee
dlee@calldei.com  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



David Carlisle wrote:
> http://www.google.co.uk/search?q=xslt+word+wrap
>
> shows up several word wrapping possibilities in xslt.
>
>
> ________________________________________________________________________
> The Numerical Algorithms Group Ltd is a company registered in England
> and Wales with company number 1249803. The registered office is:
> Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.
>
> This e-mail has been scanned for all viruses by Star. The service is
> powered by MessageLabs. 
> ________________________________________________________________________
>   

The "Ideal" solution would be to take advantage of the font 
charactoristics of the device, plug that into the processr and have it used
to word wrap at a pixel width.  (for reference I'm refering to 'legacy' 
Palm, PPC and blackberry devices using the system font).
But if thats too hard, even coming close with a charactor-count width 
would work but wouldnt produce as good results.
In particular I'm not sure how to do the tables at all with 
character-count instead of using the font metrics.
In another part of the product we solve this part by forcing tables into 
a fixed width font.  I'm not sure that the piece of code I need to shove 
this data supports fixed width intermixed with variable width.




David A. Lee
dlee@calldei.com  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Mike Sokolov wrote:
> That's an interesting problem: I'm confused about the wrapping 
> requirement though.  Do you need to wrap at a given number of 
> characters (sounded like 40?) or a pixel (or inch) width?  Your 
> comment about the variable-width font makes it sound like you are 
> printing or displaying on screen with a particular font.  It's 
> obviously much simpler if you don't have to worry about fonts, 
> variable character widths, vagaries of printer drivers and the like.
>
> -Mike
>
> David A. Lee wrote:
>> I have a project where I need to take XML and format it as ascii text.
>> Sounds simple right ? But alas the requirements are subtle.  It seems 
>> people dont do text formatting much anymore.
>> I fondly remember the days of nroff where I could do good text 
>> formatting and achieve things like word splitting (hyphenating),
>> justification etc on text content.  In particular a challenge is to 
>> support multi column tables.  Thats right, text spanning 5 columns,
>> that needs to wrap in each column.   Oh and do that with 40 column 
>> line widths. Piece of cake.
>> Oh and its a Variable width text font ... ha !
>>
>> What I tried so far which almost works sorta, just enough to not work 
>> well enough :)
>> Translate the XML to HTML with XSLT then I downloaded one of the free 
>> "HTML to Text" tools which are prevalent on the net.
>> It sorta did work, except for the tables.   But I need something that 
>> can work in a production environment.   Open Source is prefered,
>> but if it does an excellent job then a commercial product could be 
>> justified (pun intended).
>>
>> My next attempt will to try Apache FOP.    I looked at RenderX but 
>> they dont claim text output.
>> Apache FOP claims a text output but with caveots, the claim is its 
>> not robust code with poor results.
>> Maybe they are being kind and it really works great.   I will try.
>>
>> Any other suggestions ?   I can always 'roll my own' but this project 
>> is not on the budget yet and were trying to see if we can do 
>> something with minimal work.
>> Appreciated,
>>
>> thanks,
>>
>>
>>
>>
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

That's an interesting problem: I'm confused about the wrapping 
requirement though.  Do you need to wrap at a given number of characters 
(sounded like 40?) or a pixel (or inch) width?  Your comment about the 
variable-width font makes it sound like you are printing or displaying 
on screen with a particular font.  It's obviously much simpler if you 
don't have to worry about fonts, variable character widths, vagaries of 
printer drivers and the like.

-Mike

David A. Lee wrote:
> I have a project where I need to take XML and format it as ascii text.
> Sounds simple right ? But alas the requirements are subtle.  It seems 
> people dont do text formatting much anymore.
> I fondly remember the days of nroff where I could do good text 
> formatting and achieve things like word splitting (hyphenating),
> justification etc on text content.  In particular a challenge is to 
> support multi column tables.  Thats right, text spanning 5 columns,
> that needs to wrap in each column.   Oh and do that with 40 column 
> line widths. Piece of cake.
> Oh and its a Variable width text font ... ha !
>
> What I tried so far which almost works sorta, just enough to not work 
> well enough :)
> Translate the XML to HTML with XSLT then I downloaded one of the free 
> "HTML to Text" tools which are prevalent on the net.
> It sorta did work, except for the tables.   But I need something that 
> can work in a production environment.   Open Source is prefered,
> but if it does an excellent job then a commercial product could be 
> justified (pun intended).
>
> My next attempt will to try Apache FOP.    I looked at RenderX but 
> they dont claim text output.
> Apache FOP claims a text output but with caveots, the claim is its not 
> robust code with poor results.
> Maybe they are being kind and it really works great.   I will try.
>
> Any other suggestions ?   I can always 'roll my own' but this project 
> is not on the budget yet and were trying to see if we can do something 
> with minimal work.
> Appreciated,
>
> thanks,
>
>
>
>

I have a project where I need to take XML and format it as ascii text.
Sounds simple right ? But alas the requirements are subtle.  It seems 
people dont do text formatting much anymore.
I fondly remember the days of nroff where I could do good text 
formatting and achieve things like word splitting (hyphenating),
justification etc on text content.  In particular a challenge is to 
support multi column tables.  Thats right, text spanning 5 columns,
that needs to wrap in each column.   Oh and do that with 40 column line 
widths. Piece of cake.
Oh and its a Variable width text font ... ha !

What I tried so far which almost works sorta, just enough to not work 
well enough :)
Translate the XML to HTML with XSLT then I downloaded one of the free 
"HTML to Text" tools which are prevalent on the net.
It sorta did work, except for the tables.   But I need something that 
can work in a production environment.   Open Source is prefered,
but if it does an excellent job then a commercial product could be 
justified (pun intended).

My next attempt will to try Apache FOP.    I looked at RenderX but they 
dont claim text output.
Apache FOP claims a text output but with caveots, the claim is its not 
robust code with poor results.
Maybe they are being kind and it really works great.   I will try.

Any other suggestions ?   I can always 'roll my own' but this project is 
not on the budget yet and were trying to see if we can do something with 
minimal work.
Appreciated,

thanks,




-- 
David A. Lee
dlee@calldei.com  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224

btw, I am just curious if another revision of XML spec is planned for
future? If yes, then this request of a better explanation of "XML
validation" might be considered into that.

If no further revisions to XML specs are planned, then it's better to
have this request discarded and we can live with our current
understanding.

Inspite of all of these state of affairs with XML standards, XML is
certainly tremendously successful, and we all know what XML validation
mean and how it relates to DTD, XSD etc :)

I think, this thread ought to get closed now, unless somebody wants to
write anything else.

On Wed, Nov 11, 2009 at 9:42 PM, Mukul Gandhi <gandhi.mukul@gmail.com> wrote:


-- 
Regards,
Mukul Gandhi

I agree that the main reason for XML+CSS not to succeed is simply the
lack of XSLT-like features in CSS. And when you combine CSS with XSLT
or other transformation languages server-side, you can just as well
transform to XHTML/HTML and get the benefits of markup languages the
browsers and other user agents know.

When we come to think about it, it is also not exactly correct to say
that PRINCExml is a great example of XML+CSS. This system has to have
some mechanism of transformation internally, XSLT or some other
programming language.

I will update my "tutorial".

Cheers,
Jesper Tverskov

http://www.xmlkurser.dk
http://www.xmlplease.com

Hi Mike,
   At first thought, and even thinking a bit deeply I have quite a bit
of agreement to your arguments.

But I am trying to be very careful just now, before saying that "I
completely agree with you".

I still have a feeling in my mind, that "DTD validation" and "XSD
validation" both mean validating XML documents (i.e, conceptually DTD
and XSD validations are the same thing), but with different (XML)
Schema languages. I also think, the word validation in XML spec,
conceptually relates similarly to XSD or to any other XML validation
technology, as it means for DTDs.

You have said, "the word valid in the XML specification has a
different meaning from its use in the XSD specification". As I have
said above, unfortunately I am not entirely convinced by this
argumemnt :( But conceptually I agree with you, agreeing with your
argument that XML core specs, do not include the XSD spec.
Unfortunately it seems to me, that W3C XML activity sepcs are
structure in such a way, that XSD spec is separate from the XML core
specs (but the separation of XSD spec from XML core specs, look ok to
me), and that causes the confusion to readers when they read the word
"valid" in the XML spec, and that is referred to DTD only validation.

According to me, the XML spec (I am looking at the 5th edition spec,
of XML 1.0) clearly seems to convey, that XML validation is *only* a
DTD validation. And that is the main confusion to me. Because, in
present context XML validation can be done by other Schema language as
well like XSD, RelaxNG or Schematron.

I don't intend to have the core essence of XML spec changed. I even
like the description of entire DTD language in XML spec, and also
saying that DTD is a XML validation technology, in XML spec. But I
wish that if, a confusion regarding "XML validation" as mentioned in
the XML spec, which seems to be possible only with DTDs can be better
explained in the XML spec, that would be quite helpful, I believe.

Even if the word XSD cannot be mentioned explicitly in the XML spec, I
think a generic reference like "XML Schema" or perhaps "other XML
Schema languages" would look ok to me.

I feel though, that descriptions of DTDs in XML spec should definitely
be retained and being referred as a XML validation technology.

On Wed, Nov 11, 2009 at 2:41 PM, Michael Kay <mike@saxonica.com> wrote:
> When the XML specification uses the word "entity", it means something
> entirely different from what the UML spec means by "entity". That does not
> make the things it says about entities untrue: it just means that the reader
> has to be careful to remember what the meaning of the word is in the context
> of a particular specification. Similarly, the word "valid" in the XML
> specification has a different meaning from its use in the XSD specification.
> Also similarly, the word "processor" in the XML specification means
> something different from "processor" in the XSLT specification. Such
> differences are entirely legitimate. When discussing technology in a wider
> context, it's often wise to qualify your language to avoid misunderstanding,
> by talking of an "XML entity" or an "XML processor" or of documents being
> "DTD-valid". But it would be tedious to do that within a specification,
> where it's hard enough to avoid 6-tuple compound nouns at the best of times.



-- 
Regards,
Mukul Gandhi

Jim Tivy wrote:
> Hi Rick
>
> Just to clarify, I was not confusing a dictionary with a specification. My
> dictionaries are large and rectangular and most specs I read online :)
>
> I think where we may agree is, a spec has language control and many specs
> have glossaries for this.  The point is that the term "valid" and
> "validation" are being used differently across the same standards
> organization within the same family of standards.  So, I think if another
> revision were done of XML 1.0 a change as proposed would be appropriate.
> That is not to say it will be done.  Lots of things are not rewritten
> endlessly - they stay as is, written within the context of the time.
>   
Well, I would quibble that XSD no more in the same family of standards 
as XML than you could regard WSDL as being in the same family. The lumur 
and the bonobo perhaps. It has an infoset (PSVI) that is not the XML 
infoset.

But if you decided that it was important that no standard should have 
particular meanings of general terms, such as validate, character, 
string, element, object, etc. then you would have to replace them with 
specific terms, such as "DTD-valid" and "XSD-valid".  So there would be 
no use of either "valid" (because the term is only used in the 
particular sense) or "XSD-valid" (because it belongs to a different 
spec) in the XML specification. If you think that would prevent 
confusion, go for it! 

Cheers
Rick

To summarize the thread I have made a tutorial about it.

"Styling XML with CSS",
http://www.xmlplease.com/xmlcss

Cheers,
Jesper Tverskov

http://www.xmlkurser.dk
http://www.xmlplease.com

On Wed, Nov 11, 2009 at 10:47 AM, Rick Jelliffe
<rjelliffe@allette.com.au> wrote:
> So there would be no use of
> either "valid" (because the term is only used in the particular sense) or
> "XSD-valid" (because it belongs to a different spec) in the XML
> specification.

The XML 1.0 spec, 5th edition mentions (in the section, "5.1
Validating and Non-Validating Processors):
"Validating processors must, at user option, report violations of the
constraints expressed by the declarations in the DTD, and failures to
fulfill the validity constraints given in this specification.] To
accomplish this, validating XML processors must read and process the
entire DTD and all external parsed entities referenced in the
document."

This particular paragraph clearly conveys, that a XML document is
valid *only* in the context of a DTD. This is no longer true given
that XSD, RelaxNG and Schematron are also standard based XML
validation languages, and most of the world recognizes these newer
Schema languages.

As suggested by Jim Tivy earlier, a generic phrase like "XML Schema"
in the XML spec, while referring to XML validation would be helpful to
readers, in the present context.

> If you think that would prevent confusion, go for it!

yes, I believe this change would prevent this confusion to a large extent.


-- 
Regards,
Mukul Gandhi

On Tue, Nov 10, 2009 at 03:34:08PM +0100, Jesper Tverskov wrote:
> I don't thing that is the full explanation. When looking back, I find
> it incredible to believe, that many of us once thought that "XML
> Browsing", meaning CSS styled homegrown XML, was a promising road to
> follow for webpages.

Some of us were hoping for adoption of a more sophisticated styling
mechanism, such as (what became) XSL-FO.  Unfortunately, it's clear
that, even if there were Web browsers implementing XSL-FO, it's
too difficult for the target audience in the Web design world to use.

Generating (X)HTML seems to me to be the primary way forward for the Web.

Some XML editing environments (XMetal, XXE, etc) use CSS for styles, or
at least something CSS-like, usually without the cascading part. I only
know of one editor (Serna, now with a useable open source version) that
uses XSL-FO for screen display: it was a much harder technical challenge.

I always thought the real goal was to be able to get fairly arbitrary
SGML (or, later, XLM) to display in the Web browser - originally using
a browser plugin such as SoftQuad Panorama or EBT DynaWeb.  I for one
never expected CSS to be widely used for XML, since (1) the cascading
part didn't seem relevent in most cases, and (2) CSS didn't really
seem sophisticated enough for most "serious" publishing use cases.
I don't think that's a weakness of CSS - arbritary XML documents
are well outside its design, is all.

Jesper, your tutorial is almost like an advert for Prince, you could
at least include a link to e.g. FOP :-)

You also ask, Why has XML browsing turned out to be a dead end?
and "answer" it with, It makes more sense to use XHTML, but, XHTML
is based on XML, so in fact XML browsing is far from a dead end.

What is not so widely done is browsing of XML vocabularies that
are not built-in to the most common Web browsers.

For that, as Simon has pointed out, XLink's failure is a problem.
And a large part of its failure is that it never addressed the
"link discovery" or "architectural form" part of its requirements.

Some simple markup such as,
    <?xlink element="person" link="@brother" title=" 'My Brother' " ?>
    <?xlink element="person" link="concat('?ssn=', @ssn)" title="name" ?>
would have gone a long way to changing the world.

You have to look at the whole picture -- search engines, text readers,
javascript, links, forms, styles for screen and for printing, navigation...

Liam

-- 
Liam Quin, W3C XML Activity Lead, http://www.w3.org/People/Quin/
http://www.holoweb.net/~liam/ * http://www.fromoldbooks.org/

I read this with interest, and IMHO the biggest thing "missing" is what 
I think is the fundamental issue of CSS for XML.
That is you cant re-order the elements !   (Please tell me I'm wrong  .. 
but thats my understanding).

If pure CSS is used to render XML in the browser, then the resulting 
HTML HAS to be in the same basic structure as the XML.
Simple case if I have

<part>
   <name>Name</name>
   <code>Code</code>
</part>


And I want it to render like
       <P> <B>Code</B> - Name </P>

I don't believe there is any way to do this in CSS.


This, IMHO, is the fundamental limitation of CSS.  The XML has to be 
structured how you want it displayed, all the CSS can do is add some 
"styling" but it cant restructure the document.  One of the big "goals" 
of XML  (IMHO) is that it can be data centric, not display centric.  
This makes CSS rendering a data oriented XML document severely 
crippled.  It can only do so much - namely add "style" to an element, 
not reformat the document.

(again, please tell me I'm wrong ! - but this is what the CSS gurus at 
work tell me as to why I cant send them XML directly to display in a 
mobile browser).


David A. Lee
dlee@calldei.com  
http://www.calldei.com
http://www.xmlsh.org
812-482-5224



Jesper Tverskov wrote:
> To summarize the thread I have made a tutorial about it.
>
> "Styling XML with CSS",
> http://www.xmlplease.com/xmlcss
>
> Cheers,
> Jesper Tverskov
>
> http://www.xmlkurser.dk
> http://www.xmlplease.com
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>   

>> Sorry for wasting anybody's time.
>
> arguments are never a waste of time.
> The list would be fairly pointless without them.
>

Just to confirm: I greatly enjoyed this whole thread, it simply was great.

Thanks to all, and especially to Rick Jelliffe.


Cheers,
Dimitre Novatchev.

On Mon, Nov 9, 2009 at 2:44 AM, David Carlisle <davidc@nag.co.uk> wrote:
>
>
>> Sorry for wasting anybody's time.
>
> arguments are never a waste of time.
> The list would be fairly pointless without them.
>
> David
>
> ________________________________________________________________________
> The Numerical Algorithms Group Ltd is a company registered in England
> and Wales with company number 1249803. The registered office is:
> Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.
>
> This e-mail has been scanned for all viruses by Star. The service is
> powered by MessageLabs.
> ________________________________________________________________________
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>
>



-- 
Cheers,
Dimitre Novatchev
---------------------------------------
Truly great madness cannot be achieved without significant intelligence.
---------------------------------------
To invent, you need a good imagination and a pile of junk
-------------------------------------
Never fight an inanimate object
-------------------------------------
You've achieved success in your field when you don't know whether what
you're doing is work or play
-------------------------------------
I enjoy the massacre of ads. This sentence will slaughter ads without
a messy bloodbath.

On Wed, Nov 11, 2009 at 10:49 AM, David A. Lee <dlee@calldei.com> wrote:
> I read this with interest, and IMHO the biggest thing "missing" is what I
> think is the fundamental issue of CSS for XML.
> That is you cant re-order the elements !   (Please tell me I'm wrong  .. but
> thats my understanding).
>

I have to ask, why do you want to reorder the elements?  That's partly
what XSLT is for, but browser support can be unpredictable.  However,
I think what you really want to do is change the position they are
given on the screen (/page)?  If so, yes CSS can place any element in
any (mostly arbitrary) position.  However, that requires absolute
positioning which can be a real pain, in particular for things like
resizable pages.  The latter problem can somewhat be addressed by
using "em" (characters) as your positioning units instead of the more
ubiquitous "px" (pixels), but getting predictable cross browser
results is tedious at best and often impossible for anything
non-trivial...

-- 
Peter Hunsberger

Jesper Tverskov wrote:
> To summarize the thread I have made a tutorial about it.
> 
> "Styling XML with CSS",
> http://www.xmlplease.com/xmlcss

That may be the most biased "don't do that" tutorial I've seen.  I don't 
even think my XML Schema tutorials were that harsh on the material they 
purported to teach...  and there's not much discussion of the core 
display CSS property that makes it work.

I'd recommend describing it as something other than a tutorial, and 
perhaps change the title.  There's plenty of room for articles that 
illustrate gaps, but readers tend to react more happily to material 
whose title indicates gap-finding.

-- 
Simon St.Laurent
http://simonstl.com/

On Nov 11, 2009, at 8:49 AM, David A. Lee wrote:

> I read this with interest, and IMHO the biggest thing "missing" is what I think is the fundamental issue of CSS for XML.
> That is you cant re-order the elements !   (Please tell me I'm wrong  .. but thats my understanding).
> 
> If pure CSS is used to render XML in the browser, then the resulting HTML HAS to be in the same basic structure as the XML.
> Simple case if I have
> 
> <part>
> <name>Name</name>
> <code>Code</code>
> </part>
> 
> 
> And I want it to render like
>     <P> <B>Code</B> - Name </P>
> 
> I don't believe there is any way to do this in CSS.

How about

part {
display: block;
}

name {
display:inline;
float: right;
}

code: {
display:inline;
float: left;
font-weight: bold;
}

untested :)

-Rob



> 
> 
> This, IMHO, is the fundamental limitation of CSS.  The XML has to be structured how you want it displayed, all the CSS can do is add some "styling" but it cant restructure the document.  One of the big "goals" of XML  (IMHO) is that it can be data centric, not display centric.  This makes CSS rendering a data oriented XML document severely crippled.  It can only do so much - namely add "style" to an element, not reformat the document.
> 
> (again, please tell me I'm wrong ! - but this is what the CSS gurus at work tell me as to why I cant send them XML directly to display in a mobile browser).
> 
> 
> David A. Lee
> dlee@calldei.com  http://www.calldei.com
> http://www.xmlsh.org
> 812-482-5224
> 
> 
> 
> Jesper Tverskov wrote:
>> To summarize the thread I have made a tutorial about it.
>> 
>> "Styling XML with CSS",
>> http://www.xmlplease.com/xmlcss
>> 
>> Cheers,
>> Jesper Tverskov
>> 
>> http://www.xmlkurser.dk
>> http://www.xmlplease.com
>> 
>> _______________________________________________________________________
>> 
>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>> to support XML implementation and development. To minimize
>> spam in the archives, you must subscribe before posting.
>> 
>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>> subscribe: xml-dev-subscribe@lists.xml.org
>> List archive: http://lists.xml.org/archives/xml-dev/
>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>> 
> 
> _______________________________________________________________________
> 
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
> 
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> 

> This particular paragraph clearly conveys, that a XML 
> document is valid *only* in the context of a DTD. This is no 
> longer true given that XSD, RelaxNG and Schematron are also 
> standard based XML validation languages, and most of the 
> world recognizes these newer Schema languages.

When the XML specification uses the word "entity", it means something
entirely different from what the UML spec means by "entity". That does not
make the things it says about entities untrue: it just means that the reader
has to be careful to remember what the meaning of the word is in the context
of a particular specification. Similarly, the word "valid" in the XML
specification has a different meaning from its use in the XSD specification.
Also similarly, the word "processor" in the XML specification means
something different from "processor" in the XSLT specification. Such
differences are entirely legitimate. When discussing technology in a wider
context, it's often wise to qualify your language to avoid misunderstanding,
by talking of an "XML entity" or an "XML processor" or of documents being
"DTD-valid". But it would be tedious to do that within a specification,
where it's hard enough to avoid 6-tuple compound nouns at the best of times.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

> >   
> Well, I would quibble that XSD no more in the same family of 
> standards as XML than you could regard WSDL as being in the 
> same family. 

If I were asked what the term "family of standards" means, then I would say
that it means a set of standards that come from the same organization and
are planned and coordinated as a unit. XML and XSD are both within the
purview of the XML Activity within W3C, so in that sense they are most
definitely part of the same family in a way that WSDL is not. On the other
hand, XSD is not part of the "XML Core" (which includes XML, Namespaces,
XInclude, Infoset, and XML Base, as well as (somewhat bizarrely) the spec
for <?xml-stylesheet?>. So perhaps XML Core is the nuclear family, while
XSD, XSLT, XQuery, and XProc are part of the extended family. Then WSDL is a
second cousin once removed, while RelaxNG is a next-door neighbour. 

Michael Kay

PsychoPath 1.1M3 a Java 5 (1.5) or greater XPath 2.0 Schema Aware 
processor is now available for download.

It is packaged in two forms as standalone jar and as part of an eclipse 
P2 update site.  It does depend on java_cup and xerces (for xml schema 
support).

https://build.eclipse.org/hudson/view/WTP/job/cbi-wtp-wst.xsl.psychopath/82/

PsychoPath is also included in the latest WTP 3.2 Milestone 3 build as well:

http://download.eclipse.org/webtools/downloads/drops/R3.2.0/S-3.2.0M3-20091106000626/

PsychoPath now has a pass rating of 99.6% of the W3C XPath 2.0 portion 
of the XQuery Test suite. It is fully schema aware including the ability 
to handle user defined data types.   There are about 24 tests left to go 
to bring it into full compliance with the test suite.

PsychoPath provides the only known open source Java XPath 2.0 Schema
Aware processor. User Manual and sample code can be found at:

http://wiki.eclipse.org/PsychoPathXPathProcessor/UserManual

Thanks to Andrea Bittau for the original code donation. Current
development is done by Dave Carver, Mukul Gandhi, and Jesper Moller.

Hi Rick

Just to clarify, I was not confusing a dictionary with a specification. My
dictionaries are large and rectangular and most specs I read online :)

I think where we may agree is, a spec has language control and many specs
have glossaries for this.  The point is that the term "valid" and
"validation" are being used differently across the same standards
organization within the same family of standards.  So, I think if another
revision were done of XML 1.0 a change as proposed would be appropriate.
That is not to say it will be done.  Lots of things are not rewritten
endlessly - they stay as is, written within the context of the time.

Jim

-----Original Message-----
From: Rick Jelliffe [mailto:rjelliffe@allette.com.au] 
Sent: Monday, November 09, 2009 11:09 PM
To: xml-dev@lists.xml.org
Subject: Re:  XML spec and XSD

Jim Tivy wrote:
>
> The above definition of the term "valid" is not really common Xml usage
> anymore. I have seen the term "valid" used when a document has no DTD and
> "validates" against an XML schema - for instance Oxygen uses this term -
or
> Xml Schema Primer says:
>   
A standard does not reflect common usage. It defines its own little 
world of tight technical terms.  You are thinking of a dictionary perhaps?

Cheers
Rick Jelliffe

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

Jim Tivy wrote:
>
> The above definition of the term "valid" is not really common Xml usage
> anymore. I have seen the term "valid" used when a document has no DTD and
> "validates" against an XML schema - for instance Oxygen uses this term - or
> Xml Schema Primer says:
>   
A standard does not reflect common usage. It defines its own little 
world of tight technical terms.  You are thinking of a dictionary perhaps?

Cheers
Rick Jelliffe

Simon St.Laurent wrote:

> Screen reader
> developers haven't been especially enthusiastic about interpreting the
> information that CSS provides, but it's possible.  Perhaps more practical
> would be CSS properties that map to a limited number of screen-reader types,
> though that's a conversation that goes around and around...

Yes, at least in theory, a screen reader ought to be able to interpret
what CSS actually does: When the meaning of markup the CSS is applied
to is known. But if the meaning of the markup is unknown to the screen
reader, as would most often be the case with some homegrown XML, I
think it would be quite a challenge.

> None of these are impossible barriers.  XML just never achieved the momentum
> in this space to make them worth climbing.

I don't thing that is the full explanation. When looking back, I find
it incredible to believe, that many of us once thought that "XML
Browsing", meaning CSS styled homegrown XML, was a promising road to
follow for webpages.

I believe that I'm pretty good at CSS, but I have not the slightest
idea of how I could style some XML to create a table with rows and
columns, collspan, borders and shades, table headers, tbody, etc. Nor
do I have the desire to solve such problems. Next time around with
some other XML, I should reinvent the wheel one more time? Should all
web developers really work like that, when it is easy to transform XML
not made for display into XHTML made to make display easy?

Cheers,

Jesper Tverskov

http://www.xmlkurser.dk
http://www.xmlplease.com

Simon wrote: 

> I've never found the display properties to be that difficult, but then I 
> rarely work with the kinds of tables that make designers crazy.  (CALS 
> table models anyone?)  If only IE bothered to support them...

You mean as in:

http://msdn.microsoft.com/en-us/library/cc304082%28VS.85%29.aspx


Internet Explorer 8 is fully compliant with the Cascading Style Sheets
(CSS), Level 2 Revision 1 (CSS2.1) specification and supports some
features of CSS, Level 3 (CSS3). 

....

The display attribute now supports the full range of table layout values
from the W3C Cascading Style Sheets Level 2.1 Specification .  

....

David

________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

On Tue, Nov 10, 2009 at 12:39 PM, Rick Jelliffe
<rjelliffe@allette.com.au> wrote:
> A standard does not reflect common usage. It defines its own little world of
> tight technical terms.  You are thinking of a dictionary perhaps?

I agree, that standards are very specific. Standards have their own
notion of correctness :) But if something, that's said in the standard
deviates from the reality, then the standard should modify itself.
Standard revisions and erratas achieve this purpose.

We are not asking to invent something different with the XML spec. We
are I think, only asking the XML spec to express the reality.


-- 
Regards,
Mukul Gandhi

On Nov 9, 2009, at 15:10, Simon St.Laurent wrote:

> More formally, I've heard many good things about Prince, which  
> generates PDF from XML and CSS:
>
> http://www.princexml.com/overview/

Indeed, CSS makes sense for XML in Prince, since the print media is  
visual anyway. However, on the Web if you use an XML vocabulary whose  
semantics are unknown to the browser and make it look sensible using  
CSS, the user experience with screen readers suffers compared to using  
a vocabulary with semantics that the browser knows about (i.e. HTML).

-- 
Henri Sivonen
hsivonen@iki.fi
http://hsivonen.iki.fi/

On Aug 27, 2009, at 16:49, Simon St.Laurent wrote:

> Simple links are supported in a few browsers, but that particular  
> dream faded.

Support is on its way out:
https://bugzilla.mozilla.org/show_bug.cgi?id=516906
https://bugzilla.mozilla.org/show_bug.cgi?id=515494

The non-support on MathML elements is an accident. Most of the SVG  
stuff doesn't really use XLink facilities but uses the same attribute  
names, which pretty much shows how a generic mechanism is a failure in  
practice.

-- 
Henri Sivonen
hsivonen@iki.fi
http://hsivonen.iki.fi/

David Carlisle wrote:
>> I've never found the display properties to be that difficult, but then I 
>> rarely work with the kinds of tables that make designers crazy.  (CALS 
>> table models anyone?)  If only IE bothered to support them...
> 
> You mean as in:
> 
> http://msdn.microsoft.com/en-us/library/cc304082%28VS.85%29.aspx
> 
> Internet Explorer 8 is fully compliant with the Cascading Style Sheets
> (CSS), Level 2 Revision 1 (CSS2.1) specification and supports some
> features of CSS, Level 3 (CSS3). 
> 
> ....
> 
> The display attribute now supports the full range of table layout values
> from the W3C Cascading Style Sheets Level 2.1 Specification .  

That's what I meant, yes.  Some days it's great to be wrong!

-- 
Simon St.Laurent
http://simonstl.com/

Sun, 8 Nov 2009 09:52:36 +0530 Mukul Gandhi <gandhi.mukul@gmail.com> might have said:
> My averment was to give XSD the same status as DTD, as a XML
> validation technology in W3C XML spec.

I think schema languages are more like XSL than a DTD. A document is supposed to have a DTD, whereas it doesn't really "have a schema". You can apply any number of schemas to an XML document for different purposes. Each schema is not the one and only description of the document. You can apply W3C XML schema to an XML document to produce a PSVI.

In any case, if someone does end up adding a reference to a schema language to the XML spec, I don't think W3C XML Schema should be given preferred status.

Kendall

Jesper Tverskov wrote:
>> None of these are impossible barriers.  XML just never achieved the momentum
>> in this space to make them worth climbing.
> 
> I don't thing that is the full explanation. When looking back, I find
> it incredible to believe, that many of us once thought that "XML
> Browsing", meaning CSS styled homegrown XML, was a promising road to
> follow for webpages.

I don't find it that incredible, but then I was fond of CSS and looking 
for a path out of tag soup.

> I believe that I'm pretty good at CSS, but I have not the slightest
> idea of how I could style some XML to create a table with rows and
> columns, collspan, borders and shades, table headers, tbody, etc. Nor
> do I have the desire to solve such problems. Next time around with
> some other XML, I should reinvent the wheel one more time? Should all
> web developers really work like that, when it is easy to transform XML
> not made for display into XHTML made to make display easy?

I've never found the display properties to be that difficult, but then I 
rarely work with the kinds of tables that make designers crazy.  (CALS 
table models anyone?)  If only IE bothered to support them...

Colspan is the only piece I see in your list that actually looks 
difficult, and I suspect that's mostly because I haven't kept up with 
the latest in CSS.  I don't find CSS reuse particularly difficult, even 
with changing vocabularies, either, though again, I rarely tread into 
intensely detailed layouts.

On the other hand, I've never called XSLT 'easy', even with excellent 
training.

-- 
Simon St.Laurent
http://simonstl.com/

Henri Sivonen wrote:
> On Aug 27, 2009, at 16:49, Simon St.Laurent wrote:
>> Simple links are supported in a few browsers, but that particular 
>> dream faded.
> 
> Support is on its way out:
> https://bugzilla.mozilla.org/show_bug.cgi?id=516906
> https://bugzilla.mozilla.org/show_bug.cgi?id=515494
> 
> The non-support on MathML elements is an accident. Most of the SVG stuff 
> doesn't really use XLink facilities but uses the same attribute names, 
> which pretty much shows how a generic mechanism is a failure in practice.

I'd be cautious about using XLink to generalize about the failure of 
generic mechanism, given the large number of other problems it had, both 
technical and cultural.

However, it does seem clear that this dream is not merely faded, but dead.

-- 
Simon St.Laurent
http://simonstl.com/

Jesper Tverskov wrote:
> I think PrinceXML is a good example of when XML+CSS can be useful. But
> except that we have files like RSS, I find it a bad idea to use CSS to
> style XML for the web to make it look like HTML/XHTML.
> 
> The browsers don't have a default stylesheet to use. When you use
> XHTML you only need to style what you want to be different from the
> default stylesheet in the browser. When styling your own XML you must
> style everything yourself.

Given the popularity of 'reset' stylesheets in HTML, I'm not sure that's 
actually a problem for the kinds of developers who'd be likely to use 
XML this way.  "Styling everything yourself" is a feature, not a bug, 
for many people.

> Also search engines benefit from a known markup language like XHTML:
> Headings, paragraphs, lists, tables, making it easier to put special
> importance to content, judged from how it is marked-up. Using your own
> markup, it is much more difficult for a search engine like Google to
> tell headings, tables, lists, etc, apart.

It is indeed more difficult, though of course the irony is that one of 
the early promises of XML was that it would make more sophisticated 
search easier.

(Though in Google's case, I'm not sure how much HTML semantics it's 
really using - originally the answer was "almost none, just text and 
links" because of people gaming search engines, but I understand that's 
changed over time.)

> Also screen readers used by blind people, etc, read the source code.
> Homegrown markup, making sense when displayed with CSS in a browser,
> is meaningless in a screen reader not knowing when to announce
> "title", "ordered list of 5 items", heading, table, etc, to make a
> webpage easier to understand when read aloud.

This is true and important but not difficult to solve.  Screen reader 
developers haven't been especially enthusiastic about interpreting the 
information that CSS provides, but it's possible.  Perhaps more 
practical would be CSS properties that map to a limited number of 
screen-reader types, though that's a conversation that goes around and 
around...

None of these are impossible barriers.  XML just never achieved the 
momentum in this space to make them worth climbing.

-- 
Simon St.Laurent
http://simonstl.com/

Hi Mukul

The line from the XML 1.0 5th edition spec

[Definition: An XML document is valid if it has an associated document type
declaration and if the document complies with the constraints expressed in
it.]

The above definition of the term "valid" is not really common Xml usage
anymore. I have seen the term "valid" used when a document has no DTD and
"validates" against an XML schema - for instance Oxygen uses this term - or
Xml Schema Primer says:

(1) it checks for conformance to the rules, a process called schema
validation,

So Mukul's suggestion of having a forward reference using "such as" text is
definately an improvement - for example:

An XML document is valid if it has an associated document type declaration
and if the document complies with the constraints expressed in it.  As well,
the validity of an XML instance can be determined with the use of a document
type declaration if using other validation mechanisms such as XML Schema.]

There have been previous discussions - see XML entry in Wikipedia - where
people have expressed a need to see DTD's special status removed with
respect to XML.

So perhaps we can close the discussion on that note :)

Jim



-----Original Message-----
From: Mukul Gandhi [mailto:gandhi.mukul@gmail.com] 
Sent: Monday, November 09, 2009 2:41 AM
To: David Carlisle
Cc: xml-dev@lists.xml.org
Subject: Re:  XML spec and XSD

I think, I need to give up with my arguments in this thread. The
concepts now seem to go into my head. Nobody so far seem to agree with
my arguments. It seems, I was wrong.

Sorry for wasting anybody's time.

On Mon, Nov 9, 2009 at 3:34 PM, David Carlisle <davidc@nag.co.uk> wrote:
>> The spec seems to suggest that XML documents are valid only if a DTD
>> is associated with the XML document.
>
> That is not a "suggestion". valid is a technical term defined in the XML
> specification and that's a necessary condition for validity.
>
>> I don't mind validating XML documents with other technologies like
>> RelaxNG or Schematron, if there is a need for me to do so. But I
>> think, existence of technologies like RelaxNG or Schematron cannot be
>> an excuse for XML spec, not to mention XSD within it as a validating
>> technology.
>
> Whether or not a document validates against an XSD or schematron schema
> has no bearing on whether or not it is a valid as opposed to merely well
> formed XML document. You seem to want "valid" to mean validated against
> any schema language of your choice, that's OK (but confusing) if used in
> conversation, but it is not the way the term "valid" is defined in XML.
>
>
> XSD, RelaxNG, Schematron all depend on XML; not the other way round. They
> require normative references to XML, it would be entirely backwards (or
> worse, circular) for XML to reference them.
>
> David



-- 
Regards,
Mukul Gandhi

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

I think, I need to give up with my arguments in this thread. The
concepts now seem to go into my head. Nobody so far seem to agree with
my arguments. It seems, I was wrong.

Sorry for wasting anybody's time.

On Mon, Nov 9, 2009 at 3:34 PM, David Carlisle <davidc@nag.co.uk> wrote:
>> The spec seems to suggest that XML documents are valid only if a DTD
>> is associated with the XML document.
>
> That is not a "suggestion". valid is a technical term defined in the XML
> specification and that's a necessary condition for validity.
>
>> I don't mind validating XML documents with other technologies like
>> RelaxNG or Schematron, if there is a need for me to do so. But I
>> think, existence of technologies like RelaxNG or Schematron cannot be
>> an excuse for XML spec, not to mention XSD within it as a validating
>> technology.
>
> Whether or not a document validates against an XSD or schematron schema
> has no bearing on whether or not it is a valid as opposed to merely well
> formed XML document. You seem to want "valid" to mean validated against
> any schema language of your choice, that's OK (but confusing) if used in
> conversation, but it is not the way the term "valid" is defined in XML.
>
>
> XSD, RelaxNG, Schematron all depend on XML; not the other way round. They
> require normative references to XML, it would be entirely backwards (or
> worse, circular) for XML to reference them.
>
> David



-- 
Regards,
Mukul Gandhi

I think PrinceXML is a good example of when XML+CSS can be useful. But
except that we have files like RSS, I find it a bad idea to use CSS to
style XML for the web to make it look like HTML/XHTML.

The browsers don't have a default stylesheet to use. When you use
XHTML you only need to style what you want to be different from the
default stylesheet in the browser. When styling your own XML you must
style everything yourself.

Also search engines benefit from a known markup language like XHTML:
Headings, paragraphs, lists, tables, making it easier to put special
importance to content, judged from how it is marked-up. Using your own
markup, it is much more difficult for a search engine like Google to
tell headings, tables, lists, etc, apart.

Also screen readers used by blind people, etc, read the source code.
Homegrown markup, making sense when displayed with CSS in a browser,
is meaningless in a screen reader not knowing when to announce
"title", "ordered list of 5 items", heading, table, etc, to make a
webpage easier to understand when read aloud.

Cheers,
Jesper Tverskov

> Sorry for wasting anybody's time.

arguments are never a waste of time.
The list would be fairly pointless without them.

David

________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

> The spec seems to suggest that XML documents are valid only if a DTD
> is associated with the XML document.

That is not a "suggestion". valid is a technical term defined in the XML
specification and that's a necessary condition for validity.

> I don't mind validating XML documents with other technologies like
> RelaxNG or Schematron, if there is a need for me to do so. But I
> think, existence of technologies like RelaxNG or Schematron cannot be
> an excuse for XML spec, not to mention XSD within it as a validating
> technology.

Whether or not a document validates against an XSD or schematron schema
has no bearing on whether or not it is a valid as opposed to merely well
formed XML document. You seem to want "valid" to mean validated against
any schema language of your choice, that's OK (but confusing) if used in
conversation, but it is not the way the term "valid" is defined in XML.


XSD, RelaxNG, Schematron all depend on XML; not the other way round. They
require normative references to XML, it would be entirely backwards (or
worse, circular) for XML to reference them.

David

________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

> 
> Apart from the above examples, how common is it to actually 
> style XML with CSS? On the web and inside "systems"?
> 

I haven't seen it done much in anger. It's an attractive option at first
sight, but has the disadvantage that when you hit the limits of what you can
do with CSS, you have to throw everything away and switch to a different
technology.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

Hi list

All text books about XML have a pretty long chapter about how to style
XML with CSS. Almost all new browsers do that when we click on an XML
file showing a nicely indented XML tree. Most browsers also have some
JavaScript+CSS presenting RSS XML documents as if they were webpages.

Apart from the above examples, how common is it to actually style XML
with CSS? On the web and inside "systems"?

If we want to display XML in a browser, it seems much more natural and
have many benefits, to transform that XML to a well-known and
well-supported XML Application like XHTML made for the purpose.

Cheers,
Jesper Tverskov

http://www.xmlkurser.dk
http://www.xmlplease.com

Thanks, David for the insights.

looking at the following section of XML 1.0 fifth edition
specification, http://www.w3.org/TR/2008/REC-xml-20081126/#proc-types

"Validating and Non-Validating Processors"
The spec seems to suggest that XML documents are valid only if a DTD
is associated with the XML document.
But I think, that's not entirely true given that XSD is also another
XML validation language from W3C XML activity.

I don't mind validating XML documents with other technologies like
RelaxNG or Schematron, if there is a need for me to do so. But I
think, existence of technologies like RelaxNG or Schematron cannot be
an excuse for XML spec, not to mention XSD within it as a validating
technology.

On Sun, Nov 8, 2009 at 9:01 PM, David Carlisle <davidc@nag.co.uk> wrote:
> essentially for the same reason that XSD should not be referenced from
> the xml spec. XSD (like Relax NG, XSLT etc) all work on the infoset
> produced by parsing an XML document, thus they require an XML parser to
> provide them input, to expand entities etc.
>
> DTD processing on the other hand does not work by taking an infoset as
> input but is instead a modified parser that reports validation outcomes.
>
> XSD can not define entities as it is too late in the chain. If the
> document uses entity references that are not defined to the parser then
> the document is in error (or not well formed if there is no external DTD
> reference at all).



-- 
Regards,
Mukul Gandhi

Jesper Tverskov wrote:
> All text books about XML have a pretty long chapter about how to style
> XML with CSS. Almost all new browsers do that when we click on an XML
> file showing a nicely indented XML tree. Most browsers also have some
> JavaScript+CSS presenting RSS XML documents as if they were webpages.
> 
> Apart from the above examples, how common is it to actually style XML
> with CSS? On the web and inside "systems"?

It's not very common, though it does get use.  I've used CSS many times 
to make it easier to explore an XML document without the pointy brackets 
or hierarchy.

More formally, I've heard many good things about Prince, which generates 
PDF from XML and CSS:

http://www.princexml.com/overview/

The main problem I had early on in displaying XML with CSS on the public 
Web was Internet Explorer's lack of support for the display properties, 
apparently for performance reasons, which has since improved.

I wrote a now out-of-date series of articles on XML and CSS for XML.com 
back in 2000.  The summary table does a nice job of describing the state 
of the art at the last moment web developers actually seemed interested 
in using XML directly:

http://www.xml.com/pub/a/2000/05/03/browserchart/index.html

The failure of XLink is, I think, the other reason XML didn't get 
anywhere on the public web.  My XLink talks were frequently my most 
popular, full of developers looking for better hypertext possibilities. 
They generally left with a big but unfulfilled wishlist.

> If we want to display XML in a browser, it seems much more natural and
> have many benefits, to transform that XML to a well-known and
> well-supported XML Application like XHTML made for the purpose.

If you need transformations to reorder your document, make links work, 
or perform calculations, this is completely true.  Otherwise, it's a 
fairly tremendous waste of time.

-- 
Simon St.Laurent
http://simonstl.com/

David Carlisle wrote:
>> Sorry for wasting anybody's time.
> 
> arguments are never a waste of time.
> The list would be fairly pointless without them.

Indeed.  And it's often only through conversation that we can figure out 
what we really think!

That was definitely not a waste of time.

-- 
Simon St.Laurent
http://simonstl.com/

> The spec seems to suggest that XML documents are valid only if a DTD
> is associated with the XML document.

Right.  In the XML spec, valid has a very string meaning and it refers 
only to DTD's.

> But I think, that's not entirely true given that XSD is also another
> XML validation language from W3C XML activity.

No, it is true.  You might notice some people using the term schema-valid 
to describe a document that meets XSD schema validation.  This is why they 
(we:) do that.

        /r$

--
STSM, WebSphere Appliance Architect
https://www.ibm.com/developerworks/mydeveloperworks/blogs/soma/

Mukul Gandhi wrote:

> My point was, that XSD and DTD (and also XML) are "W3C" specs. RelaxNG
> is an OASIS specification. I haven't seen W3C specs, recommendating
> users to use OASIS specs, *and* mentioning these statements in W3C
> specs themselves. Or do we have any instances of W3C specs, where W3C
> recommends users to use certain OASIS specs. This more so doesn't
> happen if W3C and OASIS develop competing languages. But I don't see
> XSD and RelaxNG as competing to each other.

If you will look at newer document formats specified by W3C they use
RELAX NG as a primaty schema language -- simply because in RELAX NG it
is easier to express necessary constraints then in W3C XML Schema (and
sometimes RELAX NG is the only grammar based schema language in which
constraints can be specied). Examples are MathML 3, SVG 1.2, XHTML 2.0
was also using RELAX NG.

If you look at validator.nu and Relaxed -- the two validators who have
much better error coverage then validator.w3.org for web pages, you will
find that they are based on RELAX NG schemas + (Schematron or special
check written in procedural code).

> Looking at the XML spec, the XML spec seems to convey that DTD is an
> inseparable part of XML definition. But if DTD is inseparable from
> XML, why cannot XML spec say that XSD can be used for validation as
> well (since XSD is a better XML validation technology, and is also
> developed by W3C)?

Standard are books of code not textbooks.

> I even think, that a future XML spec other than referring to XSD as an
> alternative XML validation technology with DTDs, should also provide
> means to embed XML Schema's inline the XML documents, as we do with
> DTDs.

There is already spec which provides this capability -- XML Namespaces.

-- 
------------------------------------------------------------------
  Jirka Kosek      e-mail: jirka@kosek.cz      http://xmlguru.cz
------------------------------------------------------------------
       Professional XML consulting and training services
  DocBook customization, custom XSLT/XSL-FO document processing
------------------------------------------------------------------
 OASIS DocBook TC member, W3C Invited Expert, ISO JTC1/SC34 member
------------------------------------------------------------------

OpenPGP digital signature

Mukul Gandhi wrote:
> My point was, that XSD and DTD (and also XML) are "W3C" specs. RelaxNG
> is an OASIS specification. I haven't seen W3C specs, recommendating
> users to use OASIS specs, *and* mentioning these statements in W3C
> specs themselves. Or do we have any instances of W3C specs, where W3C
> recommends users to use certain OASIS specs. This more so doesn't
> happen if W3C and OASIS develop competing languages. But I don't see
> XSD and RelaxNG as competing to each other.
>
>   
W3C SVG for example uses RELAX NG not XSD.
                      http://www.w3.org/TR/SVGTiny12/

> My simple point was, that XML spec mentions DTD as a validation
> technology but it doesn't mention today XSD it's spec. And XSD is
> another validation technology from W3C, which is vastly better than
> DTD and is a pretty stable language now.
>   
Pretty stable? How many full implementations are there of XSD 1.1 today 
(apart from Michael's?)  

Microsoft isn't supporting it AFAIK.   Libxml's front page says it only 
has a partial implementation of XSD 1.0.  And Mukul's Xerces release 
won't be out until December, will it, hence the spruiking I guess?

>> Just like picking a programming language you
>> are free to pick the validation method you want to use.
>>     
>
> I agree. But my point as I have been saying in this thread, was that
> XML mentions DTD, but it doesn't mention XSD. This gives a naive user
> a feeling, that DTD is something which is integral to XML, while XSD
> is not. I believe, this should be corrected in XML spec.
>   
Err...the naive user is correct. The entity mechanism is integrated in 
XML, it is not a severable layer.  Entities are declared with the DTD 
mechanism; XSD provides no alternative to this.
 
Cheers
Rick Jelliffe

Len Bullard wrote:
> No.  I think it was a way of saying something simple and familiar will work
> until something more precise and powerful can replace it.  It was a
> stabilizer in a time of incredible shifts and unpredictable fortunes.  It's
> clear that validity is not intrinsic to markup functionally.  It is just as
> clear that validation is useful to applications.
>   
DTDs provide a convenient summary and test of a document; they allowed 
and encouraged a basic level of test-driven development in an area with 
a lot of cowboy programmers.

When the summary becomes larger than the document, and the test becomes 
more difficult to debug than to do by hand, then DTDs are not much use. 
Which is why so many small XML documents don't have them.

But you need a fairly large document for an XSD schema to be useful to 
programmers as a summary, and you need a very complex document before 
the schema is less difficult to install and debug than just inspecting 
the documents or writing your own tests.

The main problem with DTDs is not that they are flawed (they are nice 
simple, modest, inoffensive little things), but that that DTDs limited 
the expectations of the initial XSD Schema WG, many of whom had never 
written a schema or DTD IIRC*, about what a schema was supposed to do.  
Questions that could have been asked early include "Can this support 
HTML?", "Can this support SVG?", "Can this support RDF?", "Can this 
support XSLT?", "Can this support RSS?"   Instead, what we got was "Can 
this support Postal Addresses?"

Cheers
Rick Jelliffe

* When I was on the XSD Working Group (a decade ago), any reference I 
made to "idiomatic usages of XML" seemed to be met with utter 
incomprehension and dismissal, as if it were an eccentric concern.  
Consequently, at the last minute, when XSD was finally tested against 
real schemas such as HTML, it was found to need revisions: the whole 
complex-type derivation system was found inadequate and the <redefine> 
escape hatch had to be provided, for example. Now, after 10 years, we 
can see some recognition of this, in that XSD 1.1 has some idea that 
attributes might actually be used to make decisions on content. (Of 
course, instead of the neat RELAX NG way, XSD implements it by tacking 
it on outside content models, the chance for more bogus complexity being 
too irresistable, I suppose.) 

 

Mukul Gandhi wrote:
> Since DTD is already mentioned in the XML spec, why not specify XSD as
> well as a normative reference to a future XML spec?  
Because XSD is complete crap that utterly betrayed the premises and 
goals of XML perhaps?
> I think, mentioning XSD in the XML spec will encourage users to use
> XSD as a better validation technology, than DTD. Mentioning this as a
> reference link in XML spec, looks like a very minor and non disruptive
> addition to me, for the XML spec
The way to use XSD as a better validation technology is to learn from 
it, rather than encourage it.

Cheers
Rick

On Sun, Nov 8, 2009 at 2:51 PM, Rick Jelliffe <rjelliffe@allette.com.au> wrote:
> W3C SVG for example uses RELAX NG not XSD.
>                     http://www.w3.org/TR/SVGTiny12/

Thanks for the reference.

> Pretty stable? How many full implementations are there of XSD 1.1 today
> (apart from Michael's?)

It seems, Xerces-J' s XSD 1.1 implementation is also on the way.

I also feel, number of complete implementations of a spec is not the
only measure of usefulness of any language. The language being much
complex and providing lot of functionality, in fact I think raises the
bar on quality and skills of implementers.

> And Mukul's Xerces release won't be
> out until December, will it, hence the spruiking I guess?

I am just one of the persons, who has contributed somewhat to Xerces-J
XSD 1.1 implementation. It's not fair to say, that Xerces is mine :(
But I like Xerces very much.

> Err...the naive user is correct. The entity mechanism is integrated in XML,
> it is not a severable layer.  Entities are declared with the DTD mechanism;
> XSD provides no alternative to this.

I am really not sure, why entities are not allowed to be defined in
XSD. I believe, there is some design principle behind that, and also
the XML Schema WG must be aware about this issue I believe.


-- 
Regards,
Mukul Gandhi

On Sun, Nov 8, 2009 at 2:10 PM, Rick Jelliffe <rjelliffe@allette.com.au> wrote:
> Because XSD is complete crap that utterly betrayed the premises and goals of
> XML perhaps?

Rick, I do not know why you condemn XSD so much :) I feel, XSD doesn't
deserve so much condemnation.
If somebody doesn't like certain technologies for whatever reasons,
they should simply not use it, instead of saying words like "crap",
"poisonous" and so on.
There are individuals and organizations who have written
implementations for XSD, and I believe uncountable numbers of XML
instances get validated each day with XSD. Certainly, innumerable
number of people who are currently using XSD, do not lack good
judgement I believe :)

> The way to use XSD as a better validation technology is to learn from it,
> rather than encourage it.

I don't agree that, a technology from W3C (viz XSD) which is heavily
deployed currently, and has a mandate from W3C to improve itself (i.e,
XSD ver 1.1) should be discouraged.


-- 
Regards,
Mukul Gandhi

> I haven't seen W3C 
> specs, recommendating users to use OASIS specs, *and* 
> mentioning these statements in W3C specs themselves. Or do we 
> have any instances of W3C specs, where W3C recommends users 
> to use certain OASIS specs. 

I don't think you've looked very hard. The WDs for XHTML 2.0 and for MathML3
are two examples I quickly found of W3C specs that provide RelaxNG schemas.
In MathML we find the statements:

* MathML documents should be validated using the RelaxNG Schema for MathML

* MathML fragments can be validated using the XML Schema for MathML

(see http://www.w3.org/TR/2009/WD-MathML3-20090924/appendixa.html)

I'm sure there are many other examples. These conspiracy theories about W3C
are quite unfounded. W3C does try, on the whole, to keep its specs
architecturally aligned with each other, so there is obviously going to be
some encouragement to working groups to follow the party line (that's why
XQuery ended up being a superset of XPath rather than a completely different
language), but in the end, the technical decisions are made by individual
WGs - which means, in effect, by individuals like you and me, sometimes
influenced to a greater or lesser degree by the companies they work for. 

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

> I am really not sure, why entities are not allowed to be defined in
> XSD. I believe, there is some design principle behind that, and also
> the XML Schema WG must be aware about this issue I believe.

essentially for the same reason that XSD should not be referenced from
the xml spec. XSD (like Relax NG, XSLT etc) all work on the infoset
produced by parsing an XML document, thus they require an XML parser to
provide them input, to expand entities etc.

DTD processing on the other hand does not work by taking an infoset as
input but is instead a modified parser that reports validation outcomes.

XSD can not define entities as it is too late in the chain. If the
document uses entity references that are not defined to the parser then
the document is in error (or not well formed if there is no external DTD
reference at all).

David

________________________________________________________________________
The Numerical Algorithms Group Ltd is a company registered in England
and Wales with company number 1249803. The registered office is:
Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.

This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. 
________________________________________________________________________

Mukul Gandhi wrote:
> On Sun, Nov 8, 2009 at 2:10 PM, Rick Jelliffe <rjelliffe@allette.com.au> wrote:
>> Because XSD is complete crap that utterly betrayed the premises and goals of
>> XML perhaps?
> 
> Rick, I do not know why you condemn XSD so much :) I feel, XSD doesn't
> deserve so much condemnation.
> If somebody doesn't like certain technologies for whatever reasons,
> they should simply not use it, instead of saying words like "crap",
> "poisonous" and so on.

Sorry, Mukul - sometimes it's better to be honest and spare others the 
pain of discovering the truth too late.

-- 
Simon St.Laurent
http://simonstl.com/

On Sat, Nov 7, 2009 at 10:42 PM, Simon St.Laurent <simonstl@simonstl.com> wrote:
> It's supposed to hurt XSD.  If you're that attached to XSD that complaints
> about XSD hurt you, I'm afraid you may be too attached to the technology.

I am not too attached to any technology, whatsoever. Any technology
that can solve business and societal issue, is good for me.

> Unfortunately, it's also brittle, barely able to represent some common
> structures, offers extensibility mechanisms that aren't very extensible, and
> has lots of corners that aren't implemented consistently.

As I wrote earlier, I don't intend to judge the merits of XSD language
as such, or compare all or any of XML Schema language.

Any language (I mean, the XML Schema languages as we are discussing
here) has certain design goals, and all computational languages are
generally nice if they have met their stated design goals. What people
are saying about RelaxNG or Schematron cannot apply to XSD, as design
goals of XSD were different. And this should be true vice versa as
well.
If a community is concerned about features of any of the languages,
the language can modify itself probably keeping in mind, various
factors involved like cost, benefits and so on.

> Worst of all, its W3C-blessed status often ends conversation, which can be
> poisonous for projects that would have done better with DTDs, RELAX NG,
> Schematron, a combination of those, no schema whatsoever, or something else
> entirely.

I don't think, saying phrases like "poisonous" helps to solve a design
mistake in a spec, from a standard body :(

> Politically, of course it's possible.  And maybe the W3C, which is normally
> (though not always) fond of its works, will decide to add such a reference.

I am not suggesting to solve this issue politically. I intend to get
this solved, as a sound computational design issue.



-- 
Regards,
Mukul Gandhi

Thanks, Jim for some nice comments.

My intention was not to say, that DTD is now not useful. As you
rightly said, DTD is still preferred over XSD for certain class of
problems.

My averment was to give XSD the same status as DTD, as a XML
validation technology in W3C XML spec.

On Sun, Nov 8, 2009 at 1:31 AM, Jim Melton <jim.melton@acm.org> wrote:
> I'm one of those people who don't fall in love with technologies for their
> own sake, regardless of how beautiful and/or useful they might be.  I do
> have a fondness for elegance, as well as for practicality, but I believe
> that technology provides tools and nothing more.  XSD is one such tool, DTD
> is another.  Each has its uses, and I use them both.
>
> As Simon has pointed out, DTDs do a great job for document-centric
> applications where there is little that needs to be known about the
> semantics of atomic data types.  I use DTDs for my editing work on the SQL
> standard and on W3C documents, as well as for a host of other documents I do
> in other aspects of my day job and my life.
>
> Schemas are often very clumsy for validating "ustructured" textual
> documents, such as fiction books, biographies, and the like.  But schemas
> (XSD being the W3C-defined instance of this technology) are very useful when
> dealing with structured documents, especially those that contain lots of
> traditional data and/or those that are generated specifically from data (not
> to mention those that *are* data expressed in an XML tree).
>
> I get very, very frustrated when I'm trying to drive a nail into their wall
> using a screwdriver, as well as when I'm trying to get a screw out of the
> wall using a hammer.  That doesn't make me hate the screwdriver or the
> hammer -- nor, for that matter, to love either one when I figure out that I
> got it backwards and start using the proper tool for each job.  They're just
> tools.  Pick the right one for your job.
>
> If your management is forcing you to use XSD for unstructured document
> validation (or DTDs for highly structured data validation), don't hate XSD
> (or DTD)...hate the bureaucracy that limits your choice of tool!
>
> Hope this helps,
>   Jim
>
> ========================================================================
> Jim Melton --- Editor of ISO/IEC 9075-* (SQL)     Phone: +1.801.942.0144
>  Chair, W3C XML Query WG; XQX (etc.) editor       Fax : +1.801.942.3345
> Oracle Corporation        Oracle Email: jim dot melton at oracle dot com
> 1930 Viscounti Drive      Standards email: jim dot melton at acm dot org
> Sandy, UT 84093-1063 USA          Personal email: jim at melton dot name
> ========================================================================



-- 
Regards,
Mukul Gandhi

On Sat, Nov 7, 2009 at 10:15 PM, Dan Vint <dvint@dvint.com> wrote:
> Why and where do you stop? RelaxNG is another standard for providing
> validation and has a different/additional set of capabilities that are
> missing in both schema and DTD validation.

My point was, that XSD and DTD (and also XML) are "W3C" specs. RelaxNG
is an OASIS specification. I haven't seen W3C specs, recommendating
users to use OASIS specs, *and* mentioning these statements in W3C
specs themselves. Or do we have any instances of W3C specs, where W3C
recommends users to use certain OASIS specs. This more so doesn't
happen if W3C and OASIS develop competing languages. But I don't see
XSD and RelaxNG as competing to each other.

My simple point was, that XML spec mentions DTD as a validation
technology but it doesn't mention today XSD it's spec. And XSD is
another validation technology from W3C, which is vastly better than
DTD and is a pretty stable language now.

I think, we are not debating just now about the shortcomings of XSD.
Even DTD has shortcomings (I think, that's why XSD was developed), but
it's mentioned in the XML spec. I think, even RelaxNG has had certain
design goals, and it doesn't provides some of the capabilities with
XSD or DTD provides.

Looking at the XML spec, the XML spec seems to convey that DTD is an
inseparable part of XML definition. But if DTD is inseparable from
XML, why cannot XML spec say that XSD can be used for validation as
well (since XSD is a better XML validation technology, and is also
developed by W3C)?

> The history of XML is SGML and the use of DTDs. DTDs and SGML were initially intended
> for textual manual with little to know data types. A DTD is powerful enough
> to handle that requirement very well without overly complicating the
> specification of the document content.

That's history, and was the starting point I guess. But now,
data/messages is another important paradigm other than documents.
Keeping in view the evolution of XML (from documents to "documents &
data types"), XML validation is now an important concern for any kind
of XML data, whether that can be documents or data/messages.

> Just like picking a programming language you
> are free to pick the validation method you want to use.

I agree. But my point as I have been saying in this thread, was that
XML mentions DTD, but it doesn't mention XSD. This gives a naive user
a feeling, that DTD is something which is integral to XML, while XSD
is not. I believe, this should be corrected in XML spec.

> Would you have the standard for C point a
> programmer to Java, C#, etc just because they are newer and provide the
> same/similar/better functionality?

I agree. That won't be correct. A C spec cannot point users to Java, C# etc.
But I think, this argument doesn't map well to DTD and XSD case.

I even think, that a future XML spec other than referring to XSD as an
alternative XML validation technology with DTDs, should also provide
means to embed XML Schema's inline the XML documents, as we do with
DTDs.

I just could find this reference where Microsoft support inline XSD's
starting from MSXML 5.0 (ref,
http://msdn.microsoft.com/en-us/library/ms759142(VS.85).aspx). I
think, this design principle can be incorporated to the XML spec (or
is this already defined somewhere in W3C specs?).

I could also find this reference related to this,
http://xsd.stylusstudio.com/2005Mar/post03007.htm, which seem to
explain this design aspect from the point of view of XML spec.


-- 
Regards,
Mukul Gandhi

On Sat, Nov 7, 2009 at 7:38 PM, Simon St.Laurent <simonstl@simonstl.com> wrote:
>  Leaving aside the question of how damaging W3C XML Schema has
> been

I am not sure, how to respond to this. I guess this statement
certainly hurts the XSD community, and it certainly does hurt me.

I don't know, how a technology (i.e, XSD..) that is capable of doing
XML validation (and XSD does this well), and is implemented by number
of XML products, which is implemented widely in XML applications (I
can see uncountable XML instances been validated by XSD every day),
can be damaging.

> I suspect that counting validation as a fourth component might have eased
> some problems all around, but that wasn't clear at the outset. That kind of
> change seems like a project for XML 2.0, not for an edition change.

I'll be happy with XSD being specified in XML 2.0, if it can be.

> (And, of course, I'd argue hard against any effort to incorporate W3C XML
> Schema explicitly into XML 2.0....)

I am asking only for a simple reference in the XML spec, pointing to
the XSD spec and ideally saying somewhere in the XML spec (2.0
perhaps), that XSD is another validation technology from W3C similar
to DTD.
I still cannot think, why anybody can disagree to this. XSD and DTD
both belong to W3C, and both are W3C recommendations, so I don't think
why this simple modification to XML spec cannot take place.



-- 
Regards,
Mukul Gandhi

On Sat, Nov 7, 2009 at 7:04 PM, Michael Sokolov <sokolov@ifactory.com> wrote:
> There seems to have
> been an insistence on making validity a property of the document, rather
> than a separate concern.

I think, presence of DTD in the XML spec exactly says this (i.e,
validity is a concept inseparably tied to XML markup). But of course,
XML document validity is optional for applications to perform.

Since DTD is already mentioned in the XML spec, why not specify XSD as
well as a normative reference to a future XML spec? I think, even an
errata to a current XML spec can fix this. I am not sure, if another
edition of XML is underway [1]. If that's [1] happening at W3C at the
moment, it may even contain this normative reference I am suggesting.

Even W3C says, that XSD replaces DTD as a better XML validation
technology. And XSD 1.0 version has been in existence since many years
now, and implemented by quite a few XML products, and also deployed
widely in end user applications.
Therefore, why XML standard cannot say that XSD is a validation
technology like what DTD is?

I think, mentioning XSD in the XML spec will encourage users to use
XSD as a better validation technology, than DTD. Mentioning this as a
reference link in XML spec, looks like a very minor and non disruptive
addition to me, for the XML spec.



-- 
Regards,
Mukul Gandhi

> Complex data structures seem to lead to loathing...

I think we have a terrible tendency in this industry to confuse aesthetic
and utilitarian judgements. A thing of great ugliness can sometimes also be
of great utility; conversely, there are languages that are beautiful but
useless.

Almost everyone finds XML Schema ugly. But a lot of people also find it
useful.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

> 
> I don't know, how a technology (i.e, XSD..) that is capable 
> of doing XML validation (and XSD does this well), and is 
> implemented by number of XML products, which is implemented 
> widely in XML applications (I can see uncountable XML 
> instances been validated by XSD every day), can be damaging.

I think Simon is probably thinking in terms of opportunity costs. That is,
the idea that if this technology hadn't been so widely adopted, the
community might have adopted something better instead, and gained a better
return on its investment.

He might also be thinking that for some people who have adopted the
technology, the cost has been greater than the benefit. I think it would be
hard to demonstrate that objectively.

> 
> I am asking only for a simple reference in the XML spec, 
> pointing to the XSD spec and ideally saying somewhere in the 
> XML spec (2.0 perhaps), that XSD is another validation 
> technology from W3C similar to DTD.
> I still cannot think, why anybody can disagree to this. XSD 
> and DTD both belong to W3C, and both are W3C recommendations, 
> so I don't think why this simple modification to XML spec 
> cannot take place.
> 

We're dealing here with specifications: a legalistic contract between the
implementor of an XML parser and the user of an XML parser. You can write a
fully conformant XML parser without ever knowing that XSD even exists.
Furthermore, you can use XML without ever knowing that XSD exists.
Therefore, XSD doesn't need to be part of the XML contract. Equally, you
wouldn't want XML to be mentioned in the Unicode specifications, because
Unicode has no dependency on XML, and would still be of value if everyone
suddenly decided to abandon XML and move to YML instead.

It's of great value that specifications should NOT have unnecessary
dependencies. The fact that XML links normatively to DTDs is undoubtedly a
bad thing. In this respect the architecture of specifications is very like
the architecture of software implementations: you want the components to be
as loosely coupled as you can achieve, to allow each component to evolve or
to be replaced with minimum impact on the other components.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

++1

len

-----Original Message-----
From: Jim Melton [mailto:jim.melton@acm.org] 
Sent: Saturday, November 07, 2009 2:01 PM
To: xml-dev@lists.xml.org
Subject: Re:  XML spec and XSD

I'm one of those people who don't fall in love with technologies for 
their own sake, regardless of how beautiful and/or useful they might 
be.  I do have a fondness for elegance, as well as for practicality, 
but I believe that technology provides tools and nothing more.  XSD 
is one such tool, DTD is another.  Each has its uses, and I use them both.

As Simon has pointed out, DTDs do a great job for document-centric 
applications where there is little that needs to be known about the 
semantics of atomic data types.  I use DTDs for my editing work on 
the SQL standard and on W3C documents, as well as for a host of other 
documents I do in other aspects of my day job and my life.

Schemas are often very clumsy for validating "ustructured" textual 
documents, such as fiction books, biographies, and the like.  But 
schemas (XSD being the W3C-defined instance of this technology) are 
very useful when dealing with structured documents, especially those 
that contain lots of traditional data and/or those that are generated 
specifically from data (not to mention those that *are* data 
expressed in an XML tree).

I get very, very frustrated when I'm trying to drive a nail into 
their wall using a screwdriver, as well as when I'm trying to get a 
screw out of the wall using a hammer.  That doesn't make me hate the 
screwdriver or the hammer -- nor, for that matter, to love either one 
when I figure out that I got it backwards and start using the proper 
tool for each job.  They're just tools.  Pick the right one for your job.

If your management is forcing you to use XSD for unstructured 
document validation (or DTDs for highly structured data validation), 
don't hate XSD (or DTD)...hate the bureaucracy that limits your choice of
tool!

Hope this helps,
    Jim

========================================================================
Jim Melton --- Editor of ISO/IEC 9075-* (SQL)     Phone: +1.801.942.0144
   Chair, W3C XML Query WG; XQX (etc.) editor       Fax : +1.801.942.3345
Oracle Corporation        Oracle Email: jim dot melton at oracle dot com
1930 Viscounti Drive      Standards email: jim dot melton at acm dot org
Sandy, UT 84093-1063 USA          Personal email: jim at melton dot name
========================================================================
=  Facts are facts.   But any opinions expressed are the opinions      =
=  only of myself and may or may not reflect the opinions of anybody   =
=  else with whom I may or may not have discussed the issues at hand.  =
========================================================================  


_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

Thanks, Mike for your remarks.

On Sat, Nov 7, 2009 at 3:59 PM, Michael Kay <mike@saxonica.com> wrote:
> It's a basic issue of architectural layering. XSD has a dependency on XML,
> XML has no dependency on XSD. Nothing in the XML spec is affected if XSD
> changes.
>
> It's bad enough when you're writing a spec tracking the changes in
> technologies you depend on (like Unicode). Introducing unnecessary
> dependencies for pedagogic or marketing reasons would be a very bad thing to
> do.

Looking at your view points above, I agree to them as good
architectural principles while writing W3C specs.

But I feel, that mentioning XSD as a validation technology for XML
documents, in the XML spec is perhaps a good idea since DTD is also
mentioned in the XML spec (which is also a XML validation technology).
I feel, doing so doesn't promote any pedagogic or marketing attitudes
towards XSD.

Reading the XML spec, gives us a feeling (to me at least, I guess)
that DTD is the most important technology for validating XML. Even if
we don't mention specific versions of XSD as validating language for
XML documents (in specific XML standards, like 1.0 5th edition or XML
1.1), I think it's good to mention in references of the XML spec (I
believe, a normative reference to this would also be good in the XML
spec), that XSD is also another XML validation technology from W3C,
which achieves the same task as DTDs do. I think, referring to the
link, http://www.w3.org/XML/Schema in XML specs would serve the
purpose I am suggesting.



-- 
Regards,
Mukul Gandhi

Michael Kay wrote:
>> Complex data structures seem to lead to loathing...
> 
> I think we have a terrible tendency in this industry to confuse aesthetic
> and utilitarian judgements. A thing of great ugliness can sometimes also be
> of great utility; conversely, there are languages that are beautiful but
> useless.

I should probably be clearer - these stories weren't coming from 
architecture astronauts or aesthetes.  They were generally coming from 
people who found themselves trapped by schemas other people had created, 
in which they then had to find a place for their data and processes. 
Some of them came from people who had in fact built those traps for 
themselves.

And doubtless some of the issues were the results of style choices by 
those creating the schemas, not necessarily the specification itself.

 > Almost everyone finds XML Schema ugly. But a lot of people also find
 > it useful.

Ugliness alone is not a problem - brambles can certainly produce 
delicious fruit.  But brambles also produce spiky thickets that are hard 
to move through, or modify.

-- 
Simon St.Laurent
http://simonstl.com/

I'm one of those people who don't fall in love with technologies for 
their own sake, regardless of how beautiful and/or useful they might 
be.  I do have a fondness for elegance, as well as for practicality, 
but I believe that technology provides tools and nothing more.  XSD 
is one such tool, DTD is another.  Each has its uses, and I use them both.

As Simon has pointed out, DTDs do a great job for document-centric 
applications where there is little that needs to be known about the 
semantics of atomic data types.  I use DTDs for my editing work on 
the SQL standard and on W3C documents, as well as for a host of other 
documents I do in other aspects of my day job and my life.

Schemas are often very clumsy for validating "ustructured" textual 
documents, such as fiction books, biographies, and the like.  But 
schemas (XSD being the W3C-defined instance of this technology) are 
very useful when dealing with structured documents, especially those 
that contain lots of traditional data and/or those that are generated 
specifically from data (not to mention those that *are* data 
expressed in an XML tree).

I get very, very frustrated when I'm trying to drive a nail into 
their wall using a screwdriver, as well as when I'm trying to get a 
screw out of the wall using a hammer.  That doesn't make me hate the 
screwdriver or the hammer -- nor, for that matter, to love either one 
when I figure out that I got it backwards and start using the proper 
tool for each job.  They're just tools.  Pick the right one for your job.

If your management is forcing you to use XSD for unstructured 
document validation (or DTDs for highly structured data validation), 
don't hate XSD (or DTD)...hate the bureaucracy that limits your choice of tool!

Hope this helps,
    Jim

========================================================================
Jim Melton --- Editor of ISO/IEC 9075-* (SQL)     Phone: +1.801.942.0144
   Chair, W3C XML Query WG; XQX (etc.) editor       Fax : +1.801.942.3345
Oracle Corporation        Oracle Email: jim dot melton at oracle dot com
1930 Viscounti Drive      Standards email: jim dot melton at acm dot org
Sandy, UT 84093-1063 USA          Personal email: jim at melton dot name
========================================================================
=  Facts are facts.   But any opinions expressed are the opinions      =
=  only of myself and may or may not reflect the opinions of anybody   =
=  else with whom I may or may not have discussed the issues at hand.  =
========================================================================  

Michael Kay clarified for me:
> I think Simon is probably thinking in terms of opportunity costs. That is,
> the idea that if this technology hadn't been so widely adopted, the
> community might have adopted something better instead, and gained a better
> return on its investment.

Yes, that's a key part of the argument.  I think the W3C generally did a 
good job of creating XML specifications when it was stripping them down 
based on SGML experience.  After that initial process, though, their 
inventions are not nearly as impressive.  Other people and organizations 
did a much better job of creating useful tools, but lack the W3C brand 
power.

> He might also be thinking that for some people who have adopted the
> technology, the cost has been greater than the benefit. I think it would be
> hard to demonstrate that objectively.

In the big picture, you're quite correct that it would be hard to 
demonstrate that objectively.  It would require access to stories and 
data that aren't likely to be shared and are often difficult to measure.

Anecdotally, I've encountered people who love W3C XML Schema and people 
who despise it.  Their fondness or loathing for it seems to depend on 
what they tried to use it for.

Complex data structures seem to lead to loathing, as do document-centric 
projects.  Versioning in particular seems to lead to loathing, except in 
extremely simple projects.

I am, of course, pretty well-known to be a critic of the technology, so 
I'm sure there's a bias in which stories people choose to tell me.

-- 
Simon St.Laurent
http://simonstl.com/

Mukul Gandhi wrote:
> On Sat, Nov 7, 2009 at 7:38 PM, Simon St.Laurent <simonstl@simonstl.com> wrote:
>>  Leaving aside the question of how damaging W3C XML Schema has
>> been
> 
> I am not sure, how to respond to this. I guess this statement
> certainly hurts the XSD community, and it certainly does hurt me.

It's supposed to hurt XSD.  If you're that attached to XSD that 
complaints about XSD hurt you, I'm afraid you may be too attached to the 
technology.

> I don't know, how a technology (i.e, XSD..) that is capable of doing
> XML validation (and XSD does this well), and is implemented by number
> of XML products, which is implemented widely in XML applications (I
> can see uncountable XML instances been validated by XSD every day),
> can be damaging.

XSD is good for exactly two reasons:

* It starts conversation.

* It's widely deployed.

Unfortunately, it's also brittle, barely able to represent some common 
structures, offers extensibility mechanisms that aren't very extensible, 
and has lots of corners that aren't implemented consistently.  The 
XQuery folks encountered yet another set of problems in creating models 
based on it.

Worst of all, its W3C-blessed status often ends conversation, which can 
be poisonous for projects that would have done better with DTDs, RELAX 
NG, Schematron, a combination of those, no schema whatsoever, or 
something else entirely.

>> I suspect that counting validation as a fourth component might have eased
>> some problems all around, but that wasn't clear at the outset. That kind of
>> change seems like a project for XML 2.0, not for an edition change.
> 
> I'll be happy with XSD being specified in XML 2.0, if it can be.

Hopefully not.

>> (And, of course, I'd argue hard against any effort to incorporate W3C XML
>> Schema explicitly into XML 2.0....)
> 
> I am asking only for a simple reference in the XML spec, pointing to
> the XSD spec and ideally saying somewhere in the XML spec (2.0
> perhaps), that XSD is another validation technology from W3C similar
> to DTD.
> I still cannot think, why anybody can disagree to this. XSD and DTD
> both belong to W3C, and both are W3C recommendations, so I don't think
> why this simple modification to XML spec cannot take place.

Politically, of course it's possible.  And maybe the W3C, which is 
normally (though not always) fond of its works, will decide to add such 
a reference.

That would strike me as a continuation of the mistakes they've already 
made, but wouldn't be surprising.

-- 
Simon St.Laurent
http://simonstl.com/

At 11:12 AM 11/7/2009, you wrote:
>I am asking only for a simple reference in the XML spec, pointing to
>the XSD spec and ideally saying somewhere in the XML spec (2.0
>perhaps), that XSD is another validation technology from W3C similar
>to DTD.
>I still cannot think, why anybody can disagree to this. XSD and DTD
>both belong to W3C, and both are W3C recommendations, so I don't think
>why this simple modification to XML spec cannot take place.

Why and where do you stop? RelaxNG is another standard for providing 
validation and has a different/additional set of capabilities that 
are missing in both schema and DTD validation.

I think you may have missed a subtle point when I read: 'uncountable 
XML instances been validated by XSD every day'. Sounds like your 
dealing with data/messages, Len mentioned documents (tech manuals and 
such). The history of XML is SGML and the use of DTDs. DTDs and SGML 
were initially intended for textual manual with little to know data 
types. A DTD is powerful enough to handle that requirement very well 
without overly complicating the specification of the document 
content. Yes DTD didn't provide everything that everyone might need, 
Schema provides more capability, but still relationships between 
elements and part of a document are not possible to validate with a 
DTD or Schema. Just like picking a programming language you are free 
to pick the validation method you want to use.

I guess I'm confused at why a reference is needed. Ok, yeah there are 
new and different ways of providing the validation for which v1 of 
the XML standard were developed. Would you have the standard for C 
point a programmer to Java, C#, etc just because they are newer and 
provide the same/similar/better functionality?

..dan
---------------------------------------------------------------------------
Danny Vint

Panoramic Photography
http://www.dvint.com

voice: 502-749-6179
     

Hi all,
   Seeing the XML 1.0, 5th edition spec (ref,
http://www.w3.org/TR/REC-xml/) I cannot find any reference to the
phrase "XML Schema" or XSD (which are the XML Schema languages,
defined by the W3C specs viz specified at,
http://www.w3.org/XML/Schema).

The XML 1.0 spec, certainly refers to the word DTD and provides the
complete definition of DTD semantics, and how it integrates with XML
as a validation technology. This all is great.

But I wonder, why the XML spec (1.0 or 1.1) doesn't mention XSD (i.e,
W3C XML Schema language) as also a second XML validation language
(i.e, other than DTD)? I am curious to know, was this decision not to
mention XSD as a validation technology in XML specs, was a consicous
one? If yes, what advantages we have achieved by this decision?

This curiousity spurs to my mind, keeping in mind that XSD is a very
important XML validation technology today, which is a W3C standard and
implemented widely in software applications.


-- 
Regards,
Mukul Gandhi

> 
> But I wonder, why the XML spec (1.0 or 1.1) doesn't mention 
> XSD (i.e, W3C XML Schema language) as also a second XML 
> validation language (i.e, other than DTD)? I am curious to 
> know, was this decision not to mention XSD as a validation 
> technology in XML specs, was a consicous one? If yes, what 
> advantages we have achieved by this decision?

It's a basic issue of architectural layering. XSD has a dependency on XML,
XML has no dependency on XSD. Nothing in the XML spec is affected if XSD
changes.

It's bad enough when you're writing a spec tracking the changes in
technologies you depend on (like Unicode). Introducing unnecessary
dependencies for pedagogic or marketing reasons would be a very bad thing to
do.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

No.  I think it was a way of saying something simple and familiar will work
until something more precise and powerful can replace it.  It was a
stabilizer in a time of incredible shifts and unpredictable fortunes.  It's
clear that validity is not intrinsic to markup functionally.  It is just as
clear that validation is useful to applications.

We let the next thing go too far in terms of application.  We needed
multiple validators for different users and uses.   We have that now but
also we have entrenched factions of users.

len

-----Original Message-----
From: Mukul Gandhi [mailto:gandhi.mukul@gmail.com] 
Sent: Saturday, November 07, 2009 9:25 AM
To: Michael Sokolov
Cc: Michael Kay; xml-dev@lists.xml.org
Subject: Re:  XML spec and XSD

On Sat, Nov 7, 2009 at 7:04 PM, Michael Sokolov <sokolov@ifactory.com>
wrote:
> There seems to have
> been an insistence on making validity a property of the document, rather
> than a separate concern.

I think, presence of DTD in the XML spec exactly says this (i.e,
validity is a concept inseparably tied to XML markup). But of course,
XML document validity is optional for applications to perform.

Since DTD is already mentioned in the XML spec, why not specify XSD as
well as a normative reference to a future XML spec? I think, even an
errata to a current XML spec can fix this. I am not sure, if another
edition of XML is underway [1]. If that's [1] happening at W3C at the
moment, it may even contain this normative reference I am suggesting.

Even W3C says, that XSD replaces DTD as a better XML validation
technology. And XSD 1.0 version has been in existence since many years
now, and implemented by quite a few XML products, and also deployed
widely in end user applications.
Therefore, why XML standard cannot say that XSD is a validation
technology like what DTD is?

I think, mentioning XSD in the XML spec will encourage users to use
XSD as a better validation technology, than DTD. Mentioning this as a
reference link in XML spec, looks like a very minor and non disruptive
addition to me, for the XML spec.



-- 
Regards,
Mukul Gandhi

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

In my mind it wasn't clear where validation belonged and secondly who should
be responsible for it being done.  To clarify, SGML made it reasonably
simple for a technical writer to express document definition, not validation
per se, but an as-simple-as=possible 'these are the contents in the order
needed for this specific task or communication'.  Most of the fancy math
computer sciency database stuff got ladled on to that basic need because we
in the business of writing documents were increasingly writing for computer
science applications instead of law, regulations, etc.  We tended to bow
down to the programmers.

Dropping validation was on some minds most particularly the computer
scientists who wanted markup to serve their needs.   They have a lot of
experience string schlepping and the idea that validation was necessary to
define outside of their own code seemed wasteful and that at a time when
every CPU cycle was sacred because scarce.   To the writers having the
toolkit to define the document made sense because frankly the computer
scientists were even lousier at that then the lawyers had been.  So markup
became the possession of the computer scientists instead of the writers.

It is bit like the scene in Star Wars where the Senate cheers mightily for
the new Emperor.  They think it a pretty good idea at the time.

DTDs were and in many ways still a pretty good compromise for giving power
tools to people who don't write a lot of code.  XSD might have been a better
compromise but once again the theoreticians, the math wonks and the compiler
wonks took the field.  Like a very heavy sword, it became something very
powerful in the right hands and too hard to wield in others.

Sad but so:  Microsoft had it right in the beginning.  Politics and market
ideology were the avatars of local ambition.  It's hard to fight a truckload
of speeding money driven by ideologues at the behest of bankers.

len

-----Original Message-----
From: Simon St.Laurent [mailto:simonstl@simonstl.com] 
Sent: Saturday, November 07, 2009 8:08 AM
To: xml-dev@lists.xml.org
Subject: Re:  XML spec and XSD

Mukul Gandhi wrote:
> But I feel, that mentioning XSD as a validation technology for XML
> documents, in the XML spec is perhaps a good idea since DTD is also
> mentioned in the XML spec (which is also a XML validation technology).
> I feel, doing so doesn't promote any pedagogic or marketing attitudes
> towards XSD.

It's history.  Leaving aside the question of how damaging W3C XML Schema 
has been, it simply wasn't part of the original XML vision.

The three parts were supposed to be markup, style, and linking, learning 
from SGML, DSSSL, and HyTime respectively.  Markup, inheriting from the 
SGML vision, included DTDs.

I suspect that counting validation as a fourth component might have 
eased some problems all around, but that wasn't clear at the outset. 
That kind of change seems like a project for XML 2.0, not for an edition 
change.

(And, of course, I'd argue hard against any effort to incorporate W3C 
XML Schema explicitly into XML 2.0....)

-- 
Simon St.Laurent
http://simonstl.com/

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

Mukul Gandhi wrote:
> But I feel, that mentioning XSD as a validation technology for XML
> documents, in the XML spec is perhaps a good idea since DTD is also
> mentioned in the XML spec (which is also a XML validation technology).
> I feel, doing so doesn't promote any pedagogic or marketing attitudes
> towards XSD.

It's history.  Leaving aside the question of how damaging W3C XML Schema 
has been, it simply wasn't part of the original XML vision.

The three parts were supposed to be markup, style, and linking, learning 
from SGML, DSSSL, and HyTime respectively.  Markup, inheriting from the 
SGML vision, included DTDs.

I suspect that counting validation as a fourth component might have 
eased some problems all around, but that wasn't clear at the outset. 
That kind of change seems like a project for XML 2.0, not for an edition 
change.

(And, of course, I'd argue hard against any effort to incorporate W3C 
XML Schema explicitly into XML 2.0....)

-- 
Simon St.Laurent
http://simonstl.com/

DTD is special, sadly.  It got lodged in the XML spec in the early days, too
deeply to be excised now, a legacy of SGML I think?  There seems to have
been an insistence on making validity a property of the document, rather
than a separate concern, as it is w/XSD and others. Probably someone older
and wiser can give you a more complete explanation.

-Mike 

> -----Original Message-----
> From: Mukul Gandhi [mailto:gandhi.mukul@gmail.com] 
> Sent: Saturday, November 07, 2009 5:49 AM
> To: Michael Kay
> Cc: xml-dev@lists.xml.org
> Subject: Re:  XML spec and XSD
> 
> Thanks, Mike for your remarks.
> 
> On Sat, Nov 7, 2009 at 3:59 PM, Michael Kay <mike@saxonica.com> wrote:
> > It's a basic issue of architectural layering. XSD has a 
> dependency on 
> > XML, XML has no dependency on XSD. Nothing in the XML spec 
> is affected 
> > if XSD changes.
> >
> > It's bad enough when you're writing a spec tracking the changes in 
> > technologies you depend on (like Unicode). Introducing unnecessary 
> > dependencies for pedagogic or marketing reasons would be a very bad 
> > thing to do.
> 
> Looking at your view points above, I agree to them as good 
> architectural principles while writing W3C specs.
> 
> But I feel, that mentioning XSD as a validation technology 
> for XML documents, in the XML spec is perhaps a good idea 
> since DTD is also mentioned in the XML spec (which is also a 
> XML validation technology).
> I feel, doing so doesn't promote any pedagogic or marketing 
> attitudes towards XSD.
> 
> Reading the XML spec, gives us a feeling (to me at least, I 
> guess) that DTD is the most important technology for 
> validating XML. Even if we don't mention specific versions of 
> XSD as validating language for XML documents (in specific XML 
> standards, like 1.0 5th edition or XML 1.1), I think it's 
> good to mention in references of the XML spec (I believe, a 
> normative reference to this would also be good in the XML 
> spec), that XSD is also another XML validation technology 
> from W3C, which achieves the same task as DTDs do. I think, 
> referring to the link, http://www.w3.org/XML/Schema in XML 
> specs would serve the purpose I am suggesting.
> 
> 
> 
> --
> Regards,
> Mukul Gandhi
> 
> ______________________________________________________________
> _________
> 
> XML-DEV is a publicly archived, unmoderated list hosted by 
> OASIS to support XML implementation and development. To 
> minimize spam in the archives, you must subscribe before posting.
> 
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org List archive: 
> http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> 
> 

What I mean by interface files is ...

The database into which the XML data will be stored is just a
'staging' area. Other applications require subsets of the data in
different formats that meet both their specific needs and conform to
their application data models. So whilst some 'reports' may be sourced
by this data directly, for the most part into is just one feed into
many other 'interfaces'.

Fraser.

2009/11/4 Jim Tivy <jimt@bluestream.com>:
> Fraser
>
> Often XML coming in reflects one hierarchy of the data - for example invoice
> header and details - for example, where each invoice is a customer
> transaction at a CoffeeHouse Chain.  These incoming hierarchies can have
> redundant information - for example the address of a customer as stated in
> each invoice.  If you need to report on this data in a different hiearchy,
> first you need to define an appropriate RDB application data model and then
> import carefully into it to remove redudancy, normalize the data into
> respective entities (tables) and generally protect and enforce the quality
> of data in the database.  Then you can acurately report on the data where
> the output of the report reflects a different hierarchy than the incoming
> Xml data -  say, total product sales of product latte in January by City by
> Store.
>
> In this way, the reports you need out will carry requirements for your RDB
> application data model.
>
> Not sure what you mean by "interface files"?
>
> Jim
>
> -----Original Message-----
> From: Fraser Goffin [mailto:goffinf@googlemail.com]
> Sent: Wednesday, November 04, 2009 8:20 AM
> To: xml-dev@lists.xml.org
> Subject: Re:  Shredding XML
>
> I don't know the full detail of the interface files that will be
> produced from this store, but its probably safe to assume that these
> will need to model the data with multiple hierarchies and groupings.
>
> Whats driving this line of thought ?
>
> Fraser.
>
> 2009/11/3 Jim Tivy <jimt@bluestream.com>:
>> do they need to produce reports that present the information in
>> multiple hierarchial ways?  It might be invoices within a day or Lattes
> sold
>> within City within Province.
>>
>> -----Original Message-----
>> From: Fraser Goffin [mailto:goffinf@googlemail.com]
>> Sent: Tuesday, November 03, 2009 1:41 AM
>> To: xml-dev@lists.xml.org
>> Subject: Re:  Shredding XML
>>
>>> Do your applications that use the relational database need to update
>>
>> Thats a difficult one. As far as producing up-stream interfaces, no,
>> they are created just by selecting from the database.
>>
>> However, as I mentioned (lost in the detail) earlier, messages
>> received will have a relationship to existing records. I'm not at all
>> convinced that update will be possible because not all of the keys
>> will be available in the message received. Some form of 'fuzzy'
>> matching may be possible in some case, for example, if I receive a new
>> address then the 1st line + PostCode + the relationship of this
>> address to something else may be sufficient.
>>
>> It might be the case that we have to consider time based snap-shots,
>> i.e. the 'state' of a collection of data at a point in time. Not sure
>> yet, its one of those areas of difficulty that probably doesn't change
>> much whether I use relational of native XML stores.
>>
>> At present the primary purpose of the database is for retreival, i.e.
>> its NOT an operational store, its for MI, so optimiastions for read
>> rather than update are appropriate.
>>
>>> do they need to produce reports that present the information in multiple
>> ways
>>
>> Yes.
>>
>> Fraser.
>>
>> 2009/11/3 Jim Tivy <jimt@bluestream.com>:
>>> Do your applications that use the relational database need to update and
>> do
>>> they need to produce reports that present the information in multiple
> ways
>>> hierarchial ways?  This may suggest whether your database needs to be
>>> normalized (approaching 3rd Normal form)?
>>>
>>> -----Original Message-----
>>> From: Fraser Goffin [mailto:goffinf@googlemail.com]
>>> Sent: Monday, November 02, 2009 3:31 PM
>>> To: xml-dev@lists.xml.org
>>> Subject: Re:  Shredding XML
>>>
>>> Hi Jim,
>>>
>>> thats interesting ... which should be the 'driving' schema, XML or Db ?
>>>
>>> I guess I've been somewhat tiptoe'ing around this one.
>>>
>>> I should admit my bias if its not already apparent. I work mainly in
>>> the SOA integration space and since XML is the primary exchange format
>>> and XML schema does a reasonable job as the type system, I favour
>>> processing XML as .. well XML ... whilst I understand the argument
>>> around leveraging existing technologies and skillset .. often-times
>>> this is little more than protectionism and continually [de]composing
>>> from XML to objects then to CopyLibs and then to relational just seems
>>> unnecessary a lot of the time (sorry - soap box over).  But of course
>>> the whole world isn't XML and just like most other large organisations
>>> the vast majority of our processing capability and data isn't and
>>> probably never will be ... I have no issue with that.
>>>
>>> On the one hand it is the end product that drives the design (even if
>>> that design has a relatively short shelf-life ... but hey, we all do
>>> agile right). In that case it is definately the Database schema that
>>> prevails from the pure delivery point of view, since this is the
>>> desired source for the staging area from which to produce interface
>>> files for upstream applications. At present there appears to be no
>>> possiblility of revisiting that choice. At the same time, I don't want
>>> to 'paint myself into a corner' or promote this as an exemplar for all
>>> future approaches (unless it turns out that way :-)
>>>
>>> My unease is around the brittleness of the database schema in the face
>>> of change, but I suppose that situation is almost inevitable since I
>>> can't crystal-ball what changes might be coming along next week and
>>> its probably folly to try. XML changes that dynamic, but not
>>> completely.
>>>
>>> I have been having this internal debate about, .. if I concede I'm
>>> going to have a relational database then should its design be derived
>>> from the XML schema or should the XML schema change to accomodate the
>>> database, indeed one of the Solution Designers on this has already
>>> indicated a desire to 'flatten' the XML schema (although I have to say
>>> I disagree that it is necessary). I have some degree of opportunity to
>>> change the XML schema (although messages
>>> are received from external sources, within reason, I can transform
>>> them into any 'shape' I like so long as thats a loss-less exchange).
>>> The database is green field so it can be any shape, but clearly some
>>> designs are going to lend themsleves better than others to XML mapping
>>> I would have thought ?
>>>
>>> Surely there are some structures in XML that don't map
>>> straight-forwardly. Ted Neward called this the 'last mile' (a familiar
>>> term to us all I'm sure), where the illusion of a high fidelity
>>> solution draws us in, and indeed 80%+ appears to go quite well, but
>>> that last few % hold a disproportiate cost and increasing complexity
>>> (but you don't realise that until late on at which point some are
>>> going to object to a rethink). I want to know where that 'last mile'
>>> lives so I can try and avoid it !
>>>
>>> Fraser.
>>>
>>> 2009/11/2 Jim Tivy <jimt@bluestream.com>:
>>>> Fraser
>>>>
>>>> I am not entirely hearing firm commitment that you plan to establish an
>>> RDB
>>>> schema and make it the driving schema.  In other words, what this would
>>> mean
>>>> is that data elements cannot be put into the RDB unless they exist in
> the
>>>> RDB schema.  For example, if some new data elements show up in some
>>> external
>>>> XML to be imported then the DBA decides whether to allow them into the
>>>> appropriate RDB column or not, or drop them for the time being.
>>>>
>>>> Another option (from the infinite number) would be to let the XML schema
>>>> generate the RDB schema and the mapping code.  For your application
>>>> programmers using SQL on the RDB this would likely lead to gagging and
>>>> hacking and an "out of body experience"  This is not something I would
>>>> recommend and if this is what you want then get a database that supports
>>>> XQuery and retrain your developers.
>>>>
>>>> But I think you have to choose between these two - the first being what
>> it
>>>> sounds like you want - then work backwards from that decision.
>>>>
>>>> Jim
>>>>
>>>> -----Original Message-----
>>>> From: Fraser Goffin [mailto:goffinf@googlemail.com]
>>>> Sent: Monday, November 02, 2009 12:22 PM
>>>> To: xml-dev@lists.xml.org
>>>> Subject: Re:  Shredding XML
>>>>
>>>> Yes Jim, that is spot on.
>>>>
>>>> Whilst there has been much discussion thus far on the technolgies and
>>>> techniques of getting data out of the database (and that has been
>>>> interesting), the programming for doing so are 'bread and butter' to
>>>> our mainframe Cobol and Sapiens guys, so thats not really my problem.
>>>>
>>>> Mine is the task of getting the data from a fairly complex XML content
>>>> model into an appropriately factored relational database. The design
>>>> of that database is 'green field' but (and thanks to many on this
>>>> thread who have posted related papers) this may not be as easy at it
>>>> might at first appear, what with impedence mismatches here there and
>>>> everywhere ;-)
>>>>
>>>> Its also the case that the XML data doesn't contain enough data
>>>> inherently to represent primary or foreign key values for all of the
>>>> relationships that are likely to arise. In some cases I MAY be
>>>> permitted to generate them myself (say using a UUID) as I 'walk' the
>>>> XML, in other cases I MAY be required to get the database to provide
>>>> the value(s), not sure yet. The later may increase the complexity
>>>> somewhat (sidenote: our DBAs don't allow stored procs (don't ask)  so
>>>> I'm going to be doing whole bunches of INSERTs as part of the
>>>> tree-walk I suspect)
>>>>
>>>> I'm really interested in the gotchas and best practices. Some have
>>>> already been mentioned like the fact that the XML schema may define
>>>> optional items and unrestricted length facets and such like. Others
>>>> I've seen in reading talk about the mis-match of identity approaches
>>>> (although this was talking primarily about OO/Relational mapping but
>>>> the idea is similar I suspect). This could be important, since some
>>>> messages received may 'relate' to others already loaded and, given
>>>> what I said about not having all of the data in the XML to form all of
>>>> the keys, this might be a significant problem.
>>>>
>>>> It is my intention to look into other options (we have recently
>>>> acquired DB2 v9 which includes pureXML) but as is so often the case,
>>>> the immediate project delivery pressures won't allow it. The PM is
>>>> very nervous about using any new tech, perhaps justifiably, but my
>>>> sense of unease is more to do with the perhaps misplaced assumption
>>>> that 'tried and tested' tech like relational databases will always
>>>> provide a workable solution, imho sometimes they actually represent
>>>> the most significant constraint.
>>>>
>>>> So yes, back to the actual problem. How to come up with a database
>>>> design that provides the capability of staging the shredded XML in a
>>>> reasonable efficient manner and enables it to be loaded from XML
>>>> instances received, again efficiently (ideally without 100's of tables
>>>> and joins to negotiate). As far as efficiency of storage, well that
>>>> MAY be a concern although perhaps not a huge one so long as the Db
>>>> doesn't bloat up too much if normalisation is preferred over extra
>>>> tables.
>>>>
>>>> Please add your thoughts and suggestions and experiences as you are
>>>> able. Nothing is too trivial (or rude) to mention (i.e. if you want to
>>>> say don't do this if you want to keep your sanity, thats ok).
>>>>
>>>> regards
>>>>
>>>> Fraser.
>>>>
>>>> I'm
>>>>
>>>>
>>>> 2009/11/1 Jim Tivy <jimt@bluestream.com>:
>>>>> Interesting post, but I am not sure that "now is the time to talk of
>> many
>>>>> things".
>>>>>
>>>>> Let me try to focus:
>>>>>
>>>>> Proper software execution comes from the choice of appropriate
>>>>> actions/technologies to match the driving requirements.  But more
>>>>> importantly, the greatest Wisdom, is to frame the driving requirements
>>>>> correctly before "going off half cocked" or doing something that is
>>>>> unnecessary and unwarranted.
>>>>>
>>>>> So lets start by framing the requirements again:
>>>>>
>>>>> Fraser Gofin wrote:
>>>>>
>>>>> "
>>>>> The basics are we receive XML messages from an external trading partner
>>>> and
>>>>> process those messages, enriching and routing to a number of internal
>>>>> subscriber applications. One of these applications is MI and the deal
>>> here
>>>>> is that they want the data to been put into a relational database so
>> that
>>>>> they can create a number of interfaces 'files' which are sent to still
>>>> more
>>>>> applications.
>>>>> "
>>>>>
>>>>> OR
>>>>>
>>>>> "
>>>>> I am mainly interested in the process of LOADING XML data to a database
>>>>> rather than extracting (at least for the purposes of this discussion).
>>>>> "
>>>>>
>>>>> It is possible that the "mother persistent application datamodel" is
>>>>> contained in the relational database in all its normalized glory.  If
>> so,
>>>>> then, "processing the messages" is simply a "data import" operation.
>  So
>>>> the
>>>>> question is, how do I get XML X* to tables T*.  It would strike me that
>>>> lots
>>>>> of people are doing this.  Are there common techniques and technologies
>>>> for
>>>>> doing this import?
>>>>>
>>>>> Fraser, is that a proper framing of the question/requirements?
>>>>>
>>>>> Jim
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Petite Abeille [mailto:petite.abeille@gmail.com]
>>>>> Sent: Sunday, November 01, 2009 9:56 AM
>>>>> To: xml-dev@lists.xml.org
>>>>> Subject: Re:  Shredding XML
>>>>>
>>>>>
>>>>> On Oct 29, 2009, at 10:20 PM, Fraser Goffin wrote:
>>>>>
>>>>>> opinions on the subject of decomposing XML into relational databases
>>>>>
>>>>> Outside of the most trivial case, this is a major PITA of the same
>>>>> epic proportion as the object-relational one:
>>>>>
>>>>>
>>>
> http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx
>>>>>
>>>>> Good luck.
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________________________________
>>>>>
>>>>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>>>>> to support XML implementation and development. To minimize
>>>>> spam in the archives, you must subscribe before posting.
>>>>>
>>>>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>>>>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>>>>> subscribe: xml-dev-subscribe@lists.xml.org
>>>>> List archive: http://lists.xml.org/archives/xml-dev/
>>>>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________________________________
>>>>>
>>>>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>>>>> to support XML implementation and development. To minimize
>>>>> spam in the archives, you must subscribe before posting.
>>>>>
>>>>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>>>>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>>>>> subscribe: xml-dev-subscribe@lists.xml.org
>>>>> List archive: http://lists.xml.org/archives/xml-dev/
>>>>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>>>>
>>>>>
>>>>
>>>> _______________________________________________________________________
>>>>
>>>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>>>> to support XML implementation and development. To minimize
>>>> spam in the archives, you must subscribe before posting.
>>>>
>>>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>>>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>>>> subscribe: xml-dev-subscribe@lists.xml.org
>>>> List archive: http://lists.xml.org/archives/xml-dev/
>>>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>>>
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________________________________
>>>
>>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>>> to support XML implementation and development. To minimize
>>> spam in the archives, you must subscribe before posting.
>>>
>>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>>> subscribe: xml-dev-subscribe@lists.xml.org
>>> List archive: http://lists.xml.org/archives/xml-dev/
>>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>>
>>>
>>>
>>>
>>
>> _______________________________________________________________________
>>
>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>> to support XML implementation and development. To minimize
>> spam in the archives, you must subscribe before posting.
>>
>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>> subscribe: xml-dev-subscribe@lists.xml.org
>> List archive: http://lists.xml.org/archives/xml-dev/
>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>
>>
>>
>>
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>
>
>
>

I don't know the full detail of the interface files that will be
produced from this store, but its probably safe to assume that these
will need to model the data with multiple hierarchies and groupings.

Whats driving this line of thought ?

Fraser.

2009/11/3 Jim Tivy <jimt@bluestream.com>:
> do they need to produce reports that present the information in
> multiple hierarchial ways?  It might be invoices within a day or Lattes sold
> within City within Province.
>
> -----Original Message-----
> From: Fraser Goffin [mailto:goffinf@googlemail.com]
> Sent: Tuesday, November 03, 2009 1:41 AM
> To: xml-dev@lists.xml.org
> Subject: Re:  Shredding XML
>
>> Do your applications that use the relational database need to update
>
> Thats a difficult one. As far as producing up-stream interfaces, no,
> they are created just by selecting from the database.
>
> However, as I mentioned (lost in the detail) earlier, messages
> received will have a relationship to existing records. I'm not at all
> convinced that update will be possible because not all of the keys
> will be available in the message received. Some form of 'fuzzy'
> matching may be possible in some case, for example, if I receive a new
> address then the 1st line + PostCode + the relationship of this
> address to something else may be sufficient.
>
> It might be the case that we have to consider time based snap-shots,
> i.e. the 'state' of a collection of data at a point in time. Not sure
> yet, its one of those areas of difficulty that probably doesn't change
> much whether I use relational of native XML stores.
>
> At present the primary purpose of the database is for retreival, i.e.
> its NOT an operational store, its for MI, so optimiastions for read
> rather than update are appropriate.
>
>> do they need to produce reports that present the information in multiple
> ways
>
> Yes.
>
> Fraser.
>
> 2009/11/3 Jim Tivy <jimt@bluestream.com>:
>> Do your applications that use the relational database need to update and
> do
>> they need to produce reports that present the information in multiple ways
>> hierarchial ways?  This may suggest whether your database needs to be
>> normalized (approaching 3rd Normal form)?
>>
>> -----Original Message-----
>> From: Fraser Goffin [mailto:goffinf@googlemail.com]
>> Sent: Monday, November 02, 2009 3:31 PM
>> To: xml-dev@lists.xml.org
>> Subject: Re:  Shredding XML
>>
>> Hi Jim,
>>
>> thats interesting ... which should be the 'driving' schema, XML or Db ?
>>
>> I guess I've been somewhat tiptoe'ing around this one.
>>
>> I should admit my bias if its not already apparent. I work mainly in
>> the SOA integration space and since XML is the primary exchange format
>> and XML schema does a reasonable job as the type system, I favour
>> processing XML as .. well XML ... whilst I understand the argument
>> around leveraging existing technologies and skillset .. often-times
>> this is little more than protectionism and continually [de]composing
>> from XML to objects then to CopyLibs and then to relational just seems
>> unnecessary a lot of the time (sorry - soap box over).  But of course
>> the whole world isn't XML and just like most other large organisations
>> the vast majority of our processing capability and data isn't and
>> probably never will be ... I have no issue with that.
>>
>> On the one hand it is the end product that drives the design (even if
>> that design has a relatively short shelf-life ... but hey, we all do
>> agile right). In that case it is definately the Database schema that
>> prevails from the pure delivery point of view, since this is the
>> desired source for the staging area from which to produce interface
>> files for upstream applications. At present there appears to be no
>> possiblility of revisiting that choice. At the same time, I don't want
>> to 'paint myself into a corner' or promote this as an exemplar for all
>> future approaches (unless it turns out that way :-)
>>
>> My unease is around the brittleness of the database schema in the face
>> of change, but I suppose that situation is almost inevitable since I
>> can't crystal-ball what changes might be coming along next week and
>> its probably folly to try. XML changes that dynamic, but not
>> completely.
>>
>> I have been having this internal debate about, .. if I concede I'm
>> going to have a relational database then should its design be derived
>> from the XML schema or should the XML schema change to accomodate the
>> database, indeed one of the Solution Designers on this has already
>> indicated a desire to 'flatten' the XML schema (although I have to say
>> I disagree that it is necessary). I have some degree of opportunity to
>> change the XML schema (although messages
>> are received from external sources, within reason, I can transform
>> them into any 'shape' I like so long as thats a loss-less exchange).
>> The database is green field so it can be any shape, but clearly some
>> designs are going to lend themsleves better than others to XML mapping
>> I would have thought ?
>>
>> Surely there are some structures in XML that don't map
>> straight-forwardly. Ted Neward called this the 'last mile' (a familiar
>> term to us all I'm sure), where the illusion of a high fidelity
>> solution draws us in, and indeed 80%+ appears to go quite well, but
>> that last few % hold a disproportiate cost and increasing complexity
>> (but you don't realise that until late on at which point some are
>> going to object to a rethink). I want to know where that 'last mile'
>> lives so I can try and avoid it !
>>
>> Fraser.
>>
>> 2009/11/2 Jim Tivy <jimt@bluestream.com>:
>>> Fraser
>>>
>>> I am not entirely hearing firm commitment that you plan to establish an
>> RDB
>>> schema and make it the driving schema.  In other words, what this would
>> mean
>>> is that data elements cannot be put into the RDB unless they exist in the
>>> RDB schema.  For example, if some new data elements show up in some
>> external
>>> XML to be imported then the DBA decides whether to allow them into the
>>> appropriate RDB column or not, or drop them for the time being.
>>>
>>> Another option (from the infinite number) would be to let the XML schema
>>> generate the RDB schema and the mapping code.  For your application
>>> programmers using SQL on the RDB this would likely lead to gagging and
>>> hacking and an "out of body experience"  This is not something I would
>>> recommend and if this is what you want then get a database that supports
>>> XQuery and retrain your developers.
>>>
>>> But I think you have to choose between these two - the first being what
> it
>>> sounds like you want - then work backwards from that decision.
>>>
>>> Jim
>>>
>>> -----Original Message-----
>>> From: Fraser Goffin [mailto:goffinf@googlemail.com]
>>> Sent: Monday, November 02, 2009 12:22 PM
>>> To: xml-dev@lists.xml.org
>>> Subject: Re:  Shredding XML
>>>
>>> Yes Jim, that is spot on.
>>>
>>> Whilst there has been much discussion thus far on the technolgies and
>>> techniques of getting data out of the database (and that has been
>>> interesting), the programming for doing so are 'bread and butter' to
>>> our mainframe Cobol and Sapiens guys, so thats not really my problem.
>>>
>>> Mine is the task of getting the data from a fairly complex XML content
>>> model into an appropriately factored relational database. The design
>>> of that database is 'green field' but (and thanks to many on this
>>> thread who have posted related papers) this may not be as easy at it
>>> might at first appear, what with impedence mismatches here there and
>>> everywhere ;-)
>>>
>>> Its also the case that the XML data doesn't contain enough data
>>> inherently to represent primary or foreign key values for all of the
>>> relationships that are likely to arise. In some cases I MAY be
>>> permitted to generate them myself (say using a UUID) as I 'walk' the
>>> XML, in other cases I MAY be required to get the database to provide
>>> the value(s), not sure yet. The later may increase the complexity
>>> somewhat (sidenote: our DBAs don't allow stored procs (don't ask)  so
>>> I'm going to be doing whole bunches of INSERTs as part of the
>>> tree-walk I suspect)
>>>
>>> I'm really interested in the gotchas and best practices. Some have
>>> already been mentioned like the fact that the XML schema may define
>>> optional items and unrestricted length facets and such like. Others
>>> I've seen in reading talk about the mis-match of identity approaches
>>> (although this was talking primarily about OO/Relational mapping but
>>> the idea is similar I suspect). This could be important, since some
>>> messages received may 'relate' to others already loaded and, given
>>> what I said about not having all of the data in the XML to form all of
>>> the keys, this might be a significant problem.
>>>
>>> It is my intention to look into other options (we have recently
>>> acquired DB2 v9 which includes pureXML) but as is so often the case,
>>> the immediate project delivery pressures won't allow it. The PM is
>>> very nervous about using any new tech, perhaps justifiably, but my
>>> sense of unease is more to do with the perhaps misplaced assumption
>>> that 'tried and tested' tech like relational databases will always
>>> provide a workable solution, imho sometimes they actually represent
>>> the most significant constraint.
>>>
>>> So yes, back to the actual problem. How to come up with a database
>>> design that provides the capability of staging the shredded XML in a
>>> reasonable efficient manner and enables it to be loaded from XML
>>> instances received, again efficiently (ideally without 100's of tables
>>> and joins to negotiate). As far as efficiency of storage, well that
>>> MAY be a concern although perhaps not a huge one so long as the Db
>>> doesn't bloat up too much if normalisation is preferred over extra
>>> tables.
>>>
>>> Please add your thoughts and suggestions and experiences as you are
>>> able. Nothing is too trivial (or rude) to mention (i.e. if you want to
>>> say don't do this if you want to keep your sanity, thats ok).
>>>
>>> regards
>>>
>>> Fraser.
>>>
>>> I'm
>>>
>>>
>>> 2009/11/1 Jim Tivy <jimt@bluestream.com>:
>>>> Interesting post, but I am not sure that "now is the time to talk of
> many
>>>> things".
>>>>
>>>> Let me try to focus:
>>>>
>>>> Proper software execution comes from the choice of appropriate
>>>> actions/technologies to match the driving requirements.  But more
>>>> importantly, the greatest Wisdom, is to frame the driving requirements
>>>> correctly before "going off half cocked" or doing something that is
>>>> unnecessary and unwarranted.
>>>>
>>>> So lets start by framing the requirements again:
>>>>
>>>> Fraser Gofin wrote:
>>>>
>>>> "
>>>> The basics are we receive XML messages from an external trading partner
>>> and
>>>> process those messages, enriching and routing to a number of internal
>>>> subscriber applications. One of these applications is MI and the deal
>> here
>>>> is that they want the data to been put into a relational database so
> that
>>>> they can create a number of interfaces 'files' which are sent to still
>>> more
>>>> applications.
>>>> "
>>>>
>>>> OR
>>>>
>>>> "
>>>> I am mainly interested in the process of LOADING XML data to a database
>>>> rather than extracting (at least for the purposes of this discussion).
>>>> "
>>>>
>>>> It is possible that the "mother persistent application datamodel" is
>>>> contained in the relational database in all its normalized glory.  If
> so,
>>>> then, "processing the messages" is simply a "data import" operation.  So
>>> the
>>>> question is, how do I get XML X* to tables T*.  It would strike me that
>>> lots
>>>> of people are doing this.  Are there common techniques and technologies
>>> for
>>>> doing this import?
>>>>
>>>> Fraser, is that a proper framing of the question/requirements?
>>>>
>>>> Jim
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Petite Abeille [mailto:petite.abeille@gmail.com]
>>>> Sent: Sunday, November 01, 2009 9:56 AM
>>>> To: xml-dev@lists.xml.org
>>>> Subject: Re:  Shredding XML
>>>>
>>>>
>>>> On Oct 29, 2009, at 10:20 PM, Fraser Goffin wrote:
>>>>
>>>>> opinions on the subject of decomposing XML into relational databases
>>>>
>>>> Outside of the most trivial case, this is a major PITA of the same
>>>> epic proportion as the object-relational one:
>>>>
>>>>
>> http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx
>>>>
>>>> Good luck.
>>>>
>>>>
>>>>
>>>> _______________________________________________________________________
>>>>
>>>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>>>> to support XML implementation and development. To minimize
>>>> spam in the archives, you must subscribe before posting.
>>>>
>>>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>>>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>>>> subscribe: xml-dev-subscribe@lists.xml.org
>>>> List archive: http://lists.xml.org/archives/xml-dev/
>>>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________________________________
>>>>
>>>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>>>> to support XML implementation and development. To minimize
>>>> spam in the archives, you must subscribe before posting.
>>>>
>>>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>>>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>>>> subscribe: xml-dev-subscribe@lists.xml.org
>>>> List archive: http://lists.xml.org/archives/xml-dev/
>>>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>>>
>>>>
>>>
>>> _______________________________________________________________________
>>>
>>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>>> to support XML implementation and development. To minimize
>>> spam in the archives, you must subscribe before posting.
>>>
>>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>>> subscribe: xml-dev-subscribe@lists.xml.org
>>> List archive: http://lists.xml.org/archives/xml-dev/
>>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>>
>>>
>>>
>>>
>>
>> _______________________________________________________________________
>>
>> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
>> to support XML implementation and development. To minimize
>> spam in the archives, you must subscribe before posting.
>>
>> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
>> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
>> subscribe: xml-dev-subscribe@lists.xml.org
>> List archive: http://lists.xml.org/archives/xml-dev/
>> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>>
>>
>>
>>
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>
>
>
>