RE: AltovaXML bugs? (and other engines)

> I wonder how result comparison can be achieved, as this has 
> to be done by the driver. Further down, under "Comparing Results":
> 
>   The test harness must canonicalize both, the actual result
>   and the expected result according to the "Canonical XML"
>   recommendation [2], which refers to a number of open-source
>   implementations. Byte-comparison can then be applied to the
>   resulting XML documents.
> 
> Sounds reasonable. But can we expect an XSLT 1.0 
> implementation to comply to the "Canonical XML" recommendation?

No, that's why you have to canonicalize both the actual result and the
expected result. (In fact of course, any canonical representation will do so
long as it preserves the properties that the W3C canonicalization preserves
and loses the properties that it loses. In practice I think it's also very
useful to have an option that compares ignoring whitespace - although in
theory whitespace in the results is significant, it's always useful to know
if your variation from expected results is only a whitespace difference.
> 
> Well, if the XSLT catalog is different from the published 
> XQuery catalog, why not use it as the base of an open test 
> suite for XSLT?

It is different, though in relatively minor details. For example, it has a
stronger mechanism for saying which optional features of spec a test depends
on, and of course it has ways of setting things like initial mode and
initial named template that are XSLT-specific. Here's a sample entry for a
simple test:

   <testcase>
      <name>atrs01</name>
      <creator>Michael Kay, Saxonica</creator>
      <category>AttribSet</category>
      <description> PURPOSE: Set attribute of a LRE from single attribute
set. </description>
      <section number="10.2" title="Named Attribute Sets" spec="XSLT20"
version="2005-04-04"/>
      <input>
         <stylesheet role="principal" file="atrs/atrs01.xsl"/>
         <source-document role="principal" file="atrs/atrs01.xml"/>
      </input>
      <output same-as-1.0="true">
         <result-document role="principal" type="xml"
file="atrs/atrs01.out"/>
      </output>
   </testcase>

and here's a more complex one:

   <testcase>
      <name>schemaas20_003_01</name>
      <creator origin="IBM">Vesela Parapounska</creator>
      <category>SchemaAs</category>
      <description>Test with global xsl:variable which holds a mixed
sequence of derived built-in or
user-defined atomic values or () and @as="xs:anyAtomicType*". </description>
      <section number="9.1" spec="XSLT20" version="2005-11-03"/>
      <section number="5.3" spec="XSLT20" version="2005-11-03"/>
      <origin organization="IBM">
         <copyright>Copyright IBM Corp. 2004, 2005, 2006.</copyright>
      </origin>
      <discretionary-items>
         <discretionary-feature name="schema_aware" behavior="on"/>
      </discretionary-items>
      <input>
         <stylesheet file="schemaas/schemaas20_003.xsl" role="principal"/>
         <source-document file="schemaas/schemaas20_003.xml"
role="principal"/>
         <schema role="source-reference"
file="schemaas/variousTypesSchemaAs.xsd"/>
         <schema role="stylesheet-import"
file="schemaas/variousTypesSchemaAs.xsd"/>
      </input>
      <output same-as-1.0="false">
         <result-document file="schemaas/schemaas20_003_01.out"
role="principal" type="xml"/>
      </output>
   </testcase>

I'm hoping we can get approval to publish the framework spec even if we
can't publish the tests themselves.

Michael Kay
http://www.saxonica.com/