Re: junit test... for xslt2?

Dave Pawson wrote:

  Hi,

> Tony/Florent. I'm with you on the processing
model. Keep the
> test code as imported, then (reluctantly) separate the test
> generation from the overall result generation.. [and output
> formatting?].
Please note that XTC <http://fgeorges.org/xslt/xslt-unit/> is
still there only
for historical reasons.  I got my laptop stolen
with most of my code (call it
a youthful error), and I use Jeni's
XSpec for two years now (which I am a
maintainer of now).

> Is it practical to further than testing fairly simple
values?
> Aren't we into a full XML diff engine then?

  I tend to cut my XSLT
code down to functions and template rules
with simple responsibilities (in
addition to a few templates that
just create a bunch of the result tree, like
the overall HTML
structure or a piece of Javascript in a <script> element for
instance).  Those components are easily unit testable like any
other piece of
code in other languages, by writing assertion
expressions (is the content of
this subelement equal to this
string, or is the result of that function call
equal to 42?)

  But for a few components, mostly template rules, diffing
against an expected XML document is really what we need.  And
clearly we lack
here a general framework to perform XML
comparisons.  For instance behind able
to compare a result
document to something like:

    <root id="...">
<title>...</title>
       <para>This exact test.</para>
    </root>

  That
is, a root element, with an id attribute we don't care the
value of, with a
title with "something" as content, and a para
element with the exact same
content (and do you see the indent
text nodes? do they count in the
comparison?)  For now, XSpec
provides this exact same semantics (using "..."
to mean "every
content").  I feel we need something more rigorous, but that's
very powerful yet.

  In XTC I got a way to define your own assertions by
defining
something that looked like XSLT stylesheet functions.  But that
was
too much oriented toward XSLT (XSpec does provide XQuery
support too).
Another big area of investigation is "error expectations", as
Tony suggested.
How to test a function is correctly throwing an
error or using
xsl:message/@terminate?  That is not possible with
the current architecture as
the compiled stylesheet is a simple
stylesheet run once for every use cases
and creating as result
the XML test report.  If there is an error, there is no
output at
all.

  I think this go to the idea of test harness.  For now, all
XSpec does is compiling the test suite to an XSLT stylesheet, and
providing a
formating tool to present the report as a nice HTML
page.  But we should
create different test harnesses for Saxon,
eXist, MarkLogic, Zorba, etc.
(taking care of errors within test
cases, pecularities of specific processors,
etc.)

  In conclusion, I'd say this problem can seem to be quite
simple:
create an ad-hoc format for test suites, and compile it
(transform it using
XSLT) to an XSLT stylesheet that will run the
test cases and generate a
report.  But then you run into details
of document comparisons (trust me,
that's not an easy one), error
reporting, test harnesses for several
processors, etc.  So there
is really a benefit in using an existing,
maintained framework,
and XSpec seems to be a good candidate among the
existing ones).

  We expect to release a new version any time soon now.  Go
to
http://groups.google.com/group/xspec-users to stay tuned and to
http://xspec.googlecode.com/ to learn more about XSpec.

  Regards,

--
Florent Georges
http://www.fgeorges.org/