[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Unit tests in XSLT (was: Re: Controlling Debugging

Subject: Re: Unit tests in XSLT (was: Re: Controlling Debugging Messages)
From: "Eliot Kimber ekimber@xxxxxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>
Date: Mon, 15 Oct 2018 20:02:55 -0000
Re:  Unit tests in XSLT (was: Re: Controlling Debugging
I have only created unit tests for function packages where the functions A)
could be tested easily and B) were used widely enough to warrant testing
outside the context of the specific transform that used them.

In the context of documentation-type transforms, as you say, it is usually
prohibitive to maintain the data that drives the tests because it is either
highly sensitive to details of the transform (e.g., how attributes might be
serialized or how formatting ends up being expressed or whatever) or the input
varies so much or both. For example, trying to implement a complex publishing
transform for documents where the markup details are being worked out and are
driven, in part, but what we learn from implementing the publishing
processes.

The DITA Open Toolkit project does maintain a set of tests that compare
expected result docs to actual results and that works pretty well but that's
also a very controlled environment where the input, in particular, is highly
controlled and the output result is stable (in particular, generating HTML
from DITA source, where there's much less variation in the generated result
than their might be with other transformation targets). But those test are
also time consuming to maintain and they definitely do not represent a full
test suite over the functionality of even just the HTML5 transform.

In my experience doing primarily publishing processing, it's more efficient to
use inspection and special-purpose test documents that can be easily verified
by inspection and then let normal documentation review processes highlight
problems.

In rare cases where the data is both complex and errors may not be caught by
normal inspection I will implement a post-transform evaluation process, for
example, ensuring all elements of a specific type in the input are reflected
in the output and not duplicated (assuming duplication would be an error).

Most projects I'm involved with tend to be under-resourced and time
constrained, so it's hard to build in the cost of more formal testing to the
project bids, even if the project might benefit in the longer term. That is, I
seldom have the luxury of applying the level of software engineering attention
that I would like to bring to my XSLT development.

Cheers,

Eliot

--
Eliot Kimber
http://contrext.com


o;?On 10/15/18, 2:41 PM, "Pieter Masereeuw pieter@xxxxxxxxxxxx"
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:

    Talking about debugging, I recently attended a webinar by Oxygen about
    unit tests for XSLT (and Schematron). After watching it, I felt guilty
    that my XSLT development practice is still not test-driven. Despite of
    what was said in the webinar, it seems too much trouble and the thing
    that most often requires modification of stylesheets is changes or
    hitherto unforeseen constructs in the format of the input XML.

    It would be interesting to know if there are readers on this list who
    actually make use of units test during XSLT development and what their
    experiences are.

    Pieter Masereeuw


    On 10/15/2018 07:28 PM, Eliot Kimber ekimber@xxxxxxxxxxxx wrote:
    > Yes, in general you want debug messages to be turned off, which is why
the doDebug parameter default is "false()" in my code.
    >
    > I didn't think of use-when="$DEBUG or true()"--that would work for a lot
of my cases but it doesn't handle the case where I want to turn on debugging
for all the templates that will get called in the course of handing some
specific input.
    >
    > So that suggests that my dynamic approach is what I need generally.
    >
    > Once the code is in place and working then it would be easier to set up
a set of debugging control variables that reflect different cases or code
paths I know I might need to debug in the future but during development that
doesn't really work because of course the code is in flux you don' t
necessarily know what will be of interest and what won't.
    >
    > It might work to have per-module static debug controls. I've moved
generally to using more smaller modules, usually one per distinct mode or set
of related modes, and that would make it more natural to have global debugging
for those modes. I'll have to think about that more.
    >
    > In practice my debugging pattern isn't a burden--it's something I do
whenever I set up a new template or add an apply-templates or next-match or
call-template but it's always felt like there should be a simpler way.
    >
    > Cheers,
    >
    > Eliot
    > --
    > Eliot Kimber
    > http://contrext.com
    >
    >
    > o;?On 10/15/18, 12:11 PM, "Michael Kay mike@xxxxxxxxxxxx"
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
    >
    >     With static variables you can of course have multiple switches but
they will be statically scoped rather than dynamically scoped. You could use
multiple variables or you could use flags within a single variable
(use-when="contains($DEBUG_FLAGS, 'g')").
    >
    >     I have to confess I'm not usually that organized. I tend to have a
single variable $DEBUG which is false, and then switch on individual debug
lines using use-when="$DEBUG or true()". I tend to find that debug statements
are rarely useful once you've solved the bug that they were invented for;
except in rare cases where you persistently have problems with some particular
intermediate result passed across a key interface in your application - in
which case there may be better approaches than xsl:message to monitoring
what's passed across that boundary.
    >
    >     But I wouldn't recommend anyone to be as disorganised as me.
    >
    >     Michael Kay
    >     Saxonica
    >
    >     > On 15 Oct 2018, at 17:05, Eliot Kimber ekimber@xxxxxxxxxxxx
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
    >     >
    >     > I was just about to post about this.
    >     >
    >     > In my XSLT 2 code I have historically used this pattern:
    >     >
    >     > <xsl:template match="foo">
    >     >  <xsl:param name="doDebug" as="xs:boolean" tunnel="yes"
select="false()"/>
    >     >
    >     >  <xsl:if test="$doDebug">
    >     >   <xsl:message>+ [DEBUG] Handling <xsl:value-of
name="concat(name(..), '/', name(.))"/>...</xsl:message>
    >     >
    >     >  <xsl:apply-templates>
    >     >    <xsl:with-param name="doDebug" as="xsl:boolean" tunnel="yes"
select="$doDebug"/>
    >     >  </xsl:apply-templates>
    >     > </xsl:template>
    >     >
    >     > This allows me to selectively turn debugging on and off in
specific parts of the code but does require this somewhat heavy weight code.
    >     >
    >     > With @use-when, can I get the same level of local control?
    >     >
    >     > That is, with the above, I can add:
    >     >
    >     > <xsl:variable name="doDebug" as="xs:Boolean" select="true()"/>
    >     >
    >     > In any block to turn debugging on just there.
    >     >
    >     > If I understand the implications of static variables allowed in
@use-when, the debugging switch is globally all-or-nothing, or at least global
within a given package.
    >     >
    >     > Is that correct?
    >     >
    >     > If that is correct, is there a better way to do the selective,
dynamically-controlled debug messaging shown above?
    >     >
    >     > Cheers,
    >     >
    >     > E.
    >     >
    >     > --
    >     > Eliot Kimber
    >     > http://contrext.com
    >     >
    >     >
    >     > o;?On 10/15/18, 9:08 AM, "Michael Kay mike@xxxxxxxxxxxx"
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
    >     >
    >     >    These days you can do
    >     >
    >     >    <xsl:message use-when="$DEBUG" ....>
    >     >
    >     >    with $DEBUG defined as a static parameter.
    >     >
    >     >    <xsl:param name="DEBUG" as="xs:boolean" static="true"
select="false()"/>
    >     >
    >     >    No need for the run-time check with xsl:if.
    >     >
    >     >    You can also use xsl:assert to define assertions. In Saxon,
assertion checking can be enabled from the command line using -ea.
    >     >
    >     >    Michael Kay
    >     >    Saxonica
    >     >
    >     >> On 15 Oct 2018, at 14:54, Dave Pawson dave.pawson@xxxxxxxxx
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
    >     >>
    >     >> MIght even surround it with
    >     >> <xsl:if test="$debug">
    >     >>
    >     >> To ease insertion / removal when testing?
    >     >>
    >     >> HTH
    >     >> On Mon, 15 Oct 2018 at 14:31, Wendell Piez
wapiez@xxxxxxxxxxxxxxx
    >     >> <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
    >     >>>
    >     >>> Eliot writes:
    >     >>>
    >     >>>> I also depend heavily on using messages to test my
assumptions.
    >     >>>
    >     >>>> For example, I might do something like:
    >     >>>
    >     >>>> <xsl:message>+ [DEBUG] jpeg_few={$jpeg_few => string-join(',
')}</xsl:message>
    >     >>>> <xsl:message>+ [DEBUG] jpeg_many={$jpeg_many => string-join(',
')}</xsl:message>
    >     >>>
    >     >>> This is a key technique when developing XSLT. The language is
designed
    >     >>> to "fail gracefully" most of the time -- which puts the burden
on the
    >     >>> programmer to ensure things don't fail catastrophically. :-)
    >     >>>
    >     >>> Cheers, Wendell
    >     >>>
    >     >>> On Sun, Oct 14, 2018 at 7:10 PM Eliot Kimber
ekimber@xxxxxxxxxxxx
    >     >>> <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
    >     >>>>
    >     >>>> Looking at the XPath 3 Functions and Operators specification
and searching on "intersect" (hoping to also find "disjoint") I find this
discussion:
    >     >>>>
    >     >>>> D.4.2.3 eg:value-except
    >     >>>> eg:value-except(        $arg1    as xs:anyAtomicType*,
    >     >>>> $arg2    as xs:anyAtomicType*) as xs:anyAtomicType*
    >     >>>> This function returns a sequence containing all the distinct
items that appear in $arg1 but not in $arg2, in an arbitrary order.
    >     >>>>
    >     >>>> XSLT implementation
    >     >>>>
    >     >>>> <xsl:function name="eg:value-except" as="xs:anyAtomicType*">
    >     >>>> <xsl:param name="arg1" as="xs:anyAtomicType*"/>
    >     >>>> <xsl:param name="arg2" as="xs:anyAtomicType*"/>
    >     >>>> <xsl:sequence
    >     >>>>    select="fn:distinct-values($arg1[not(.=$arg2)])"/>
    >     >>>> </xsl:function>Which is in
https://www.w3.org/TR/xpath-functions-31/#other-functions (Appendix D).
    >     >>>>
    >     >>>> So basically
    >     >>>>
    >     >>>> distinct-values($jpeg_few[not(. = $jpeg_many)]
    >     >>>>
    >     >>>> Should give you the answer you seek.
    >     >>>>
    >     >>>> I agree with Mike that being obsessive about putting data types
on all variables and function return values (and templates when the templates
should return atomic types or specific element types) will help a lot.
    >     >>>>
    >     >>>> If your code is working without types but failing with them it
means your code is "working" but probably not for the reasons you think.
    >     >>>>
    >     >>>> Working carefully through the stages of the expressions by
setting each intermediate result into variable will help a lot.
    >     >>>>
    >     >>>> I also depend heavily on using messages to test my
assumptions.
    >     >>>>
    >     >>>> For example, I might do something like:
    >     >>>>
    >     >>>> <xsl:message>+ [DEBUG] jpeg_few={$jpeg_few => string-join(',
')}</xsl:message>
    >     >>>> <xsl:message>+ [DEBUG] jpeg_many={$jpeg_many => string-join(',
')}</xsl:message>
    >     >>>>
    >     >>>> Or if those lists are very long, use count() or get the first n
items or whatever to make it clear that you're working with the values you
think you are.
    >     >>>>
    >     >>>> Also, remember that <xsl:value-of> ({} in string result
contexts) is different from <xsl:sequence>, which returns the actual value,
not a string representation.
    >     >>>>
    >     >>>> For example, given a variable that is an attribute node,
value-of will return string value of the attribute but xsl:sequence will
return the attribute node and Saxon will serialize it as <attribute name="foo"
value="bar"> (or something similar to that.
    >     >>>>
    >     >>>> It's easy to accidently create a sequence of attributes when
what you wanted was a sequence of strings (or visa versa) and using
xsl:value-of can obscure that mistake.
    >     >>>>
    >     >>>> I've also started using the XQuery-required explicating casting
of values even though XSLT usually lets you get away with implicit casting,
because it makes it clearer to me what my intent was (and makes it easier to
copy XPath expressions into XQuery, if that's something you need to do).
    >     >>>>
    >     >>>> Cheers,
    >     >>>>
    >     >>>> Eliot
    >     >>>> --
    >     >>>> Eliot Kimber
    >     >>>> http://contrext.com
    >     >>>>
    >     >>>>
    >     >>>> o;?On 10/14/18, 3:53 PM, "Dave Lang emaildavelang@xxxxxxxxx"
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
    >     >>>>
    >     >>>>> That error can only come from an expression that calls
tokenize(). It's therefore clearly not your declaration of
jpgs_in_xml_not_directories that's at fault.
    >     >>>>
    >     >>>>   Fair enough - but when I run the transformation without that
declaration
    >     >>>>   everything works fine. Is there something I can do to the
variables that
    >     >>>>   are included in it to make the declaration work?
    >     >>>>
    >     >>>>
    >     >>>>
    >     >>>>
    >     >>>
    >     >>>
    >     >>>
    >     >>> --
    >     >>> Wendell Piez | http://www.wendellpiez.com
    >     >>> XML | XSLT | electronic publishing
    >     >>> Eat Your Vegetables
    >     >>> _____oo_________o_o___ooooo____ooooooo_^
    >     >>>
    >     >>
    >     >>
    >     >>
    >     >> --
    >     >> Dave Pawson
    >     >> XSLT XSL-FO FAQ.
    >     >> Docbook FAQ.

Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.