[XSL-LIST Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Unit tests in XSLT

Subject: Re: Unit tests in XSLT
From: "Michael Kay mike@xxxxxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>
Date: Wed, 24 Oct 2018 17:19:45 -0000
Re:  Unit tests in XSLT
Many of the tests in the XSLT 3.0 test suite use a similar approach, although
the assertions are always XPath expressions and we try to keep them much
simpler than this.

In fact there's a variety of styles:

        <all-of>
            <assert>count(//Kapital) = 929</assert>
            <assert>count(//Kapital/nr) = 929</assert>
            <assert>count(//Kapital/chtitle) = 929</assert>
            <assert>not(//Kapital/@*)</assert>
            <assert>not(//Kapital/v)</assert>
         </all-of>

This kind of assertion is quite good at spotting if the results aren't
correct, but not that good at telling you how or why they differ.

        <all-of xmlns:my="http://my.com/ns" xmlns:your="http://your.com/ns">
            <assert>count(//Loan) = 5</assert>
            <assert>/out/Loan/your:extra/@my:value="22"</assert>
            <assert>/out/Loan/your:extra=242</assert>
            <assert>/out/Loan/your:extra/comment()='comment'</assert>
            <assert>/out/Loan/your:extra/processing-instruction('PI')=''</ass
ert>
         </all-of>

         <all-of>
            <assert>count(/out/Property/Address) = 5</assert>
            <assert>/out/Property/Address/PostalCode = (22102, 71045, 33303,
31926, 28165)</assert>
            <assert>/out/Property/Address/State = ('VA', 'TX', 'MD', 'CA',
'AZ')</assert>
         </all-of>

        <assert>deep-equal(/out/State/string(), ("##VA##", "##TX##", "##MD##",
"##CA##", "##AZ##"))</assert>

In these three examples, the focus of the assertions is (probably) on the
things that the test is designed to be testing.

      <result>
         <assert-xml><![CDATA[<out><Property/><Property/><Property/><Property
>CA</Property><Property>28165</Property></out>]]></assert-xml>
      </result>

If the result is short then it's OK to assert XML-equivalence, which can be
tested by canonicalizing. But comparing XML documents gets very unwieldy if
they are large.

In all these cases, the key is to find a set of assertions that (a) aren't
going to fail because of irrelevant properties of the result (e.g. whitespace,
or order of attributes), and (b) aren't going to succeed when the output of
the stylesheet is actually wrong.

Michael Kay
Saxonica


> On 24 Oct 2018, at 17:42, Eliot Kimber ekimber@xxxxxxxxxxxx
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>
> I just realized that I have started doing a form of testing which is not
quite unit tests but has similar features. Basically, I write XQuery's that
analyze the output result and report on things in a way that makes it easy to
see if the result is as expected.
>
> For example, I'm currently implementing code that recognizes various forms
of citation and adds markup for them in existing XML. To test my result I use
this XQuery (I'm using it from Oxygen against a single result but could of
course be used in a more automated way or across a corpus using an XQuery
database):
>
> (: Report the citations by total count, unique value, and count per value
:)
>
> declare namespace map = "http://www.w3.org/2005/xpath-functions/map";
> let $fedcites := //*:span[@class = 'cite-federal-reg']
> let $statecites := //*:span[@class = 'cite-state-statute']
> let $fedmap as map(*) := map:merge(
> (
> for $cite in $fedcites
> return
>      map{
> 	string($cite) : $cite}
> ),
> 	 map{ 'duplicates' : 'combine'}
>      )
> let $statemap as map(*) := map:merge(
>      (
>      for $cite in $statecites
>      return
>      map{string($cite) : $cite}
>      ),
>      map{ 'duplicates' : 'combine'}
>      )
>      return
> <cite-analysis>
> <fed-cites total="{count($fedcites)}">{
> for $key in (map:keys($fedmap) => sort())
> let $instances := map:get($fedmap, $key)
> return <citation count="{count($instances)}">{$key}</citation>
> }</fed-cites>
> <state-cites total="{count($statecites)}">{
> for $key in (map:keys($statemap) => sort())
> let $instances := map:get($statemap, $key)
> return <citation count="{count($instances)}">{$key}</citation>
> }</state-cites>
> </cite-analysis>
>
> This produces a result like:
>
> <cite-analysis>
>   <fed-cites total="32">
>      <citation count="2">15 USC 78</citation>
>      <citation count="2">33 USC 1251</citation>
>      <citation count="1">33 USC 1292</citation>
>      <citation count="1">33 USC 1317</citation>
>      <citation count="2">40 CFR 136</citation>
>      <citation count="2">40 CFR 136.3</citation>
>      <citation count="1">40 CFR 2.302</citation>
>      <citation count="1">40 CFR 261</citation>
>      <citation count="1">40 CFR 261.21</citation>
>      <citation count="1">40 CFR 403</citation>
>      <citation count="2">40 CFR 403.13</citation>
>      <citation count="1">40 CFR 403.15</citation>
>      <citation count="1">40 CFR 403.5(a)(2)</citation>
>      <citation count="1">40 CFR 403.6(a)(4)</citation>
>      <citation count="2">40 CFR 403.6(c)</citation>
>      <citation count="2">40 CFR 403.6(e)</citation>
>      <citation count="1">40 CFR 403.8(f)(6)</citation>
>      <citation count="2">40 CFR 405b471</citation>
>      <citation count="1">44 CFR 59.22</citation>
>      <citation count="1">44 CFR 59.22(a)(9)(v)</citation>
>      <citation count="1">44 CFR 60.3</citation>
>      <citation count="1">44 CFR 60.3(c)</citation>
>      <citation count="1">44 CFR 60.3(d)</citation>
>      <citation count="1">7 USC 2012</citation>
>   </fed-cites>
>   <state-cites total="137">
>      <citation count="1">Code of Virginia, B' 1-200</citation>
>      <citation count="1">Code of Virginia, B' 10.1-100</citation>
>      <citation count="1">Code of Virginia, B' 10.1-1127.1</citation>
>      <citation count="2">Code of Virginia, B' 10.1-1414</citation>
>      <citation count="1">Code of Virginia, B' 15.2-107</citation>
>      <citation count="1">Code of Virginia, B' 15.2-1100</citation>
>      <citation count="1">Code of Virginia, B' 15.2-1400</citation>
>      <citation count="1">Code of Virginia, B' 15.2-1425</citation>
>      <citation count="1">Code of Virginia, B' 15.2-1429</citation>
>      <citation count="1">Code of Virginia, B' 15.2-1430</citation>
>      <citation count="2">Code of Virginia, B' 15.2-1432</citation>
>      <citation count="1">Code of Virginia, B' 15.2-1433</citation>
>      <citation count="1">Code of Virginia, B' 15.2-200</citation>
>      <citation count="2">Code of Virginia, B' 15.2-2200</citation>
>      <citation count="9">Code of Virginia, B' 15.2-2204</citation>
>      ...
>      <citation count="1">Code of Virginia, B' 62.1-255</citation>
>   </state-cites>
> </cite-analysis>
>
> That makes it easy to see what's being added to the source and whether or
not the result is consistent with expectations. I could refine this, for
example, to report things that should have matched but didnbt  (for example,
look for "Code of Virginia" that *isn't* within the wrapping <span> elements
my code is adding).
>
> This kind of analysis produces a much more controlled result that would be
amenable to putting into a repeatable unit test.
>
>
> --
> Eliot Kimber
> http://contrext.com
>
>
> o;?On 10/16/18, 4:00 AM, "Tom Cleghorn tom.cleghorn@xxxxxxx"
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>
>    On 15/10/18 20:41, Pieter Masereeuw pieter@xxxxxxxxxxxx wrote:
>> It would be interesting to know if there are readers on this list who
>> actually make use of units test during XSLT development and what their
>> experiences are.
>
>    I've also always found unit testing XSLT a little too painful to
>    actually get on with and *do* - especially when working on pre-existing
>    code.
>
>    However, I've recently been getting promising results by generating
>    XSpec tests from a set of requirement docs filled in by stakeholders -
>    these still need some dev massage before they're ready to produce
>    working scenarios, but in the context of this particular project, it's
>    proving a good way to both discipline the code (not to mention the
>    coder...) and prevent regressions, and provide a window on to the things
>    stakeholders care about - i.e. what goes in and what comes out, rather
>    than what happens on the inside.
>    IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.

Current Thread

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.