[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Testing XML don't use xUnit

  • From: Michael Sokolov <msokolov@safaribooksonline.com>
  • To: Michael Kay <mike@saxonica.com>
  • Date: Wed, 10 Apr 2013 11:41:01 -0400

Re:  Testing XML don't use xUnit
On 4/10/2013 10:27 AM, Michael Kay wrote:
>>> It sounds like you are suggesting it's bad to have xpaths in junit
>>> tests?  ...and to use Schematron instead?
>>>
>> Yes. Because changes to the structure of the xml will break the JUnit
>> test whereas Schematron assertions are resilient to that.
>>
> I'm completely bemused. I thought Schematron assertions, like XMLUnit assertions, were both XPath. Why is one better than the other?
>
> Unit testing has always been exposed to the problem that when things change, tests break. My response to that has always been to have very little internal testing - the vast majority of tests are "black box" tests working against stable product interfaces rather than against internal components, because internal components change much more than external product interfaces. But XPath has proved a very good language for writing the assertions, and I've always thought that is why both Schematron and XMLUnit use it - as do the latest generation of W3C test suites, with considerable success.
There is an art to writing robust assertions, whatever language or 
framework is used.  We use XPath throughout our testing suites; in 
selenium integration tests that examine HTML markup, and in java unit 
tests that just look at more isolated XSLT-driven processes. The key is 
ensuring that the assertion tests what it is supposed to, and not a 
coincidental feature of the implementation.  That's not always easy, and 
brittle tests aren't helpful.  I often favor looser tests for this 
reason: ie just make sure the identifier appears somewhere in the 
markup, or use a "//" as was alluded to earlier, rather than a full path.

Re: integration vs. unit tests; they both have merits.   It's hard to 
generalize about kinds of tests since software systems vary in which 
parts are stable or unstable. Highly granular focused tests (usually 
called unit tests) are helpful since when they fail they pinpoint the 
cause more specifically.  We often find that our version of black-box 
testing: integration tests, in a web application, test too many 
different things at once though, and in fact are unstable because they 
are sensitive to page layout, which tends to change a lot in 
uninteresting ways.  So it's not just an internal/external dynamic, I 
think, but it's definitely important to test things that are important, 
that are actual intended behaviors.

I think this issue actually relates somewhat to the whole schema 
brouhaha that is going on in a parallel thread, since schemas can be 
thought of as a kind of testing assertion, but I haven't used them 
enough to say anything intelligent about it :)  We tend to just make our 
assertions in custom code (java unit tests).

-Mike


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]


PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
 

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.


Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.