[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: Testing XML don't use xUnit
On Thu, Apr 11, 2013 at 11:37 AM, Stephen D Green <stephengreenubl@gmail.com> wrote: > > I also do a lot of work with testers writing tests for HTML pages using the > very well-known integration test tool Selenium IDE (for Firefox) and that > makes heavy use of XPath even though the target is HTML. Here we end > up with many tests which include XPaths which break every time a > significant change is made to the HTML. So we have to keep mending the > XPath expressions (even though we try to write them with //, etc, to avoid > this). So I know the problem. It doesn't stop people favouring Selenium > IDE for website integration tests though. It's well worth the hassle to > many. > I guess it might be different if we were using XPath for unit tests but I > don't know about that. I guess modern unit tests get associated with TDD > and Agile, so maybe having separate test assertions (written in prose) > would be a 'no no' for many unit test writers as it might be seen as going > against Agile mantras. You'd have to update the assertions less frequently > than the unit tests but having to update something other than the tests > and the XML / code would be just too much for many Agile teams. So > if you wanted to inut test your XML you'd probably be stuck with having to > keep revising a load of XPath expressions. It goes with the territory. > > Best regards > > Steve > Very very interesting. Sitting on my hard drive is an advanced draft of an unpublished paper (Declarative Automated XML Testing) and 40 odd pages of slides for an ungiven talk/training course (Schematron for Testers) that I had forgotten I had ever written. Both were prepared in 2 years ago and targeted at the testing community and the Selenium experience was exactly what was at the forefront of my mind. Very enlightening summarisation of the reception I could have expected. Some excerpts from the slides (hence the formatting) Imagine a test tool that afforded you the ability to specify and cross-reference XML/web test cases to requirements as well as automatically check the result of executing them WITHOUT having to program. Imagine a free test tool based on a familiar technology (XML) that obviated the need to learn a proprietary language. Required minimal prerequisites knowledge to use. Did not require familiarisation with a proprietary user interface. Afforded the ability to issue customised user-friendly diagnostics that pinpoint the problem area with exactitude. Afforded test case reuse by allowing you to define templates for commonly occurring patterns that can be parameterised and instantiated into actual test cases. Afforded test case reuse with include mechanisms. If such a tool existed would you be interested in using it ? Such a tool does exist. A Schematron schema is a validation specification that enables natural language assertions to be made and validated against the content of an XML document. It is not designed to be a test tool but can be used as one. No new programming user interface to master. A Schematron schema is plain XML specified using elements and attributes from the Schematron namespace. A Schematron schema is a declarative specification from which an XSLT stylesheet is automatically generated that will navigate an XML instance and check the constraints it specifies. The power of Schematron can also be applied to database/file testing if data is first converted to an XML format. It can be embedded directly into test/functional specifications You don't have to learn how to program to use it. ”Let me start by saying that writing GUI-based tests for Web apps is no fun, no matter what your testing tool is. You need to navigate through pages, fill and submit forms, and verify that certain elements are present on the pages. Doing all this manually can quickly become tedious and kill whatever joy you may find in testing.” Many tools attack this problem with record and playback. One of the many drawbacks of this is that it generates very inefficient, long and brittle XPath because all expressions navigate from the root. This means that tests are vulnerable to structural changes in the XML. Schematron automatically takes care of navigating the XML instance. If you specify test cases on say the account element, it will work out for itself how to locate the relevant nodes in the instance. There is no need for Record and Playback at the test verification stage and test cases are better insulated against changes to the data structures. An extract from the paper Several automated test tools support XPath but using any of these tools entails accidental temporal cohesion that forces business rule testing to be tightly coupled with GUI test. Their ability to auto-generate XPath to retrieve specific content following a capture replay session may seem like a boon but the XPath generated is brittle (and inefficient) because it is coded using absolute paths and thus vulnerable to structural changes in the XML. Non-trivial business tests will inevitably entail comparisons with data in other XML documents or databases so one cannot be entirely reliant on autogenerated XPath. Once learnt, you can convert database or other formats data to XML and thereby gain leverage for your XPath interrogation skills. XPath has a natural habitat - XSLT - and that is the vehicle advocated in this article for housing your XML tests. The reason for this outlandish suggestion is the availability of Schematron, a tool that supports the specification of test cases and will generate the XSLT to validate the test cases for you. Schematron only requires you to familiarise yourself with 4 elements to use it. If you are willing to learn more of the elements you have an even more powerful tool and because it generates XSLT and provides hooks that support customisations the full power of XSLT is an option at your disposal. Let us first clarify that the testing scenario under contemplation is one where the test data has been created (whether by executing a test or derivation) and exists in an XML document waiting to be validated against expected results.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] |
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|